Apparatus and method for adaptive streaming of geometry-based point clouds

Abstract

Systems, methods, and instrumentalities are disclosed for receiving content related to geometry-based point clouds. A client device can receive a media presentation description (MPD) file from a content server. A preselection element set can be identified from the MPD file. One or more adaptation sets associated with at least one preselection element of the preselection element set can be identified. The adaptation set can be indicated by an attribute associated with one of the preselection elements. A geometry-based point cloud compression (GPCC) tile identifier associated with a viewport can be determined. The GPCC tile identifier can be determined based on a first descriptor received in the MPD file. One or more adaptation sets associated with the GPCC tile identifier can be selected using a second descriptor. Point cloud components associated with the selected one or more adaptation sets can be requested. The point cloud components can be received.

Description

[0001] Cross-references to related applications This application claims the benefit of U.S. Provisional Patent Application No. 63 / 042,481, filed June 22, 2020, and U.S. Provisional Patent Application No. 63 / 084,758, filed September 29, 2020, the disclosures of which are incorporated herein by reference in their entirety. Background Technology

[0002] Video coding systems can be used to compress digital video signals, for example, to reduce the storage and / or transmission bandwidth required for such signals. Video coding systems can include, for example, wavelet-based systems, object-based systems, and / or block-based systems (such as hybrid block-based video coding systems). Video coding systems can support both the encoding and storage aspects of point clouds. However, the system may lack appropriate mechanisms to support the streaming of point cloud data over a network. Summary of the Invention

[0003] Systems, methods, and means for adaptive streaming of visual media content, such as geometry-based point clouds, are disclosed. Elements, attributes, and metadata associated with point cloud components can be signaled, for example, to enable a streaming client to identify point cloud streams and their component substreams within a Media Presentation Descriptor (MPD), and to enable the streaming client to select versions of the point cloud and / or its components, for example, based on client support. In examples, the streaming client may utilize guidance (e.g., an instruction signaled in the MPD file) to determine different representations of the point cloud content. For example, the instruction may indicate which set of representations across different point cloud components constitutes a specific quality level. Components of the point cloud content may be divided into multiple tiles or tile portions. The client may, for example, stream specific tile portions of a geometry component (e.g., selected tile portions) based on bandwidth availability (e.g., instead of streaming all point cloud data). Point cloud component tile bitstreams may be available at different adaptive sets, for example, where each adaptive set (e.g., each adaptive set) can represent a point cloud component tile.

[0004] Geometric-based point cloud compression (G-PCC) components can be signaled in Dynamic Streaming over HTTP (DASH). For example, a DASH manifest file or MPD file can be used to signal G-PCC components. In the example, G-PCC components (e.g., each G-PCC component) can be represented as adaptive sets (e.g., independent adaptive sets) in the DASH MPD file. An adaptive set (e.g., a master adaptive set) can act as the primary access point for G-PCC content. In the example, adaptive sets (e.g., a single adaptive set) can be signaled per component at different resolutions.

[0005] For example, the G-PCC component descriptor can be signaled to enable the streaming client to identify the type of point cloud components in the adaptive set and / or representation. The G-PCC descriptor allows the streaming client to distinguish between different point cloud streams existing in the MPD file. The streaming client can then identify the component stream for the corresponding point cloud stream.

[0006] For example, a list of identifiers (IDs) can be used to signal G-PCC preselection (e.g., in MPD), which includes, for example, the master adaptive set ID for volumetric media and the adaptive set ID corresponding to the G-PCC component. Preselection can be signaled, for example, using a preselection element within a period element, and / or using a preselection descriptor at the adaptive set level.

[0007] Multiple versions of G-PCC media can be signaled. For example, independent pre-selections can be used to signal multiple versions of the same point cloud media. Pre-selections representing alternative versions of the same geometry-based point cloud media can include, for example, G-PCC descriptors with the same attribute values.

[0008] One or more G-PCC tiles can be signaled. Tile bounding box information can be signaled, for example, in the case of multiple tiles existing in a geometry-based point cloud. The client can select a tile ID from tile inventory bounding box information (e.g., in MPD), for example, to stream tile-based G-PCC component data.

[0009] The client can identify the tile ID for a point cloud component in an adaptive set, for example, by examining the G-PCC component descriptor. The G-PCC tile ID descriptor can be signaled, for example, to enable the streaming client to distinguish G-PCC tile streams.

[0010] The characteristics of a spatial region and / or the mapping between the region and G-PCC tiles can be signaled, for example, when the 3D spatial region in geometry-based volumetric media content is static. The characteristics of a spatial region and / or the mapping between the region and the corresponding adaptive set of G-PCC components can be signaled (e.g., using a G-PCC 3D region descriptor), for example, when the 3D spatial region is static and / or tile inventory information is unavailable. The mapping between a spatial region and the corresponding adaptive set of G-PCC components can be signaled (e.g., via a G-PCC region ID descriptor or a G-PCC component descriptor).

[0011] Timing metadata tracks (e.g., indicating the position and / or size of a 3D region on the display timeline) can be signaled, for example, in an adaptive set with representation (e.g., for dynamic spatial regions) and associated with the main G-PCC adaptive set.

[0012] Streaming client behavior can be based on signaling. For example, a DASH client can be guided using information provided in the MPD (Multi-Level Design).

[0013] Systems, methods, and means for receiving content related to geometry-based point clouds are disclosed. In examples, a Media Presentation Description (MPD) file may be received, for instance, from a content server. A pre-selected set of elements may be identified from the MPD file. One or more adaptive sets associated with at least one pre-selected element in the pre-selected set may be identified. For example, the one or more adaptive sets may be indicated by an attribute associated with one of the pre-selected elements.

[0014] A geometry-based point cloud compression (GPCC) tile identifier associated with the viewport can be determined. For example, the GPCC tile identifier can be determined based on a first descriptor received in the MPD file. In the example, the first descriptor can be a three-dimensional (3D) region descriptor. The 3D region descriptor may include the region location, one or more region sizes, and / or the set of tiles associated with the 3D region.

[0015] A second descriptor can be used to select one or more adaptive sets associated with a GPCC tile identifier. In the example, the second descriptor can be a component descriptor. This component descriptor may include a component type, attribute type, index, and / or a tile set associated with a bitstream. A point cloud component associated with the selected one or more adaptive sets can be requested. In the example, this point cloud component can be received.

[0016] Each feature disclosed anywhere in this document is described, and such feature may be implemented separately / individually and in any combination with any other feature disclosed herein and / or with any feature disclosed elsewhere, whether implicitly or explicitly mentioned herein or otherwise falling within the scope of the subject matter disclosed herein. Attached Figure Description

[0017] Figure 1A This is a system diagram illustrating an exemplary communication system that can be implemented in one or more of the disclosed embodiments.

[0018] Figure 1B It is shown in the implementation plan. Figure 1A A system diagram of an exemplary wireless transmit / receive unit (WTRU) used within the communication system shown.

[0019] Figure 1C It is shown in the implementation plan. Figure 1A The diagram shows an exemplary radio access network (RAN) and an exemplary core network (CN) used within the communication system.

[0020] Figure 1D It is shown in the implementation plan. Figure 1A The system diagram shown illustrates another exemplary RAN and another exemplary CN used within the communication system.

[0021] Figure 2 This is a schematic diagram illustrating an exemplary video encoder.

[0022] Figure 3 This is a schematic diagram illustrating an example of a video decoder.

[0023] Figure 4 This is a schematic diagram illustrating an example of a system in which various aspects and examples can be implemented.

[0024] Figure 5 An example of a bitstream structure for geometry-based point cloud compression (G-PCC) is shown.

[0025] Figure 6 An example of a sample structure is shown when the G-PCC geometry and attribute bitstream is stored in a single track.

[0026] Figure 7 An example of a multi-track G-PCC container is shown.

[0027] Figure 8 An exemplary Media Presentation Description (MPD) hierarchical data model is shown.

[0028] Figure 9 An example of grouping G-PCC components in MPD using preselection is shown.

[0029] Figure 10 An example is shown of grouping multiple versions of the G-PCC component in MPD using pre-selection.

[0030] Figure 11 An example of G-PCC content with multiple tile tracks is shown. Detailed Implementation

[0031] A detailed description of exemplary embodiments will now be described with reference to the various accompanying drawings. Although this specification provides detailed examples of possible specific implementations, it should be noted that the details are intended to be exemplary and in no way limit the scope of this application.

[0032] Figure 1AThis is a schematic diagram illustrating an exemplary communication system 100 that can be implemented in one or more of the disclosed embodiments. Communication system 100 can be a multiple access system providing content such as voice, data, video, messaging, and broadcasting to multiple wireless users. Communication system 100 enables multiple wireless users to access such content through the sharing of system resources, including wireless bandwidth. For example, communication system 100 can employ one or more channel access methods, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal FDMA (OFDMA), Single Carrier FDMA (SC-FDMA), Zero-Tail Unique Word DFT Extended OFDM (ZT UW DTS-s OFDM), Unique Word OFDM (UW-OFDM), Resource Block Filtered OFDM, Filter Bank Multicarrier (FBMC), etc.

[0033] like Figure 1A As shown, the communication system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, RAN 104 / 113, CN 106 / 115, Public Switched Telephone Network (PSTN) 108, Internet 110, and other networks 112. However, it should be understood that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Each of the WTRUs 102a, 102b, 102c, and 102d may be any type of device configured to operate and / or communicate in a wireless environment. As examples, WTRUs 102a, 102b, 102c, and 102d (any of which may be referred to as a “station” and / or “STA”) may be configured to transmit and / or receive wireless signals and may include user equipment (UE), mobile stations, fixed or mobile user units, subscription-based units, pagers, cellular phones, personal digital assistants (PDAs), smartphones, laptops, netbooks, personal computers, wireless sensors, hotspots or Mi-Fi devices, Internet of Things (IoT) devices, watches or other wearable devices, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in industrial and / or automated processing chain environments), consumer electronics devices, devices operating on commercial and / or industrial wireless networks, etc. Any of UEs 102a, 102b, 102c, and 102d may be interchangeably referred to as WTRUs.

[0034] The communication system 100 may also include base station 114a and / or base station 114b. Each of base stations 114a and 114b may be any type of device configured to wirelessly interface with at least one of WTRUs 102a, 102b, 102c, and 102d to facilitate access to one or more communication networks, such as CN 106 / 115, Internet 110, and / or other networks 112. As an example, base stations 114a and 114b may be base transceiver stations (BTS), Node Bs, evolved Node Bs, home Node Bs, home evolved Node Bs, gNBs, NR Node Bs, site controllers, access points (APs), wireless routers, etc. Although base stations 114a and 114b are each depicted as a single element, it should be understood that base stations 114a and 114b may include any number of interconnected base stations and / or network elements.

[0035] Base station 114a may be part of RAN 104 / 113, which may also include other base stations and / or network elements (not shown), such as base station controllers (BSCs), radio network controllers (RNCs), relay nodes, etc. Base station 114a and / or base station 114b may be configured to transmit and / or receive radio signals on one or more carrier frequencies (which may be referred to as cells (not shown)). These frequencies may be in licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide coverage of radio services to a specific geographic area, which may be relatively fixed or changeable over time. A cell may be further divided into cell sectors. For example, the cell associated with base station 114a may be divided into three sectors. Therefore, in an embodiment, base station 114a may include three transceivers, i.e., one transceiver for each sector of the cell. In an embodiment, base station 114a may employ multiple-input multiple-output (MIMO) technology and may utilize multiple transceivers for each sector of the cell. For example, beamforming may be used to transmit and / or receive signals in desired spatial directions.

[0036] Base stations 114a and 114b can communicate with one or more of WTRUs 102a, 102b, 102c, and 102d via air interface 116, which can be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). Any suitable radio access technology (RAT) can be used to establish air interface 116.

[0037] More specifically, as noted above, the communication system 100 may be a multiple access system and may employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, base stations 114a and WTRUs 102a, 102b, and 102c in RAN 104 / 113 may implement radio technologies such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may use Wideband CDMA (WCDMA) to establish air interfaces 115 / 116 / 117. WCDMA may include communication protocols such as High-Speed ​​Packet Access (HSPA) and / or evolved HSPA (HSPA+). HSPA may include High-Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High-Speed ​​UL Packet Access (HSUPA).

[0038] In the implementation scheme, base station 114a and WTRUs 102a, 102b, 102c can implement radio technologies such as evolved UMTS terrestrial radio access (E-UTRA), which can use Long Term Evolution (LTE) and / or Advanced LTE (LTE-A) and / or Advanced LTE Pro (LTE-A Pro) to establish air interface 116.

[0039] In the implementation scheme, base station 114a and WTRUs 102a, 102b, 102c enable radio technology such as NR radio access, which can use New Radio (NR) to establish air interface 116.

[0040] In the implementation scheme, base station 114a and WTRUs 102a, 102b, and 102c can implement various radio access technologies. For example, base station 114a and WTRUs 102a, 102b, and 102c can, for example, use a dual connectivity (DC) principle to implement both LTE and NR radio access together. Therefore, the air interface utilized by WTRUs 102a, 102b, and 102c can be characterized by various types of radio access technologies and / or transmissions sent to / from various types of base stations (e.g., eNBs and gNBs).

[0041] In other implementations, base station 114a and WTRUs 102a, 102b, and 102c can implement radio technologies such as IEEE 802.11 (i.e., WiFi), IEEE 802.16 (i.e., WiMAX), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Provisional Standard 2000 (IS-2000), Provisional Standard 95 (IS-95), Provisional Standard 856 (IS-856), Global System for Mobile Communications (GSM), GSM Enhanced Data Rate Evolution (EDGE), and GSM EDGE (GERAN).

[0042] Figure 1A Base station 114b can be, for example, a wireless router, a home node B, a home evolution node B, or an access point, and can utilize any suitable RAT to facilitate wireless connectivity in localized areas such as commercial locations, homes, vehicles, campuses, industrial facilities, air corridors (e.g., for use by drones), roads, etc. In one embodiment, base station 114b and WTRUs 102c, 102d can implement radio technologies such as IEEE 802.11 to establish a wireless local area network (WLAN). In another embodiment, base station 114b and WTRUs 102c, 102d can implement radio technologies such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, base station 114b and WTRUs 102c, 102d can utilize cellular-based RATs (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.) to establish picocells or femtocells. Figure 1A As shown, base station 114b may have a direct connection to Internet 110. Therefore, base station 114b may not need to access Internet 110 via CN 106 / 115.

[0043] RAN 104 / 113 can communicate with CN 106 / 115, which can be any type of network configured to provide voice, data, application, and / or Voice over Internet Protocol (VoIP) services to one or more of WTRU 102a, 102b, 102c, and 102d. Data can have different Quality of Service (QoS) requirements, such as different throughput requirements, latency requirements, error tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. CN 106 / 115 can provide call control, billing services, location-based services, prepaid calling, internet connectivity, video distribution, etc., and / or perform advanced security functions such as user authentication. Although not explicitly stated... Figure 1AAs shown, but it should be understood that RAN 104 / 113 and / or CN 106 / 115 can communicate directly or indirectly with other RANs that use the same RAT as RAN 104 / 113 or a different RAT. For example, in addition to being connected to RAN 104 / 113 which can utilize NR radio technology, CN 106 / 115 can also communicate with another RAN (not shown) that uses GSM, UMTS, CDMA2000, WiMAX, E-UTRA or WiFi radio technology.

[0044] CN 106 / 115 may also act as a gateway for WTRU 102a, 102b, 102c, 102d to access PSTN 108, Internet 110, and / or other networks 112. PSTN 108 may include a circuit-switched telephone network providing Common Old-Style Telephone Service (POTS). Internet 110 may include a global system of interconnected computer networks and devices using common communication protocols such as Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and / or Internet Protocol (IP) from the TCP / IP Internet Protocol suite. Network 112 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, network 112 may include another CN connected to one or more RANs, which may use the same RAT as RAN 104 / 113 or a different RAT.

[0045] Some or all of the WTRUs 102a, 102b, 102c, and 102d in the communication system 100 may include multi-mode capabilities (e.g., WTRUs 102a, 102b, 102c, and 102d may include multiple transceivers for communicating with different wireless networks via different wireless links). For example, Figure 1A The WTRU 102c shown can be configured to communicate with a base station 114a that can employ cellular-based radio technology and with a base station 114b that can employ IEEE 802 radio technology.

[0046] Figure 1B This is a system diagram illustrating an exemplary WTRU 102. (See diagram below.) Figure 1B As shown, WTRU 102 may include a processor 118, a transceiver 120, a transmitting / receiving element 122, a speaker / microphone 124, a keypad 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power supply 134, a Global Positioning System (GPS) chipset 136, and / or other peripheral devices 138, etc. It should be understood that, while remaining consistent with the implementation, WTRU 102 may include any sub-combination of the foregoing elements.

[0047] Processor 118 can be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. Processor 118 can perform signal encoding, data processing, power control, input / output processing, and / or any other functions that enable WTRU 102 to operate in a wireless environment. Processor 118 can be coupled to transceiver 120, which can be coupled to transmitting / receiving element 122. Although Figure 1B The processor 118 and transceiver 120 are depicted as separate components, but it should be understood that the processor 118 and transceiver 120 may be integrated together in an electronic package or chip.

[0048] Transmitting / receiving element 122 may be configured to transmit signals to or receive signals from a base station (e.g., base station 114a) via air interface 116. For example, in one embodiment, transmitting / receiving element 122 may be an antenna configured to transmit and / or receive RF signals. In one embodiment, transmitting / receiving element 122 may be a transmitter / detector configured to transmit and / or receive, for example, IR, UV, or visible light signals. In yet another embodiment, transmitting / receiving element 122 may be configured to transmit and / or receive both RF and optical signals. It should be understood that transmitting / receiving element 122 may be configured to transmit and / or receive any combination of wireless signals.

[0049] Although the transmitting / receiving element 122 is in Figure 1B While depicted as a single element, WTRU 102 may include any number of transmitting / receiving elements 122. More specifically, WTRU 102 may employ MIMO technology. Therefore, in one embodiment, WTRU 102 may include two or more transmitting / receiving elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals via air interface 116.

[0050] Transceiver 120 can be configured to modulate signals transmitted by transmitting / receiving element 122 and demodulate signals received by transmitting / receiving element 122. As noted above, WTRU 102 may have multi-mode capability. For example, transceiver 120 may therefore include multiple transceivers to enable WTRU 102 to communicate via various RATs such as NR and IEEE 802.11.

[0051] The processor 118 of WTRU 102 may be coupled to a speaker / microphone 124, a keypad 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) unit or an organic light-emitting diode (OLED) display unit) and may receive user input data therefrom. The processor 118 may also output user data to the speaker / microphone 124, keypad 126, and / or display / touchpad 128. Furthermore, the processor 118 may access information from any type of suitable memory (such as non-removable memory 130 and / or removable memory 132) and store data in any type of suitable memory. Non-removable memory 130 may include random access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. Removable memory 132 may include a user identity module (SIM) card, memory stick, secure digital storage (SD) card, etc. In other embodiments, the processor 118 may access information from memory that is not physically located on WTRU 102 (such as on a server or home computer (not shown)) and store data in that memory.

[0052] The processor 118 may receive power from the power supply 134 and may be configured to distribute and / or control power to other components in the WTRU 102. The power supply 134 may be any suitable device for powering the WTRU 102. For example, the power supply 134 may include one or more dry cell battery packs (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0053] The processor 118 may also be coupled to a GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) about the current location of the WTRU 102. In addition to or instead of the information from the GPS chipset 136, the WTRU 102 may receive location information from base stations (e.g., base stations 114a, 114b) via air interface 116 and / or determine its location based on the timing of signals received from two or more nearby base stations. It should be understood that, while remaining consistent with the implementation, the WTRU 102 may acquire location information using any suitable location determination method.

[0054] The processor 118 may also be coupled to other peripheral devices 138, which may include one or more software and / or hardware modules that provide additional features, functions, and / or wired or wireless connectivity. For example, peripheral device 138 may include an accelerometer, electronic compass, satellite transceiver, digital camera (for photos and / or video), Universal Serial Bus (USB) port, vibration device, television transceiver, hands-free headset, Bluetooth.® Modules, FM radio units, digital music players, media players, video game player modules, internet browsers, virtual reality and / or augmented reality (VR / AR) devices, activity trackers, etc. Peripheral devices 138 may include one or more sensors, which may be one or more of the following: gyroscopes, accelerometers, Hall effect sensors, magnetometers, orientation sensors, proximity sensors, temperature sensors, time sensors; geolocation sensors; altimeters, light sensors, touch sensors, magnetometers, barometers, gesture sensors, biometric sensors, and / or humidity sensors.

[0055] WTRU 102 may include a full-duplex radio for which the transmission and reception of some or all signals (e.g., associated with specific subframes for UL (e.g., for transmission) and downlink (e.g., for reception)) may be concurrent and / or simultaneous. The full-duplex radio may include an interference management unit for reducing and / or substantially eliminating self-interference through signal processing via hardware (e.g., a choke) or via a processor (e.g., a separate processor (not shown) or via processor 118). In one embodiment, WTRU 102 may include a half-duplex radio for which the transmission and reception of some or all signals (e.g., associated with specific subframes for UL (e.g., for transmission) or downlink (e.g., for reception)) may be concurrent and / or simultaneous.

[0056] Figure 1C This is a system diagram illustrating RAN 104 and CN 106 according to one embodiment. As described above, RAN 104 can communicate with WTRUs 102a, 102b, and 102c via air interface 116 using E-UTRA radio technology. RAN 104 can also communicate with CN 106.

[0057] RAN 104 may include evolved Node Bs 160a, 160b, and 160c; however, it should be understood that RAN 104 may include any number of evolved Node Bs while remaining consistent with the implementation scheme. Each evolved Node B 160a, 160b, and 160c may include one or more transceivers for communicating with WTRUs 102a, 102b, and 102c via air interface 116. In the implementation scheme, evolved Node Bs 160a, 160b, and 160c may implement MIMO technology. Therefore, evolved Node B 160a may, for example, use multiple antennas to transmit radio signals to and / or receive radio signals from WTRU 102a.

[0058] Each of the evolved nodes B 160a, 160b, and 160c can be associated with a specific cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, and user scheduling in the UL and / or DL, etc. Figure 1C As shown, evolution nodes B 160a, 160b, and 160c can communicate with each other via the X2 interface.

[0059] Figure 1C The CN 106 shown may include a Mobility Management Entity (MME) 162, a Serving Gateway (SGW) 164, and a Packet Data Network (PDN) Gateway (or PGW) 166. While each of the foregoing elements is depicted as part of the CN 106, it should be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0060] The MME 162 can connect to each of the evolved nodes B 162a, 162b, and 162c in RAN 104 via the S1 interface and can be used as a control node. For example, the MME 162 can be responsible for authenticating users of WTRUs 102a, 102b, and 102c, bearer activation / deactivation, selecting a specific serving gateway during the initial attachment of WTRUs 102a, 102b, and 102c, etc. The MME 162 can provide control plane functions for handover between RAN 104 and other RANs (not shown) employing other radio technologies such as GSM and / or WCDMA.

[0061] The SGW 164 can connect to each of the evolved Nodes B 160a, 160b, and 160c in RAN 104 via the S1 interface. The SGW 164 typically routes and forwards user data packets to and from WTRUs 102a, 102b, and 102c. The SGW 164 can perform other functions such as anchoring the user plane during inter-evolved Node B handovers, triggering paging when DL data is available for WTRUs 102a, 102b, and 102c, and managing and storing the context of WTRUs 102a, 102b, and 102c.

[0062] SGW 164 can be connected to PGW 166, which provides WTRU 102a, 102b, 102c with access to packet-switched networks (such as Internet 110) to facilitate communication between WTRU 102a, 102b, 102c and IP-enabled devices.

[0063] CN 106 can facilitate communication with other networks. For example, CN 106 can provide WTRUs 102a, 102b, and 102c with access to circuit-switched networks (such as PSTN 108) to facilitate communication between WTRUs 102a, 102b, and 102c and conventional landline communication equipment. For example, CN 106 may include, or communicate with, an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) serving as an interface between CN 106 and PSTN 108. Additionally, CN 106 can provide WTRUs 102a, 102b, and 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0064] Despite WTRU in Figures 1A to 1D While described as a wireless terminal, it is conceivable that in some representative implementations, such a terminal may (e.g., temporarily or permanently) use a wired communication interface with a communication network.

[0065] In a representative implementation, the other network 112 may be a WLAN.

[0066] A WLAN in Infrastructure Basic Services Set (BSS) mode may have an access point (AP) for the BSS and one or more sites (STAs) associated with the AP. The AP may have access or an interface to a distribution system (DS) or another type of wired / wireless network that carries traffic to and / or carries traffic out of the BSS. Traffic originating outside the BSS and destined for a STA can reach and be delivered to the STA via the AP. Traffic originating from a STA and destined for a destination outside the BSS can be sent to the AP for delivery to the appropriate destination. Traffic between STAs within the BSS can be sent via the AP, for example, where a source STA can send traffic to the AP, and the AP can deliver the traffic to the destination STA. Traffic between STAs within the BSS can be considered and / or referred to as point-to-point traffic. Point-to-point traffic can be sent between source and destination STAs (e.g., directly between them) using Direct Link Establishment (DLS). In some representative implementations, the DLS may use 802.11e DLS or 802.11z Tunneled DLS (TDLS). WLANs using Standalone BSS (IBSS) mode may not have access points (APs), and STAs within the IBSS or using the IBSS (e.g., all STAs) can communicate directly with each other. IBSS communication mode may sometimes be referred to as "ad-hoc" communication mode in this document.

[0067] When operating in 802.11ac infrastructure mode or a similar mode, the AP can transmit beacons on a fixed channel, such as the primary channel. The primary channel can be of fixed width (e.g., a 20 MHz wide bandwidth) or dynamically set via signaling. The primary channel can be the operating channel of the BSS and can be used by the STA to establish a connection with the AP. In some representative implementations, such as in an 802.11 system, Carrier Sense Multiple Access / Collision Avoidance (CSMA / CA) can be implemented. For CSMA / CA, each STA (including the AP) can listen to the primary channel. If the primary channel is listened to / detected and / or determined to be busy by a particular STA, that STA can back off. A single STA (e.g., only one station) can transmit at any given time within a given BSS.

[0068] High-throughput (HT) STAs can communicate using a 40MHz wide channel, for example, by combining a primary 20MHz channel with adjacent or non-adjacent 20MHz channels to form a 40MHz wide channel.

[0069] The Very High Throughput (VHT) STA supports channels with widths of 20MHz, 40MHz, 80MHz, and / or 160MHz. 40MHz and / or 80MHz channels can be formed by combining consecutive 20MHz channels. A 160MHz channel can be formed by combining eight consecutive 20MHz channels, or by combining two non-consecutive 80MHz channels (this can be referred to as an 80+80 configuration). For the 80+80 configuration, after channel coding, data can be processed by a segment parser that can split the data into two streams. Inverse Fast Fourier Transform (IFFT) processing and time-domain processing can be performed separately on each stream. These streams can be mapped to two 80MHz channels, and data can be transmitted via the transmitting STA. At the receiver of the receiving STA, the operations described above for the 80+80 configuration can be reversed, and the combined data can be sent to Media Access Control (MAC).

[0070] 802.11af and 802.11ah support operating modes below 1 GHz. Compared to those used in 802.11n and 802.11ac, 802.11af and 802.11ah reduce channel operating bandwidth and carrier. 802.11af supports 5 MHz, 10 MHz, and 20 MHz bandwidths in the TV white space (TVWS) spectrum, while 802.11ah supports 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz bandwidths using non-TVWS spectrum. According to representative implementations, 802.11ah may support instrument-type control / machine-type communications, such as MTC devices in macro coverage areas. MTC devices may have certain capabilities, such as limited capabilities, including support (e.g., only support) certain bandwidths and / or limited bandwidths. MTC devices may include batteries with battery life above a threshold (e.g., to maintain a very long battery life).

[0071] WLAN systems supporting multiple channels, as well as channel bandwidths such as 802.11n, 802.11ac, 802.11af, and 802.11ah, include channels that can be designated as primary channels. A primary channel can have a bandwidth equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel can be set and / or limited by STAs operating in the BSS (each supporting a minimum bandwidth operating mode). In the 802.11ah example, for STAs supporting (e.g., only supporting) a 1MHz mode (e.g., MTC-type devices), the primary channel can be 1MHz wide, even if the AP and other STAs in the BSS support 2MHz, 4MHz, 8MHz, 16MHz, and / or other channel bandwidth operating modes. Carrier Sense and / or Network Allocation Vector (NAV) settings can depend on the status of the primary channel. If the primary channel is busy, for example, because an STA (supporting only the 1MHz operating mode) is transmitting to the AP, the entire available band can be considered busy even if most of the band remains idle and potentially available.

[0072] In the United States, the available frequency bands for 802.11ah are 902MHz to 928MHz. In South Korea, the available frequency bands are 917.5MHz to 923.5MHz. In Japan, the available frequency bands are 916.5MHz to 927.5MHz. The total available bandwidth for 802.11ah ranges from 6MHz to 26MHz, depending on the country code.

[0073] Figure 1D This is a system diagram illustrating RAN 113 and CN 115 according to one implementation scheme. As noted above, RAN 113 can communicate with WTRUs 102a, 102b, and 102c via air interface 116 using NR radio technology. RAN 113 can also communicate with CN 115.

[0074] RAN 113 may include gNBs 180a, 180b, and 180c; however, it should be understood that RAN 113 may include any number of gNBs while remaining consistent with the implementation scheme. Each of gNBs 180a, 180b, and 180c may include one or more transceivers for communication with WTRUs 102a, 102b, and 102c via air interface 116. In the implementation scheme, gNBs 180a, 180b, and 180c may implement MIMO technology. For example, gNBs 180a and 180b may utilize beamforming to transmit signals to and / or receive signals from gNBs 180a, 180b, and 180c. Therefore, gNB 180a may, for example, use multiple antennas to transmit radio signals to and / or receive radio signals from WTRU 102a. In the implementation scheme, gNBs 180a, 180b, and 180c can implement carrier aggregation technology. For example, gNB 180a can transmit multiple component carriers to WTRU 102a (not shown). A subset of these component carriers may be on unlicensed spectrum, while the remaining component carriers may be on licensed spectrum. In the implementation scheme, gNBs 180a, 180b, and 180c can implement Cooperative Multipoint (CoMP) technology. For example, WTRU 102a can receive cooperative transmissions from gNBs 180a and 180b (and / or gNB 180c).

[0075] WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c using transmissions associated with an scalable set of parameters. For example, OFDM symbol spacing and / or OFDM subcarrier spacing can vary depending on different transmissions, different cells, and / or different portions of the radio transmission spectrum. WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c using subframes or transmission time intervals (TTIs) of various or scalable lengths (e.g., containing different numbers of OFDM symbols and / or continuously varying absolute time lengths).

[0076] gNBs 180a, 180b, and 180c can be configured to communicate with WTRUs 102a, 102b, and 102c in standalone and / or non-standalone configurations. In standalone configuration, WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c without accessing other RANs (e.g., evolved Node Bs 160a, 160b, and 160c). In standalone configuration, WTRUs 102a, 102b, and 102c can use one or more of gNBs 180a, 180b, and 180c as mobility anchors. In standalone configuration, WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c using signals in unlicensed frequency bands. In a non-standalone configuration, WTRUs 102a, 102b, and 102c can communicate or connect to gNBs 180a, 180b, and 180c, and also communicate or connect to other RANs (such as evolved Node Bs 160a, 160b, and 160c). For example, WTRUs 102a, 102b, and 102c can implement DC principles to communicate substantially simultaneously with one or more gNBs 180a, 180b, and 180c and one or more evolved Node Bs 160a, 160b, and 160c. In a non-standalone configuration, evolved Node Bs 160a, 160b, and 160c can be used as mobility anchors for WTRUs 102a, 102b, and 102c, and gNBs 180a, 180b, and 180c can provide additional coverage and / or throughput for serving WTRUs 102a, 102b, and 102c.

[0077] Each of gNBs 180a, 180b, and 180c can be associated with a specific cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, user scheduling in the UL and / or DL, network slicing support, dual connectivity, interoperability between NR and E-UTRA, routing of user plane data to User Plane Functions (UPF) 184a and 184b, routing of control plane information to Access and Mobility Management Functions (AMF) 182a and 182b, etc. Figure 1D As shown, gNB180a, 180b, and 180c can communicate with each other via the Xn interface.

[0078] Figure 1DThe CN 115 shown may include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one Session Management Function (SMF) 183a, 183b, and possibly a Data Network (DN) 185a, 185b. Although each of the foregoing elements is depicted as part of the CN 115, it should be understood that any of these elements may be owned and / or operated by an entity other than a CN operator.

[0079] AMF 182a and 182b can connect to one or more of gNB 180a, 180b, and 180c via the N2 interface in RAN 113 and can be used as control nodes. For example, AMF 182a and 182b can be responsible for authenticating users of WTRU 102a, 102b, and 102c, supporting network slicing (e.g., handling different PDU sessions with different requirements), selecting specific SMF 183a and 183b, managing registration areas, terminating NAS signaling, mobility management, etc. AMF 182a and 182b can use network slicing to customize CN support for WTRU 102a, 102b, and 102c based on the type of service used by WTRU 102a, 102b, and 102c. For example, different network slices can be established for different use cases, such as services that rely on Ultra-Reliable Low Latency (URLLC) access, services that rely on Enhanced Mobile Broadband (eMBB) access, services that rely on Machine Type Communication (MTC) access, etc. The AMF162 can provide control plane functions for switching between RAN 113 and other RANs (not shown) that employ other radio technologies, such as LTE, LTE-A, LTE-A Pro, and / or non-3GPP access technologies (such as WiFi)).

[0080] SMFs 183a and 183b can connect to AMFs 182a and 182b in CN 115 via the N11 interface. SMFs 183a and 183b can also connect to UPFs 184a and 184b in CN 115 via the N4 interface. SMFs 183a and 183b can select and control UPFs 184a and 184b, and configure traffic routing through UPFs 184a and 184b. SMFs 183a and 183b can perform other functions, such as managing and allocating UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, and providing downlink data notifications. PDU session types can be IP-based, non-IP-based, Ethernet-based, etc.

[0081] UPF 184a and 184b can be connected via the N3 interface to one or more of the gNBs 180a, 180b, and 180c in RAN 113. These gNBs can provide WTRU 102a, 102b, and 102c with access to packet-switched networks (such as Internet 110) to facilitate communication between WTRU 102a, 102b, and 102c and IP-enabled devices. UPF 184 and 184b can perform other functions such as routing and forwarding packets, enforcing user plane policies, supporting multihomed PDU sessions, handling user plane QoS, buffering downlink packets, and providing mobility anchoring.

[0082] CN 115 may facilitate communication with other networks. For example, CN 115 may include, or be able to communicate with, an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) serving as an interface between CN 115 and PSTN 108. Additionally, CN 115 may provide WTRUs 102a, 102b, and 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers. In one embodiment, WTRUs 102a, 102b, and 102c may be connected to DNs 185a and 185b via UPFs 184a and 184b through an N3 interface to UPFs 184a and 184b and an N6 interface between UPFs 184a and 184b and local data networks (DNs) 185a and 185b.

[0083] Given Figures 1A to 1D as well as Figures 1A to 1D The corresponding descriptions herein refer to one or more of the functions described below, which may be performed by one or more emulation devices (not shown): WTRU102a-d, base station 114a-b, evolved Node B 160a-c, MME 162, SGW 164, PGW 166, gNB 180a-c, AMF182a-b, UPF 184a-b, SMF 183a-b, DN 185a-b, and / or any other device described herein. An emulation device may be one or more devices configured to mimic one or more of the functions described herein. For example, an emulation device may be used to test other devices and / or simulate network and / or WTRU functions.

[0084] Simulation devices can be designed to perform one or more tests on other devices in laboratory and / or carrier network environments. For example, the one or more simulation devices may perform one or more or all functions while being fully or partially implemented and / or deployed as part of a wired and / or wireless communication network to test other devices within the communication network. The one or more simulation devices may perform one or more or all functions while being temporarily implemented / deployed as part of a wired and / or wireless communication network. Simulation devices may be directly coupled to another device for testing purposes and / or may use over-the-air wireless communication to perform tests.

[0085] The one or more simulation devices may perform one or more (including all) functions without being implemented / deployed as part of a wired and / or wireless communication network. For example, the simulation devices may be used in test scenarios within a test laboratory and / or non-deployed (e.g., testing) wired and / or wireless communication networks to perform testing of one or more components. The one or more simulation devices may be test equipment. Direct RF coupling and / or wireless communication via an RF circuit system (e.g., which may include one or more antennas) may be used by the simulation devices to transmit and / or receive data.

[0086] This application describes multiple aspects, including tools, features, examples or implementations, models, methods, etc. Many of these aspects are described in a particular manner and, at least to illustrate individual characteristics, are generally described in a way that may sound restrictive. However, this is for clarity and does not limit the application or scope of these aspects. In fact, all the different aspects can be combined and interchanged to provide further aspects. Furthermore, these aspects can also be combined and interchanged with aspects described in earlier filings.

[0087] The aspects described and envisioned in this application can be implemented in many different forms. Figures 5 to 8 Some implementation schemes are available, but other implementation schemes are also envisioned. Figures 5 to 8 The discussion does not limit the breadth of specific implementations. At least one of these aspects generally relates to video encoding and decoding, and at least one other aspect generally relates to transmitting a bitstream generated or encoded. These and other aspects can be implemented as methods, apparatus, computer-readable storage media having instructions stored thereon for encoding or decoding video data according to any of the methods, and / or computer-readable storage media having a bitstream generated according to any of the methods stored thereon.

[0088] In this application, the terms “reconstruction” and “decoding” are used interchangeably, the terms “pixel” and “sample” are used interchangeably, and the terms “image”, “picture” and “frame” are used interchangeably.

[0089] This document describes various methods, and each method includes one or more steps or actions for implementing the method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions may be modified or combined. Furthermore, terms such as "first," "second," etc., are used in various embodiments to modify elements, components, steps, operations, etc., such as "first decoding" and "second decoding." Unless specifically required, the use of such terms does not imply a sequence of modified operations. Therefore, in this example, the first decoding does not need to be performed before the second decoding and may occur, for example, before, during, or in overlapping time periods of the second decoding.

[0090] The various methods and other aspects described in this application may (e.g., be used to) modify modules, such as... Figure 2 and Figure 3 The video encoder 200 and video decoder 300 are respectively shown with precoding processing 201, intra-frame prediction 260, entropy coding 245, and / or entropy decoding module 330, intra-frame prediction 360, and post-decoding processing 385. Furthermore, the subject matter disclosed herein presents aspects not limited to VVC or HEVC and can be applied to, for example, any type, format, or version of video coding (whether described in standards or recommendations, whether pre-existing or future-developed), and any extensions to such standards and recommendations (e.g., including VVC and HEVC). Unless otherwise indicated or technically excluded, the aspects described in this application may be used alone or in combination.

[0091] Various numerical values ​​are used in the examples described in this application, such as minimum and maximum value ranges (e.g., 0 to 1, 0 to N, or 0 to 255), bit values ​​for indication or determination, default values, ID numbers (e.g., for adaptive IDs), etc. These and other specific values ​​are for the purpose of describing examples, and the aspects described are not limited to these specific values.

[0092] Figure 2 This is a schematic diagram illustrating an exemplary video encoder. Variations of the exemplary encoder 200 are contemplated, but encoder 200 is described below for clarity, without describing all anticipated variations.

[0093] Before encoding, the video sequence may undergo pre-coding (201), for example, applying color transformations to the input color image (e.g., converting from RGB 4:4:4 to YCbCr 4:2:0), or performing remapping of the input image components to obtain a signal distribution that is more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with pre-processing and appended to the bitstream.

[0094] As described below, in encoder 200, the image is encoded by encoder elements. The image to be encoded is partitioned (202) and processed in units such as coding units (CUs). For example, each unit is encoded using either intra-frame mode or inter-frame mode. When a unit is encoded in intra-frame mode, intra-frame prediction (260) is performed on that unit. In inter-frame mode, motion estimation (275) and compensation (270) are performed. The encoder determines (205) which of the intra-frame mode or inter-frame mode to use for encoding the unit, and indicates the intra-frame / inter-frame decision by, for example, a prediction mode flag. For example, the prediction residual is calculated by subtracting (210) the prediction block from the initial image block.

[0095] Then, the predicted residual is transformed (225) and quantized (230). The quantized transform coefficients, motion vectors, and other syntax elements are entropy encoded (245) to output a bitstream. The encoder can skip the transform and apply quantization directly to the untransformed residual signal. The encoder can bypass both the transform and quantization, that is, encode the residual directly without applying the transform or quantization process.

[0096] The encoder decodes the coded blocks to provide a reference for further prediction. The quantized transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residuals. The decoded prediction residuals and prediction blocks are combined (255) to reconstruct an image block. A loop filter (265) is applied to the reconstructed image to perform, for example, deblocking / SAO (Sample Adaptive Shift) filtering, thereby reducing coding artifacts. The filtered image is stored in a reference image buffer (280).

[0097] Figure 3 This is a schematic diagram illustrating an example of a video decoder. In the exemplary decoder 300, the bitstream is decoded by decoder elements, as described below. The video decoder 300 generally performs the same operations as... Figure 2 The encoder 200 is the opposite of the encoding process described herein, performing a decoding process. The encoder 200 may also typically perform video decoding as part of the encoding of the video data. For example, the encoder 200 may perform one or more video decoding steps as presented herein. The encoder may, for example, reconstruct the decoded image to maintain synchronization with the decoder relative to one or more of the following: a reference picture, entropy coding context, and other decoder-related state variables.

[0098] Specifically, the decoder's input includes a video bitstream, which can be generated by the video encoder 200. First, the bitstream is entropy decoded (330) to obtain transform coefficients, motion vectors, and other encoded information. Image partitioning information indicates how the image is partitioned. Therefore, the decoder can partition (335) the image based on the decoded image partitioning information. The transform coefficients are dequantized (340) and inverse transformed (350) to decode the prediction residuals. The decoded prediction residuals and prediction blocks are combined (355) to reconstruct the image blocks. Prediction blocks (370) can be obtained from intra-frame prediction (360) or motion-compensated prediction (i.e., inter-frame prediction) (375). A loop filter (365) is applied to the reconstructed image. The filtered image is stored in a reference image buffer (380).

[0099] The decoded image can also undergo post-decoding processing (385), such as inverse color transformation (e.g., a transformation from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping of the remapping process performed in the pre-encoding process (201). Post-decoding processing can utilize metadata derived in the pre-encoding process and signaled in the bitstream.

[0100] Figure 4 This is a schematic diagram illustrating an example of a system in which the various aspects and embodiments described herein may be implemented. System 400 may be embodied as a device that includes the various components described below and is configured to perform one or more aspects described in this document. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. The elements of system 400 may be embodied individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one example, the processing and encoder / decoder elements of system 400 are distributed across multiple ICs and / or discrete components. In various embodiments, system 400 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various embodiments, system 400 is configured to implement one or more aspects described in this document.

[0101] System 400 includes at least one processor 410 configured to execute instructions loaded therein for implementing various aspects, such as those described in this document. Processor 410 may include embedded memory, input / output interfaces, and various other circuitry known in the art. System 400 includes at least one memory 420 (e.g., a volatile memory device and / or a non-volatile memory device). System 400 includes a storage device 440 that may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, storage device 440 may include internal storage devices, attached storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0102] System 400 includes an encoder / decoder module 430 configured to, for example, process data to provide encoded or decoded video, and the encoder / decoder module 430 may include its own processor and memory. The encoder / decoder module 430 represents a module that can be included in a device to perform encoding and / or decoding functions. It is well known that a device may include one or both of an encoding module and a decoding module. Alternatively, the encoder / decoder module 430 may be implemented as a separate element of system 400 or may be incorporated within processor 410 as a combination of hardware and software known to those skilled in the art.

[0103] Program code to be loaded onto processor 410 or encoder / decoder 430 to execute the various aspects described in this document may be stored in storage device 440 and subsequently loaded onto memory 420 for execution by processor 410. According to various embodiments, one or more of processor 410, memory 420, storage device 440, and encoder / decoder module 430 may store one or more items from various categories during the execution of the processes described in this document. Such stored items may include, but are not limited to, input video, decoded or partially decoded video, bitstreams, matrices, variables, and intermediate or final results of processing equations, formulas, operations, and operational logic.

[0104] In some embodiments, the memory within processor 410 and / or encoder / decoder module 430 is used to store instructions and provide working memory for processing required during encoding or decoding. However, in other embodiments, memory external to the processing device (e.g., the processing device may be processor 410 or encoder / decoder module 430) is used for one or more of these functions. The external memory may be memory 420 and / or storage device 440, such as volatile memory and / or non-volatile flash memory. In several embodiments, external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one implementation, a fast external dynamic volatile memory such as RAM is used as working memory for video encoding and decoding operations, such as MPEG-2 (MPEG stands for Moving Picture Experts Group, MPEG-2 is also known as ISO / IEC 13818, and 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC stands for High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2) or VVC (Universal Video Coding, a new standard developed by the Joint Video Experts Group JVET).

[0105] As indicated in block 445, inputs to the components of system 400 may be provided through various input devices. Such input devices include, but are not limited to: (i) a radio frequency (RF) section that receives, for example, RF signals transmitted over the air by a broadcaster; (ii) component (COMP) input terminals (or a set of COMP input terminals); (iii) universal serial bus (USB) input terminals; and / or (iv) high-definition multimedia interface (HDMI) input terminals. Figure 4 Other examples not shown include composite video.

[0106] In various embodiments, the input device of block 445 has associated corresponding input processing elements known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also known as selecting a signal, or limiting a signal band to a band), (ii) down-converting the selected signal, (iii) re-band-limiting the signal to a narrower band to select (e.g.,) a signal band that may be referred to as a channel in some embodiments), (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select the desired data packet stream. The RF section of various embodiments includes one or more elements for performing these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, downconverters, demodulators, error correctors, and demultiplexers. The RF section may include tuners that perform various functions among these functions, including, for example, down-converting received signals to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or to baseband. In one set-top box implementation, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection by filtering, down-converting, and re-filtering to the desired frequency band. Various implementations rearrange the order of the aforementioned (and other) components, remove some of these components, and / or add other components that perform similar or different functions. Adding components may include inserting components between existing components, such as inserting amplifiers and analog-to-digital converters. In various implementations, the RF section includes an antenna.

[0107] Additionally, USB and / or HDMI terminals may include corresponding interface processors for connecting system 400 to other electronic devices across USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented as needed, for example, within a separate input processing IC or within processor 410. Similarly, various aspects of USB or HDMI interface processing may be implemented as needed, either within a separate interface IC or within processor 410. The demodulated, error-corrected, and demultiplexed stream is provided to various processing elements, including, for example, processor 410 and encoder / decoder 430, which operate in conjunction with memory and storage elements to process the data stream as needed for display on the output device.

[0108] Various components of system 400 can be housed within an integrated housing. Within the integrated housing, various components can be interconnected using a suitable connection arrangement 425 (e.g., internal buses known in the art, including inter-IC (I2C) buses, wiring, and printed circuit boards) and data can be transferred between these components.

[0109] System 400 includes a communication interface 450 capable of communicating with other devices via a communication channel 460. The communication interface 450 may include, but is not limited to, a transceiver configured to transmit and receive data via the communication channel 460. The communication interface 450 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 460 may be implemented, for example, within a wired and / or wireless medium.

[0110] In various implementations, wireless networks such as Wi-Fi networks, such as IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers), are used to stream or otherwise provide data to system 400. In these examples, the Wi-Fi signal is received via a communication channel 460 and a communication interface 450 suitable for Wi-Fi communication. The communication channel 460 in these implementations is typically connected to an access point or router that provides access to external networks, including the Internet, for allowing streaming applications and other over-the-air (OTT) communications. Other implementations use a set-top box to provide streaming data to system 400, delivering data via an HDMI connection in input box 445. Other implementations use an RF connection in input box 445 to provide streaming data to system 400. As mentioned above, various implementations provide data in a non-streaming manner. Additionally, various implementations use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0111] System 400 can provide output signals to various output devices, including display 475, speaker 485, and other peripheral devices 495. Display 475 in various embodiments includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. Display 475 can be used in televisions, tablets, laptops, mobile phones, or other devices. Display 475 can also be integrated with other components (e.g., in a smartphone) or can be standalone (e.g., an external monitor for a laptop computer). In various examples of embodiments, other peripheral devices 495 include one or more of a standalone digital video disc (or digital universal optical disc) (DVR, which may represent both terms), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 495 that provide functionality based on the output of system 400. For example, a disc player performs the function of playing the output of system 400.

[0112] In various embodiments, signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable device-to-device control with or without user intervention are used to transmit control signals between system 400 and display 475, speaker 485, or other peripheral devices 495. These output devices are communicatively coupled to system 400 via dedicated connections through corresponding interfaces 470, 480, and 490. Alternatively, output devices may be connected to system 400 via communication interface 450 using communication channel 460. Display 475 and speaker 485 may be integrated into a single unit with other components of system 400 in electronic devices such as televisions. In various embodiments, display interface 470 includes a display driver, such as, for example, a timing controller (TCon) chip.

[0113] For example, if the RF section of input 445 is part of a separate set-top box, then display 475 and speaker 485 may optionally be separate from one or more other components. In various embodiments where display 475 and speaker 485 are external components, output signals may be provided via dedicated output connections, including, for example, HDMI ports, USB ports, or COMP outputs.

[0114] These implementations can be executed by computer software implemented by processor 410, by hardware, or by a combination of hardware and software. As a non-limiting example, these implementations can be implemented by one or more integrated circuits. As a non-limiting example, memory 420 can be of any type suitable for the technical environment and can be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. Processor 410 can be of any type suitable for the technical environment and, as a non-limiting example, can encompass one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures.

[0115] Various specific implementations involve decoding. As used in this application, "decoding" can encompass, for example, all or part of the process performed on a received encoded sequence to produce a final output suitable for display. In various implementations, such processes may include one or more processes typically performed by a decoder, such as entropy decoding, inverse quantization, inverse transform, and differential decoding. In various implementations, such processes also include, or alternatively include, processes performed by a decoder of the various specific implementations described in this application, such as receiving, decoding, and interpreting signals indicating elements, attributes, and metadata associated with point cloud components (e.g., as described herein); identifying point cloud streams and their component substreams within a Media Display Descriptor (MPD); identifying versions of point clouds and / or their components; decoding the MPD to identify master adaptive sets and other adaptive sets, thereby identifying G-PCC components in geometry-based point cloud compression (G-PCC) content; decoding the MPD to identify the type of point cloud components in the adaptive sets or representations; decoding the MPD to identify This includes: pre-selecting one or more pre-selected regions; decoding the MPD to identify one or more versions of the G-PCC media; decoding the MPD to identify one or more G-PCC tile groups; decoding the MPD to identify one or more tile IDs for G-PCC components in an adaptive set; decoding the MPD to identify one or more characteristics of a spatial region and the mapping between the region and G-PCC tiles, the characteristics of the spatial region and the mapping between the region and the corresponding adaptive set of the G-PCC component, and / or the mapping between the spatial region and the corresponding adaptive set of the G-PCC component; decoding the MPD to identify timing metadata tracks for dynamic spatial regions; etc.

[0116] As another implementation, in one example, the decoding is signified entropy decoding; in another implementation, signified differential decoding; and in yet another implementation, a combination of signified entropy decoding and differential decoding. Whether the phrase decoding process is intended to specifically refer to a subset of operations or to broadly refer to a wider decoding process will be clear based on the specific context of the description and is believed to be well understood by those skilled in the art.

[0117] Various specific implementations may involve encoding. In a manner similar to the discussion above regarding decoding, encoding, as used in this application, may encompass, for example, all or part of the process performed on an input video sequence to produce an encoded bitstream. In various implementations, such processes include one or more processes typically performed by an encoder, such as partitioning, differential coding, transform, quantization, and entropy coding. In various implementations, such processes also include, or alternatively include, processes performed by an encoder of the various specific implementations described in this application, such as generating, encoding, and transmitting signals indicating elements, attributes, and metadata associated with point cloud components (e.g., as described herein); encoding the MPD to indicate the point cloud stream and its component substreams; encoding the MPD to indicate a master adaptive set and other adaptive sets to support the identification of G-PCC components in geometry-based point cloud compression (G-PCC) content; encoding the MPD to support the identification of the type of point cloud components in the adaptive set or representation; encoding the MPD to identify one or more pre-selections; and encoding the MPD to... Encoding to support the identification of one or more versions of G-PCC media; encoding MPD to support the identification of one or more G-PCC tile groups; encoding MPD to support the identification of one or more tile IDs of G-PCC components in an adaptive set; encoding MPD to support the identification of one or more characteristics of a spatial region and the mapping between the region and G-PCC tiles, the characteristics of a spatial region and the mapping between the region and the corresponding adaptive set of G-PCC components, and / or the mapping between a spatial region and the corresponding adaptive set of G-PCC components; decoding MPD to identify timing metadata tracks for dynamic spatial regions, etc.

[0118] As another example, in one embodiment, the encoding may refer to entropy encoding; in another embodiment, the encoding may refer to differential encoding; and in yet another embodiment, the encoding may refer to a combination of differential and entropy encoding. Whether the phrase encoding process is intended to specifically refer to a subset of operations or to broadly refer to a wider encoding process will be clear based on the specific context of the description and is believed to be well understood by those skilled in the art.

[0119] It should be noted that the grammatical elements used in this document (such as those indicated in Tables 1 through 23 and otherwise presented herein as arguments or figures) are descriptive terms. Therefore, they do not preclude the use of other grammatical element names.

[0120] When the accompanying drawings are presented as flowcharts, it should be understood that block diagrams of the corresponding devices are also provided. Similarly, when the accompanying drawings are presented as block diagrams, it should be understood that flowcharts of the corresponding methods / processes are also provided.

[0121] During the encoding process, a balance or trade-off between rate and distortion is typically considered, often taking into account computational complexity constraints. Rate-distortion optimization is generally formulated as minimizing a rate-distortion function, which is a weighted sum of rate and distortion. Different approaches exist to solve the rate-distortion optimization problem. For example, these methods may be based on extensive testing of all encoding options (including all considered modes or encoding parameter values) and a complete evaluation of their encoding costs and the associated distortion of the reconstructed signal after encoding and decoding. Faster methods can be used to reduce encoding complexity, particularly for calculating approximate distortion based on prediction or prediction of the residual signal rather than the reconstructed residual signal. A hybrid of these approaches can be used, for example, by using approximate distortion for some of the possible encoding options and full distortion for others. Other methods evaluate subsets of possible encoding options. More generally, many methods employ any of a variety of techniques to perform optimization, but optimization is not necessarily a complete evaluation of both encoding costs and associated distortion.

[0122] The specific embodiments and aspects described herein may be implemented, for example, in methods or processes, apparatus, software programs, data streams, or signals. Even if discussed only in the context of a single form of specific embodiment (e.g., discussed only as a method), specific embodiments of the discussed features may be implemented in other forms (e.g., apparatus or program). Apparatus may be implemented, for example, in suitable hardware, software, and firmware. These methods may be implemented, for example, in a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices, such as, for example, computers, mobile phones, portable / personal digital assistants (PDAs), and other devices that facilitate information communication between end users.

[0123] The references to “an implementation,” “implementation,” “example,” or “a specific implementation,” or “specific implementation,” and their variations, mean that the specific features, structures, characteristics, etc., described in connection with the implementation are included in at least one implementation. Therefore, the appearance of the phrases “in an implementation,” “in an implementation,” “in an example,” or “in an implementation,” and any other variations appearing throughout this application, do not necessarily refer to the same implementation or example.

[0124] Additionally, this application may involve "determining" various types of information. Determining information may include, for example, one or more of estimated information, calculated information, predicted information, or information retrieved from memory. Obtaining may include receiving, retrieving, constructing, generating, and / or determining.

[0125] Furthermore, this application may relate to "accessing" various types of information. Accessing information may include, for example, receiving information, retrieving information (e.g., from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information, or more of these.

[0126] Furthermore, this application may relate to "receiving" various types of information. Like "access," "receiving" is intended to be a broad term. Receiving information may include, for example, accessing information or retrieving information (e.g., from memory) or more. Moreover, "receiving" typically involves one or more of the following during operations such as, for example, storing information, processing information, transmitting information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0127] It should be understood that, for example, in the cases of “A / B,” “A and / or B,” and “at least one of A and B,” the use of any of the following “ / ,” “and / or,” and “at least one” is intended to cover selecting only the first listed option (A), or only the second listed option (B), or both options (A and B). As a further example, in the cases of “A, B, and / or C” and “at least one of A, B, and C,” such phrases are intended to cover selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A, B, and C). As will be apparent to those skilled in the art and related fields, this can be extended to as many items as possible listed.

[0128] Moreover, as used herein, the term "signaling" refers to (among other things) instructing the corresponding decoder to do something. For example, in some embodiments, the encoder (e.g., to the decoder) signals the MPD, adaptive set, representation, preselection, G-PCC component, G-PCCComponent descriptor, G-PCC descriptor or basic attribute descriptor, supplementary attribute descriptor, G-PCC tile inventory descriptor, G-PCC static spatial region descriptor, GPCCTileId descriptor, GPCC3DRegionID descriptor, other descriptors, elements and attributes, metadata, patterns, etc. (e.g., included in Tables 1 through 23 as disclosed herein). Thus, in one embodiment, the same parameter can be used at both the encoder and decoder sides. Therefore, for example, the encoder can transmit (explicit signaling) a specific parameter to the decoder so that the decoder can use the same specific parameter. Conversely, if the decoder already has the specific parameter as well as other parameters, signaling can be used without transmission (implicit signaling) to simply allow the decoder to know and select the specific parameter. Bit savings are achieved in various embodiments by avoiding the transmission of any actual functionality. It should be understood that signaling can be implemented in various ways. For example, in various implementation schemes, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder. Although the verb form of the word "signal" was mentioned above, the word "signal" can also be used as a noun in this article.

[0129] It will be apparent to those skilled in the art that the embodiments may produce various signals formatted to carry, for example, storable or transmissible information. The information may include, for example, instructions for performing a method or data generated by one of the embodiments. For example, the signal may be formatted to carry a bit stream of that embodiment. Such signals may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or baseband signals. Formatting may include, for example, encoding the data stream and using a modulated carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal can be transmitted via a variety of different wired or wireless links. The signal may be stored on a processor-readable medium.

[0130] 3D point clouds can represent (e.g., for representing) immersive media. A point cloud can include a set of points represented in three-dimensional (3D) space. In an example, a point (e.g., each point) can be associated with one or more coordinates indicating the point's location and / or one or more attributes (e.g., point color, transparency, acquisition time, laser reflectivity, material properties, etc.). Point clouds can be captured or deployed, for example, using one or more cameras, depth sensors, and / or light detection and ranging (LiDAR) laser scanners. A point cloud can include multiple points. In an example, a point (e.g., each point) can be represented by a set of coordinates mapped in 3D space (e.g., x-coordinate, y-coordinate, z-coordinate). Points can be generated based on sampling of objects. In an example, the number of points within a point cloud can be approximately millions or billions. Point clouds can be used to reconstruct one or more objects and / or scenes. For example, point clouds can be represented and / or compressed, for example, to store and / or transmit (e.g., to efficiently store and / or transmit) point cloud data. Point cloud compression supports lossy and / or lossless encoding (e.g., encoding or decoding) of the geometric coordinates and / or attributes of point clouds. Point clouds can be deployed to support various applications (e.g., telepresence, virtual reality (VR), and / or large-scale dynamic 3D mapping). In one example, a library for mesh and point cloud compression supports compression of vertex positions, normals, colors, texture coordinates, and other common vertex attributes, for example, to improve the efficiency and speed of transmitting 3D content. An example of such a library is one developed by Google. ™ Developed DRACO ™ .

[0131] Figure 5 An example of a bitstream structure for geometry-based point cloud compression (G-PCC) is shown. A G-PCC bitstream may include a set of G-PCC cells; for example, a G-PCC cell may be referred to as a type-length-value (TLV) encapsulation structure, such as... Figure 5 As shown. G-PCC and GPCC are used interchangeably in this document. Figure 5 As shown, a G-PCC cell may include information about the G-PCC tlv_type and the G-PCC tlv cell payload. Figure 5Various TLV unit payload types are illustrated. Table 1 shows examples of G-PCC TLV unit syntax. In the examples, a G-PCC TLV unit (e.g., each G-PCC TLV unit) may include a TLV type, a G-PCC TLV unit payload length, and / or a G-PCC TLV unit payload. A TLV type (e.g., tlv_type as shown in Table 1) indicates the G-PCC unit type. Table 2 shows examples of TLV types (e.g., tlv_type as shown in Table 1) and associated data unit descriptions. For example, a G-PCC TLV unit with unit type 2 can be a geometric data unit, and a G-PCC TLV unit with unit type 4 can be an attribute data unit. Point clouds can be reconstructed, for example, based on geometric data units and attribute data units. Geometric and / or attribute G-PCC unit payloads may correspond to media data units (e.g., TLV units) that can be decoded by a G-PCC decoder. G-PCC units with geometric and attribute parameter sets can specify a G-PCC decoder to decode the corresponding TLV unit. The G-PCC Bitstream High-Level Syntax (HLS) supports slices and / or tile groups for geometric and attribute data. A frame can be divided into multiple tiles and slices. A slice can be a set of points that can be encoded or decoded (e.g., independently). In the example, a slice may include geometric data units and zero or more attribute data units. Attribute data units may depend on, for example, corresponding geometric data units within the same slice. Within a slice, geometric data units may appear before any associated attribute units. Data units in a slice can be consecutive. The order of slices within a frame may not be specified. Slice groups can be identified by a common tile identifier. A tile inventory describing the bounding boxes of tiles (e.g., each tile) can be implemented. Tiles may overlap with other tiles within a bounding box. Each slice may include an index identifying which tile the slice belongs to. Table 1 shows an example of the G-PCC TLV encapsulation unit payload syntax, Table 2 shows an example of G-PCC TLV types and data unit descriptions, and Table 3 shows an example of the G-PCC TLV encapsulation unit payload syntax. Table 1 Table 2 Table 3 .

[0132] This document illustrates and describes examples of elements, attributes, syntax, and semantics. Elements are distinct from attributes. Attributes are identified by an "@" sign preceding the attribute. Examples of elements using value ranges can be provided in the following format: <minimum value>...<maximum value>, where the value of N indicates that the value is unbounded. The elements, attributes, syntax, and semantics described herein are non-limiting examples, and in various specific implementations, with or without exemplary use, these non-limiting examples may or may not be implemented alone or in various combinations.

[0133] It can implement the G-PCC container file format. Figure 6 An example sample structure is shown when the G-PCC geometry and attribute bitstream can be stored in a single track. For example, in the case where the G-PCC bitstream is carried in a single track, the video encoding device may require the G-PCC encoded bitstream to be represented by a single-track declaration. The single-track encapsulation of G-PCC data can utilize simple encapsulation (e.g., ISO Basic Media File Format (ISOBMFF) encapsulation), such as by storing the G-PCC bitstream in a single track without processing (e.g., further processing). A sample in a single track (e.g., each sample) may include one or more G-PCC components. Each sample may include one or more TLV encapsulation structures.

[0134] Figure 7 An example of a multi-track (e.g., ISOBMFF) G-PCC container structure is shown. With the encoded G-PCC geometry bitstream and the encoded G-PCC attribute bitstream stored in separate tracks, each sample in the track may include at least one TLV encapsulation structure carrying G-PCC component data.

[0135] A multi-track G-PCC ISOBMFF container may include G-PCC tracks, which include sets of geometric parameters, sets of sequence parameters, and / or samples of geometric bitstreams carrying TLV units of geometric data. G-PCC tracks may include track references to other tracks carrying payloads of G-PCC attribute components. A multi-track G-PCC ISOBMFF container may include zero or more G-PCC tracks, each including a set of attribute parameters for the corresponding attribute and samples of attribute bitstreams carrying TLV units of attribute data.

[0136] When carrying G-PCC bitstreams across multiple tracks, track referencing tools can be used to link between G-PCC component tracks. For example, TrackReferenceTypeBoxes can be added to a TrackReferenceBox within a TrackBox of a G-PCC track. A TrackReferenceTypeBox can include an array of track_IDs, specifying, for example, the track referenced by the G-PCC track. To link a G-PCC geometry track to a G-PCC attribute track, the reference_type of the TrackReferenceTypeBox in the G-PCC geometry track can be implemented to identify the associated attribute track. The 4CC of the track reference type can be "gpca". In the example, the referenced track can include an encoded bitstream of G-PCC attribute data.

[0137] When the geometry stream of a G-PCC bitstream comprises multiple tiles, each tile or group of tiles can be encapsulated in an independent track (such as a geometry tile track). In the example, a geometry tile track can carry one or more TLV cells of geometry tiles, which allows direct access to the tiles. The attribute stream of a G-PCC bitstream comprises multiple tiles that can be carried in multiple attribute tile tracks.

[0138] G-PCC tile data can be carried in separate geometry and attribute tile tracks within a container. Partial access to the G-PCC encoded stream is supported within the ISOBMFF container. Tiles corresponding to spatial regions within the point cloud scene can be signaled in samples of timing metadata tracks (such as tracks with Dynamic3DSpatialRegionSampleEntry) or in GPCCSpatialRegionInfoBox boxes. This allows players and / or streaming clients to retrieve sets of tile tracks carrying the information needed to render specific spatial regions or tiles within the point cloud scene.

[0139] The G-PCC base track can carry a TLV encapsulation structure. The TLV encapsulation structure may contain (e.g., may contain only) a Sequence Parameter Set (SPS), a Geometry Parameter Set (GPS), an Attribute Parameter Set (APS), and tile inventory information. To link a G-PCC base track to a geometry tile track, a four-character code (4CC) (“gccg”) can be used to define a track reference with a new track reference type. This new type of track reference can be used to link the G-PCC base track to geometry tile tracks (e.g., each geometry tile track within a geometry tile track).

[0140] The track referencing tool can be used to link geometric tile tracks (e.g., each geometric tile track) with other attribute G-PCC tile tracks that carry attribute information for the corresponding tile or tile group. The 4CC for these track reference types can be "gpca".

[0141] Alternate tracks can be indicated by an alternative track mechanism (e.g., the `alternate_group` field of a `TrackHeaderBox`). In the example, G-PCC component tile tracks that include the same `alternate_group` value can be different encoded versions of the same G-PCC component. Volumetric visual scenes can be encoded within alternative tracks. In this case, G-PCC tracks that are alternative tracks to each other, for example, can include the same `alternate_group` value in their `TrackHeaderBox`s.

[0142] G-PCC component tile tracks may include alternative tracks. In this case, G-PCC component tile tracks belonging to an alternative group (e.g., all G-PCC component tile tracks) may be referenced by the G-PCC base track and / or the corresponding G-PCC geometry tile track. In the example, G-PCC component tile tracks that are alternative tracks to each other may use an alternative grouping mechanism.

[0143] MPEG Dynamic Adaptive Streaming over HTTP (MPEG-DASH) is a delivery format that can dynamically adapt to changes in network delivery conditions to provide, for example, a better video experience to the end user.

[0144] Dynamic HTTP streaming can deliver multimedia content at one or more bitrates available at the server. The multimedia content may include multiple media components (e.g., audio, video, and / or text media components). Different media components may include different characteristics. One or more characteristics of a media component may be described, for example, by a Media Presentation Description (MPD).

[0145] Figure 8 An exemplary MPD hierarchical data model is shown. Figure 8 As shown, MPD can describe a sequence of periods (e.g., time intervals). For example, a set of encoded versions of a media content component may remain unchanged during a certain period. A period (e.g., each period) may have a start time and an associated duration. A period (e.g., each period) may include one or more adaptation sets (e.g., AdaptationSet, such as AdaptationSet 1, etc.). Figure 8(As shown in the diagram). The Adaptation Set (Adaptation Set), Adaptation Set, and Adaptation Set are used interchangeably in this document. In one example, the DASH streaming client can be a WTRU, for example, as shown in this document relative to... Figures 1A to 1D In another example, the DASH streaming client may include a head-mounted device, a head-mounted projector, and / or a head-up display. In another example, the DASH streaming client may include a 3D television. In yet another example, the DASH streaming client may include one or more cameras (e.g., advanced cameras).

[0146] An adaptation set (e.g., Adaptation Set, Adaptation Set, Adaptation Set, or Adaptation Set) can represent a collection of encoded versions of one or more media content components that share one or more attributes (e.g., the same attribute), such as language, media type, image aspect ratio, character, accessibility, viewpoint, rating attribute, etc. In the example, an Adaptation Set may include different bitrates for geometric and / or attribute components of multimedia content (e.g., G-PCC content). An Adaptation Set may include different bitrates for audio components (e.g., lower quality stereo and / or higher quality surround sound) of multimedia content (e.g., the same multimedia content). In the example, an Adaptation Set (e.g., each adaptation set) may include multiple representations.

[0147] A representation can describe a deliverable encoded version of one or more media components. The terms "representation" and "representation" are used interchangeably in this document. A representation can differ from other representations, for example, in terms of bitrate, resolution, number of channels, and / or other characteristics. A representation (e.g., each representation) can include one or more fragments. Attributes of the representation element (e.g., @id, @bandwidth, @qualityRanking, and / or @dependencyId) can (e.g., can be used to) specify one or more attributes of the representation.

[0148] Fragments can be retrieved using HTTP requests. Fragments (e.g., each fragment) may include URLs (e.g., addressable locations on a server). In the example, fragments can be downloaded, for example, using an HTTP GET or an HTTP GET with byte ranges.

[0149] A DASH client can parse MPD XML documents. For example, a DASH client can select a collection of AdaptationSets (e.g., applicable to the DASH client environment) based on elements of the AdaptationSet (e.g., information provided in each element of the AdaptationSet). The client can select a Representation for the AdaptationSet (e.g., within each AdaptationSet). The client can select a Representation based on, for example, the value of the @bandwidth attribute, the client's encoding capabilities, and / or the client's rendering capabilities. The client can download an initial fragment of the selected Representation. The client can access the content (e.g., by requesting the entire fragment or a byte range of a fragment). The client can continue consuming media content, for example, at the start of a presentation or during a presentation. The client can request (e.g., continuously) media segments and / or portions of media segments during a presentation. The client can play content according to the media presentation timeline. The client can switch from a first Representation to a second Representation based, for example, on updated information from the client environment. The client can play content continuously, for example, across one or more cycles. Media presentations can be terminated (e.g., consumed by the client in segments), can begin a cycle, and / or can be re-acquired, for example, towards the end of the media announced in the Representation.

[0150] MPEG-DASH descriptors provide specific information about media content. Descriptor element structures can be similar. A descriptor element may include, for example, an `@schemeIdUri` attribute (which provides a URI to identify the scheme), an `@value` attribute, and / or an `@id` attribute. Element semantics can be specific to the scheme employed. The URI identifying the scheme can be, for example, a URN or a URL. The MPD provides information about how to use the element. Applications using the DASH format can, for example, utilize scheme information to instantiate descriptor elements. DASH applications using elements (e.g., descriptor elements) can define (e.g., first define) a scheme identifier (e.g., in the form of a URI) and can define a value space for that element (e.g., for when to use the scheme identifier). In the example, extended elements and / or attributes can be defined in, for example, a separate namespace for structured data. Descriptors can appear at multiple levels within the MPD. For example, the presence of an element at an MPD level can indicate that the element is a child element of an MPD element. For example, the presence of an element at an AdaptationSet level can indicate that the element is a child element of an AdaptationSet element. For example, the presence of an element at the Representation level indicates that the element is a child element of the Representation element.

[0151] A bundle (e.g., in MPEG-DASH) may be a set of media components that can be consumed jointly by decoder instances (e.g., a single decoder instance). Each bundle (e.g., each bundle) may include a media component (e.g., a main media component) that may include decoder-specific information and / or guide the decoder. Pre-selection may refer to, identify, and / or define a subset of media components in a bundle that are, for example, jointly consumable (e.g., expected to be jointly consumable).

[0152] The AdaptationSet that includes the main media component may be referred to as the Main Adaptation Set. The Main Adaptation Set, main Adaptation Set, and main adaptation set (e.g., any variant based on or compounded with an initial capitalization of the adaptation set, e.g., adaptationset) are used interchangeably herein. The main media component may be included in a preselection that may be associated with a bundle. Bundles (e.g., each bundle) may include one or more partial AdaptationSets. Partial AdaptationSets may be processed in combination with the main Adaptation Set.

[0153] Table 4 shows examples of preselection element semantics. Preselection can be defined, for example, by preselection elements (as shown in Table 4). In the examples, selection of a preselection can be based on attributes and / or elements that can be included in the preselection element. Table 4 .

[0154] 3D point clouds can be used to implement or represent multimedia applications (such as virtual reality (VR) and immersive 3D graphics) that enable interaction and / or communication with one or more virtual worlds in updated forms. Static and dynamic point clouds can generate vast amounts of information. Efficient encoding algorithms can be used to compress point cloud information, for example, to reduce the utilization of storage and / or transmission resources. For instance, a compressed bitstream of dynamic point cloud information can utilize fewer transmission resources than an uncompressed bitstream.

[0155] Point cloud applications may utilize encoding, storage, and / or network resources (e.g., streaming point cloud data over a network). In the example, a point cloud application may perform live or on-demand streaming of point cloud content, depending on how the content can be generated. Point cloud applications may create, process, and / or send or receive large amounts of information representing point clouds. Point cloud applications may support adaptive streaming techniques, for example, to avoid network overload and / or provide an optimized viewing experience, such as relative to varying network capacity and / or other operating conditions.

[0156] MPEG-DASH can (e.g., can be used to) provide adaptive delivery of point clouds. MPEG-DASH can be implemented, for example, using signaling that supports point cloud media (including point cloud streams). Signaling elements can instruct or enable streaming clients to identify point cloud streams and their component substreams within an MPD file. Signaling elements can instruct or enable streaming clients to identify one or more types of metadata that can be associated with point cloud components, for example, to enable streaming clients to select a version (e.g., optimal version) of point cloud or point cloud component that the streaming client can be configured or can be configured to support.

[0157] Components of point cloud content may be available in different representations. In the example, multiple representations (e.g., each of multiple representations) may represent different quality levels. Streaming clients may utilize guidance regarding different representations (e.g., indications signaled in the MPD file). For example, this indication may specify which set of representations across different components constitutes a particular quality level (e.g., to perform appropriate quality degradation). Components of point cloud content may be divided into multiple tiles. Clients may, for example, stream specific tile portions of a geometric component (e.g., selected tile portions) based on bandwidth availability (e.g., rather than streaming all point cloud data). G-PCC component tile bitstreams may be available at different adaptive sets, for example, where each adaptive set (e.g., each adaptive set) represents a G-PCC component tile.

[0158] G-PCC media content may include multiple components, such as geometry and / or attributes. Components (e.g., each of multiple components) may be encoded individually as substreams, such as G-PCC bitstreams. Components (such as geometry and attributes) may be encoded, for example, using a G-PCC encoder. Substreams may be decoded collectively (e.g., along with metadata), for example, to render point clouds.

[0159] Elements and / or attributes can be defined, for example, as XML elements and / or XML attributes. XML elements can be defined, for example, in a separate namespace (e.g., "urn:mpeg:mpegI:gpcc:2020"). The namespace designator "gpcc:" can be used herein to refer to, for example, a separate namespace.

[0160] G-PCC components can be signaled in the DASH MPD. In the example, each G-PCC component (e.g., each G-PCC component) can be represented as an independent AdaptationSet in the DASH manifest file (e.g., the MPD file), which may be called a component adaptation set. An adaptation set including geometric information can be, for example, a master adaptation set that can act as an access point (e.g., a primary access point) for G-PCC content. In the example, an adaptation set (e.g., a single adaptation set) can be signaled per component by resolution. In the example, the master adaptation set may include a @codecs attribute set to "gpc1".

[0161] For example, the EssentialProperty descriptor can be used with the @schemeIdUri attribute, which is set to "urn:mpeg:mpegI:gpcc:2020:component", to identify the type of G-PCC component in the component adaptive set. The EssentialProperty descriptor can be referred to as, for example, the GPCCComponent descriptor.

[0162] In the example (e.g., at the adaptive set level), the GPCCComponent descriptor (e.g., a GPCCComponent descriptor) can be signaled to the point cloud components (e.g., each point cloud component) that exist in the Representation of the adaptive set.

[0163] Table 5 shows examples of elements and properties for the GPCCComponent descriptor. In the examples, the @value property of the GPCCComponent descriptor may not exist. The GPCCComponent descriptor may include properties defined in Table 5. Table 5 .

[0164] Table 6 shows an example of the XML schema corresponding to the GPCCComponent descriptor in Table 5. Table 6 .

[0165] The master adaptation set may include, for example, an initialization fragment at the adaptation set level (e.g., a single initialization fragment) or multiple initialization fragments at the representation level (e.g., one initialization fragment per representation). The initialization fragment may include a set of G-PCC parameters that can (e.g., can be used to) initialize the G-PCC decoder. A set of G-PCC parameters for one or more representations (e.g., all representations) may be included in the initialization fragment, for example, where an initialization fragment (e.g., a single initialization fragment) is present.

[0166] In the example, the initialization fragment for a representation (e.g., each representation) may include the G-PCC parameter set for that representation and the geometry data for that representation, for example, when more than one representation is signaled in the primary adaptation set. Representations for other component adaptation sets (e.g., other component adaptation sets of a point cloud) may, for example, use the `@dependencyId` attribute to list the corresponding representation identifier from the primary adaptation set. Representations in the primary adaptation set can be mapped to corresponding representations in the G-PCC component AdaptationSet. Media fragments for a representation of the primary adaptation set may include, for example, one or more track fragments of a G-PCC track. Media fragments for a representation of a component AdaptationSet may include, for example, one or more track fragments of a corresponding component track (e.g., at the file format level).

[0167] In the example, the role descriptor element can be used with values ​​that can be defined for G-PCC components. For example, one or more geometry components may include a corresponding value for gpcc-geometry, and / or one or more attribute components may include a corresponding value for gpcc-attribute. The EssentialProperty descriptor element can be signaled at the adaptive set level (e.g., similar to the EssentialProperty descriptor element described for the example shown in Table 5). In the example, the EssentialProperty descriptor element can be signaled without the component_type attribute (e.g., at the adaptive set level). The EssentialProperty descriptor element can be signaled, for example, to identify geometry components and / or attribute components.

[0168] In the example, one of multiple versions of a G-PCC component (e.g., each version) can be signaled in a separate AdaptationSet, where the value of the @codecs attribute is set according to the media codec used, for example, in the case where multiple versions of the G-PCC component are encoded using different codecs. Switching between representations (e.g., seamless switching) across multiple versions of the G-PCC component's AdaptationSet can be supported. Each AdaptationSet in the multiple AdaptationSets can include a SupplementalProperty descriptor, such as where @schemeIdURI is set to urn:mpeg:dash:adaptation-set-switching:2016 and / or @value includes a comma-separated list of AdaptationSet IDs corresponding to other available versions, for example, to indicate support for seamless switching between representations across multiple versions of the G-PCC component's AdaptationSet. In the example, one or more rules can be applied to support switching across AdaptationSets.

[0169] The G-PCC tile track can be signaled. When multiple tile tracks exist within the G-PCC container, the master adaptive set may (e.g., may only) include a set of parameters and tile inventory information from the G-PCC base track. Geometric and / or attribute data may be absent from the master adaptive set and its representation. In the example, the @codecs attribute for the master adaptive set can be set to "gpcb," for example, indicating that the adaptive set includes base track data that includes (e.g., only) the SPS, GPS, APS, and tile inventory information of the G-PCC content.

[0170] Component tile tracks (e.g., each component tile track) can be signaled within an independent adaptive set. An independent adaptive set may be referred to as a tile component adaptive set. When multiple versions of a component for the same tile (e.g., the same tile set) exist and are carried / or in independent tile tracks, each version can be signaled in the representation of the tile component adaptive set. The @codecs attribute of the component tile track representing the G-PCC media content for the tile component adaptive set can be set to "gpt1".

[0171] At the tile component adaptive set level, the GPCCComponent descriptor can be signaled. In the example, the GPCCComponent descriptor may include the attribute (e.g., an additional attribute) @tile_ids, indicating, for example, a list of tiles present in the tile bitstream. The GPCCComponent descriptor may include (e.g., conditionally include) the XML attribute @attr_index, for example, when the component represented by the enclosing adaptive set is a G-PCC attribute component. The @attr_index attribute can signal the order of G-PCC attribute components in the SPS and / or enable differentiation of G-PCC attribute components, for example, when multiple G-PCC attribute components with the same attribute type (e.g., more than one color attribute) exist in the G-PCC content. The GPCCComponent descriptor at the tile component adaptive set level may include elements and / or attributes as defined in Table 7. Table 7 - Elements and Attributes for the GPCCComponent Descriptor .

[0172] Table 8 shows an example of the XML schema corresponding to the GPCCComponent descriptor in Table 7. Table 8 - Examples of XML schemas for the GPCCComponent descriptor .

[0173] In the example, when multiple tile tracks exist within the container, each representation in the geometry tile component adaptive set can, for example, use the `@dependencyId` attribute to reference the corresponding representation in the main adaptive set. Similarly, each representation in the property tile component adaptive set can, for example, use the `@dependencyId` attribute to reference the corresponding representation in the geometry tile component adaptive set.

[0174] The MPD can signal to G-PCC component tile tracks that have the same alternate_group value, for example, as a representation of the tile component adaptive set.

[0175] The G-PCC descriptor can be signaled. The streaming client can (e.g., be able to or be configured to) identify the point cloud component type in the AdaptationSet and / or Representation, for example, by examining the GPCCComponent descriptor within the corresponding element. The streaming client can distinguish between different geometric point cloud streams existing in the MPD file.

[0176] G-PCC descriptors may include, for example, a SupplementalProperty element with a @schemeIdUri attribute, for example, equal to "urn:mpeg:mpegI:gpcc:2020:gpc". Table 9 shows examples of attributes for G-PCC descriptors. In the examples, one or more (e.g., at most one) G-PCC descriptors may exist at the adaptive set level of the master adaptive set for G-PCC media. Table 9 .

[0177] The data type for an attribute can be defined, for example, in an XML schema. Table 10 shows an example of an XML schema for a G-PCC descriptor. A schema can be represented, for example, as an XML schema with the namespace urn:mpeg:mpegI:gpcc:2020. Table 10 .

[0178] The GPCCTileId descriptor can be signaled. A streaming client can, for example, identify one or more tile IDs existing in the G-PCC tile component adaptation set by examining the GPCCComponent descriptor. In the example, one or more components (e.g., all components) of a G-PCC tile may be stored in a single track. For example, the GPCCComponent descriptor may not be signaled in the AdaptationSet associated with that track. In the example, the streaming client can distinguish different G-PCC tile tracks that may exist in the MPD file. For example, the streaming client can distinguish different G-PCC tile tracks by identifying the corresponding tile streams.

[0179] In the example, a SupplementalProperty element with a @schemeIdUri attribute equal to "urn:mpeg:mpegI:gpcc:2020:tileID" can be (for example, referred to as) a GPCCTileId descriptor. The GPCCTileId descriptor can be used to distinguish different G-PCC tile streams. In the example, one (for example, at most one) GPCCTileId descriptor can be signaled and / or presented for G-PCC tile media at the adaptive set level. One (for example, at most one) GPCCTileId descriptor can be signaled or presented at the adaptive set level, for example, when the GPCCComponent descriptor is not available at the adaptive set level (for example, when all G-PCC components of a tile or tile group are in one track).

[0180] In the example, the @value attribute of the GPCCTileId descriptor may not exist. The GPCCTileId descriptor may include one or more attributes as shown in Table 11. Table 11 .

[0181] The data type for the attribute can be as provided in the XML schema. In the example, the XML schema for the GPCCTileId descriptor can be shown as shown in the following exemplary schema. This schema can be represented as an XML schema including the namespace urn:mpeg:mpegI:gpcc:2020, and can be specified as shown in Table 12 below. Table 12 .

[0182] G-PCC preselection can be signaled. For example, preselection elements (e.g., as defined in DASH) can be used to signal G-PCC preselection in the MPD, where the list of identifiers (IDs) for the @preselectionComponents attribute includes, for example, the primary adaptation set ID for the volumetric media and (e.g., subsequently) the AdaptationSet ID corresponding to the G-PCC component. In the example, the @codecs attribute for the preselection could be set to "gpc1", indicating, for example, that the preselected media is a geometry-based point cloud. Preselection can be signaled, for example, using preselection elements within a periodic element, and / or using a preselection descriptor at the adaptation set level.

[0183] Figure 9 An example of grouping G-PCC components in MPD using preselection is shown. Figure 9 An exemplary DASH configuration is shown for grouping G-PCC components that may belong to volumetric media (e.g., a single volumetric media) within an MPEG-DASH MPD file.

[0184] Multiple versions of G-PCC media can be signaled. In the example, independent preselections can be used, for example, to signal multiple versions of the same point cloud media. Preselections representing alternative versions of the same geometry-based point cloud media can include, for example, G-PCC descriptors with the same @gpcId value. One or more (e.g., at most one) G-PCC descriptors can exist, for example, at the preselection level. Preselection can be an alternative preselection of selectable components. The list of IDs for the @preselectionComponents attribute can include the primary adaptive set ID following the IDs of the remaining component adaptive sets, for example, when the @codecs attribute is set to "gpc1".

[0185] Figure 10 An example is shown of grouping multiple versions of the G-PCC component in MPD using pre-selection. Figure 10 An example of a DASH configuration for grouping multiple versions of a G-PCC component that may belong to a single point cloud within an MPEG-DASH MPD file is shown. Preselection descriptors can be used, for example, to signal the grouping / association. Table 13 shows an example of using preselection to signal multiple versions of the G-PCC component in the MPD. Table 13 .

[0186] In the example, the G-PCC component AdaptationSet of the point cloud or the Representation of the AdaptationSet can use, for example, the @dependencyId attribute to list the identifiers of the main AdaptationSet and / or Representation. For example, in the case of combining fragments from the AdaptationSet of the point cloud component to decode fragments in the main AdaptationSet to reconstruct the point cloud, a dependency (e.g., intrinsic dependency) may exist.

[0187] In the example, G-PCC tile pre-selection can be implemented. When multiple tile tracks are used to carry G-PCC content, the master adaptive set can signal the G-PCC base track data. The tile component adaptive set can signal the G-PCC geometry and / or attribute tile track data.

[0188] Preselection can be signaled in MPD using preselection elements as described herein. In one example, the preselection's @codecs attribute could be set to "gpt1", indicating, for example, that the preselection medium is a collection of geometry-based point cloud tiles. Preselection can also be signaled using preselection elements within periodic elements, as described herein. Preselection can also be signaled using preselection descriptors at the tile component adaptive set level.

[0189] Preselection elements can include a list of IDs for the `@preselectionComponents` property. For G-PCC tile preselections, the list of `@preselectionComponents` property IDs can include the geometric tile component adaptive set ID, followed by the corresponding attribute tile component adaptive set ID. For example, the `@dependencyId` property, which signals in the adaptive set's representation, can be used to identify the representation of the primary adaptive set corresponding to the selected geometric tile component adaptive set's representation.

[0190] G-PCC tile preselections (e.g., each G-PCC tile preselection) may include one or more GPCCTileId descriptors. This allows identification of the tiles referenced in each preselection. When no GPCCTileId descriptor exists, tiles belonging to a G-PCC tile preselection can be identified by: finding the adaptive set of geometry tile components from the list of IDs in the @preselectionComponents property; and examining the list of tile IDs from the GPCCComponent descriptors present in the adaptive set of geometry tile components.

[0191] Figure 11 An example of G-PCC content with multiple tile tracks is shown. Figure 11An exemplary DASH configuration can be provided. G-PCC content may include geometric components and one or more attribute components (e.g., three attribute components). In this example, the G-PCC bitstream includes six tiles grouped into two tile sets. The first tile set includes tiles 1, 2, and 3, and the second tile set includes tiles 4, 5, and 6. Components for each tile set may be available in two different versions (e.g., encoded with different qualities). Component versions for each tile set (e.g., per component version) may be carried in separate G-PCC tile tracks within the ISOBMFF container file. The MPD file may include a tile component adaptive set for each component of the two tile sets. The tile component adaptive set (e.g., per tile component adaptive set) may include two representations (e.g., one representation per version of the component). Two preselections may be used in the MPD to signal the presence of the two tile sets in the G-PCC bitstream.

[0192] Table 14 shows an example of how DASH MPD files use multiple tile tracks and preselected descriptors to signal G-PCC content. Table 14 .

[0193] In the example, independent preselection can be used to signal media data, for example, when multiple point cloud media are available. Preselection representing geometry-based point cloud media data may include a G-PCC descriptor with a unique @gpcId value. One or more (e.g., at most one) G-PCC descriptors may exist, for example, at the preselection level. The following may exist: the primary adaptation set ID; the ID in the list of adaptation set IDs for the @preselectionComponents attribute (e.g., the first ID); and / or (e.g., subsequent) the AdaptationSet ID corresponding to the point cloud component. The point cloud may be identified, for example, using a unique value of the @gpcId attribute that can be defined in the G-PCC descriptor.

[0194] Signaling can be used to notify G-PCC tile groups. In the example, tile bounding box information can be signaled (e.g., using the GPCCTileInventory descriptor), for example, in the case of multiple tiles in a geometry-based point cloud. The GPCCTileInventory descriptor can be a SupplementalProperty element, which, for example, has the @schemeIdUri attribute (e.g., set to "urn:mpeg:mpegI:gpcc:2020:gptl"). The GPCCTileInventory descriptor can exist at the adaptive set level of the master adaptive set for G-PCC media, for example, in the case where the G-PCC media is tiled. Table 15 shows examples of elements and attributes for the GPCCTileInventory descriptor. Table 15 .

[0195] Table 16 shows an example of an XML schema for the GPCCTileInventory descriptor. Data types for various elements and attributes for the GPCCTileInventory descriptor can be defined, for example, based on an XML schema such as the exemplary schema shown in Table 16. Table 16 .

[0196] The client can select (e.g., choose first) a tile ID from the tile inventory bounding box information stored in the MPD, for example, in a case where the client will stream tiled G-PCC component data from the server. In the example, a G-PCC component with the selected tile_id can be streamed to the client.

[0197] Dynamic G-PCC tile inventory information can be signaled. When parameter set data and / or tile inventory information change dynamically, information about such changes can be carried in samples of the G-PCC base orbit. In the example, when multiple tiles exist in a geometry-based point cloud and tile bounding box information changes dynamically, the tile bounding box information (e.g., along with the parameter set data) can be carried in a media clip of the representation of the master adaptive set.

[0198] Spatial regions can be static. The characteristics of a spatial region and / or the mapping between regions and G-PCC tiles can be signaled (e.g., using GPCC3DRegions descriptors), for example, when the 3D spatial region is static. 3D spatial regions can be static, for example, where the position and size of the regions (e.g., each region) do not change over time. GPCC3DRegions descriptors can be SupplementalProperty elements, which, for example, have an @schemeIdUri attribute equal to "urn:mpeg:mpegI:gpcc:2020:gpsr". GPCC3DRegions descriptors (e.g., a single GPCC3DRegions descriptor) can exist, for example, at the adaptive set level and / or representation level in the main G-PCC track, or at a pre-selected level for geometry-based volumetric media content.

[0199] The @value attribute of the GPCC3DRegions descriptor may not exist. The GPCC3DRegions descriptor may include elements and / or attributes (such as those specified in Table 17). Table 17 shows examples of elements and attributes associated with the GPCC3DRegions descriptor. Table 17 .

[0200] Data types for various elements and attributes for GPCC3DRegions descriptors can be defined by schemas (such as the XML schemas shown in Table 18). Table 18 shows an example of an XML schema for GPCC3DRegions descriptors. Table 18 .

[0201] In the example, the characteristics of a spatial region and / or the mapping between the spatial region and the corresponding AdaptationSet of the G-PCC component can be signaled (e.g., using GPCC3DRegions descriptors), for example, when the 3D spatial region is static and tile inventory information is unavailable. A GPCC3DRegions descriptor can be a SupplementalProperty element, which, for example, has an @schemeIdUri attribute equal to "urn:mpeg:mpegI:gpcc:2020:gpsr". GPCC3DRegions descriptors (e.g., a single GPCC3DRegions descriptor) can exist, for example, at the AdaptationSet level and / or Representation level in the main G-PCC track, or at a pre-selected level for geometry-based volumetric media content.

[0202] The @value attribute of the GPCC3DRegions descriptor may not exist. The GPCC3DRegions descriptor may include elements and attributes (such as those specified in Table 19). Table 19 shows examples of elements and attributes for the GPCC3DRegions descriptor. Table 19 .

[0203] Data types for various elements and attributes for the GPCC3DRegions descriptor can be defined, for example, based on a schema (such as an XML schema). Table 20 shows an example of an XML schema for the GPCC3DRegions descriptor. Table 20 .

[0204] In the example, the GPCC3DRegionId descriptor can be used to signal the mapping between spatial regions and their corresponding AdaptationSets of G-PCC components, for example, when the 3D spatial regions are static. This descriptor can be a SupplementalProperty element with a @schemeIdUri attribute equal to "urn:mpeg:mpegI:gpcc:2020:gp3rid". A single GPCC3DRegionId descriptor can exist at the AdaptationSet level of each G-PCC component (e.g., per G-PCC component). The GPCC3DRegionId may not exist, for example, if the gpsr.spatialRegion@asIds attribute exists in the GPCC3DRegions descriptor.

[0205] The @value attribute of the GPCC3DRegionId descriptor may not exist. The GPCC3DRegionId descriptor may include one or more attributes as shown in Table 21. Table 21 .

[0206] The data type for the attribute can be provided as in the XML schema. The following shows the XML schema for the GPCC3DRegionID descriptor. This schema can be represented as an XML schema including the namespace urn:mpeg:mpegI:gpcc:2020, and is specified in Table 22. Table 22 .

[0207] In the example, the GPCCComponent descriptor can be used to signal the mapping between spatial regions and their corresponding AdaptationSets of G-PCC components, for example, when the 3D spatial region is static. The GPCCComponent descriptor can include elements and attributes as defined in Table 23. The GPCC3DRegionID descriptor may not exist, for example, if the @region_Id attribute exists in the GPCCComponent descriptor. Table 23 .

[0208] An example of the XML schema for the GPCCComponent descriptor is shown in Table 24 below. Table 24 .

[0209] In one example, one or more spatial regions can be dynamic. In the case of dynamic 3D partitions, timing metadata tracks used to signal the position and / or size of the 3D regions (e.g., each 3D region) on the display timeline can be carried in a separate adaptive set with a single representation. The timing metadata tracks can be associated with (e.g., linked) the main G-PCC adaptive set. The attributes used may include the @associationId attribute and the @associationType value, which includes a 4CC "gpdr" for the corresponding AdaptationSet or Representation.

[0210] Streaming client behavior can be based on signaling, such as signaling of one or more descriptors. The DASH client can be guided, for example, by information provided in the MPD. The following is an example of client behavior for streaming geometry-based point cloud compressed content, for example using the signaling examples disclosed herein. Exemplary client behavior may assume, for example, using a G-PCC descriptor to signal the association between the component adaptation set and the master point cloud adaptation set.

[0211] A streaming client may issue (e.g., first) an HTTP request, where the destination is set to a content server. The streaming client may download an MPD file from the content server. The client may parse the MPD file, for example, to generate an in-memory representation of the corresponding XML elements in the MPD file.

[0212] Streaming clients can inspect pre-selected elements at the cycle level (e.g., with @codecs attributes set to "gpc1" or "gpt1") to identify available G-PCC media content within a cycle.

[0213] For example, you can identify an AdaptationSet (e.g., all AdaptationSets) belonging to the point cloud content represented by a preselected element by examining the list of IDs in the preselected @preselectionComponents property. The primary AdaptationSet may include an @id value that is equal to the @id value of the first ID in that list.

[0214] A streaming client can identify the number of unique point clouds, for example, by checking the G-PCC descriptor of the AdaptationSet and / or groups of AdaptationSets with the same @gpcId value in their G-PCC descriptors, as versions of the same content.

[0215] A streaming client (e.g., by examining the GPCCComponent descriptor of the remaining AdaptationSet referenced in the ID list of the @preselectionComponent attribute) can identify the components of the point cloud and can map each component (e.g., each component) to its corresponding AdaptationSet. In the example, there may be more than one point cloud component in the AdaptationSet.

[0216] For example, based on point cloud content that the user might be interested in streaming, an AdaptationSet group with a @gpcId value existing in the G-PCC descriptor can be selected from the ID list of the @preselectionComponent attribute, corresponding to the desired content. The streaming client can select an AdaptationSet group with supported versions (e.g., supported resolutions), for example, if multiple preselected descriptors with the same @gpcId value exist. A unique AdaptationSet group can be selected, for example, if multiple preselected descriptors with the same @gpcId value do not exist.

[0217] A client can begin streaming point clouds, for example, by downloading an initialization segment for the master-adaptive set, including a parameter set for initializing the G-PCC decoder. The initialization segment for the encoded component stream can be downloaded and / or cached in memory.

[0218] A streaming client can begin downloading time-aligned media segments from a master adaptive set and / or a component adaptive set (e.g., in parallel via HTTP). In the example, the downloaded segments can be stored in an in-memory segment buffer. Time-aligned media segments can be removed from their respective buffers and / or concatenated with their corresponding initialization segments.

[0219] Media containers (e.g., ISO Basic Media File Format (ISOBMFF)) can be parsed to, for example, extract basic stream information and construct a G-PCC bitstream, which can then be passed to a G-PCC decoder.

[0220] Client behavior for streaming G-PCC media with multiple tiles can be implemented, for example, using MPD signaling as described herein. The client (e.g., a streaming client) can issue HTTP requests and / or download MPD files from a content server. The client can parse the MPD file to generate corresponding in-memory representations of the XML elements in the MPD file.

[0221] For example, a client can inspect an AdaptationSet element with an @codecs attribute set to "gpcb", and can inspect pre-selected elements at the cycle level with an @codecs attribute set to "gpt1", for example, to identify G-PCC tiled media content available in the cycle.

[0222] When G-PCC tile-based media content is present, the client can identify tiles of interest in the point cloud bitstream, for example, based on the client's current viewport. The client can resolve GPCC3DRegions descriptors and / or find the corresponding tiles within the viewport. When 3D partitioning is dynamic, media segments with a timed metadata adaptive set can be downloaded, carrying the position and / or size of 3D regions (e.g., each 3D spatial region) on the display timeline. 3D regions within the viewport can be identified. The corresponding tiles belonging to that region can be identified.

[0223] If a tile of interest is found, the client can select the preselected elements containing the tile, for example, by resolving the GPCCTileId descriptor present in the preselected elements (e.g., each preselected element). The @tile_Ids attribute in the GPCCTileId descriptor lists the available tiles. Preselected elements containing the tile of interest can be selected. Preselections (e.g., other preselections) can be ignored.

[0224] When the GPCCTileId descriptor is unavailable, tiles present in the preselection element can be identified, for example, by finding the adaptive set of geometric tile components from the list of ids in the @preselectionComponents attribute; and by finding the list of tile IDs from the GPCCComponent descriptors present in the adaptive set of geometric tile components. If the tiles of interest exist in the preselection element, the client can then select the preselection.

[0225] Based on the selected preselection, the tile component adaptive set group to be used for downloading media clips can be identified from the list of IDs in the `@preselectionComponents` property. In the example, the `@preselectionComponents` list may include the geometric tile component adaptive set IDs. The `@preselectionComponents` list may also include the tile component adaptive set IDs of the remaining components. The primary adaptive set ID may not be present in the `@preselectionComponents` list. For example, the primary adaptive set ID can be identified using the `@dependencyId` property in the representation of the geometric tile component adaptive set.

[0226] A client can begin streaming a point cloud, for example, by downloading an initialization fragment from the master-adaptive set. The initialization fragment may include a set of parameters for initializing the G-PCC decoder.

[0227] Initialization fragments for the encoded component stream (e.g., if present) can be downloaded and / or cached in memory.

[0228] The streaming client can download time-aligned media segments from the geometric tile component adaptive set and / or the associated attribute tile component adaptive set. Downloads can be performed in parallel via HTTP, and the downloaded segments can be stored in an in-memory segment buffer.

[0229] Time-aligned media segments can be removed from their respective buffers and / or concatenated with their respective initialization segments.

[0230] Media containers (e.g., ISOBMFF) can be parsed to extract basic stream information. Media containers can be constructed, and the resulting bitstream can be passed to a G-PCC decoder.

[0231] This document describes numerous embodiments. Features of the embodiments may be provided individually or in any combination across various claim classes and types. Furthermore, embodiments may include one or more of the features, devices, or aspects described individually or in any combination across various claim classes and types, such as, for example, any of the following.

[0232] Decoders (such as exemplary decoder 300) are configured to: receive, decode, and interpret signals indicating elements, attributes, and metadata associated with point cloud components (e.g., as described herein); identify point cloud streams and their component substreams within a Media Presentation Descriptor (MPD); identify versions of point clouds and / or their components; decode the MPD to identify master adaptive sets and other adaptive sets, thereby identifying G-PCC components in geometry-based point cloud compression (G-PCC) content; decode the MPD to identify the type of point cloud components in the adaptive sets or representations; decode the MPD to identify one or more pre-selections; and perform further processing on the MPD. Line decoding to identify one or more versions of G-PCC media; MPD decoding to identify one or more G-PCC tile groups; MPD decoding to identify one or more tile IDs for G-PCC components in an adaptive set; MPD decoding to identify one or more characteristics of a spatial region and the mapping between the region and G-PCC tiles, the characteristics of the spatial region and the mapping between the region and the corresponding adaptive set of the G-PCC component, and / or the mapping between the spatial region and the corresponding adaptive set of the G-PCC component; MPD decoding to identify timing metadata tracks for dynamic spatial regions; etc.

[0233] Decoding tools and techniques, including one or more of entropy decoding, inverse quantization, inverse transform, and differential decoding, are used to implement the examples described herein in the decoder.

[0234] Encoders (such as exemplary encoder 200) are configured to: for example, generate, encode, and transmit signals indicating elements, attributes, and metadata associated with point cloud components (e.g., as described herein); encode MPDs to indicate point cloud streams and their component substreams; encode MPDs to indicate master adaptive sets and other adaptive sets, thereby supporting the identification of G-PCC components in geometry-based point cloud compression (G-PCC) content; encode MPDs to support the identification of the type of point cloud components in adaptive sets or representations; encode MPDs to identify one or more pre-selections; and encode MPDs to support the identification of G-PCC media. Identification of one or more versions of the body; encoding the MPD to support the identification of one or more G-PCC tile groups; encoding the MPD to support the identification of one or more tile IDs of G-PCC components in the adaptive set; encoding the MPD to support the identification of one or more characteristics of a spatial region and the mapping between the region and G-PCC tiles, the characteristics of a spatial region and the mapping between the region and the corresponding adaptive set of G-PCC components, and / or the mapping between a spatial region and the corresponding adaptive set of G-PCC components; decoding the MPD to identify timing metadata tracks for dynamic spatial regions; etc.

[0235] Encoding tools and techniques, including one or more of quantization, entropy coding, inverse quantization, inverse transform, and differential coding, are used to implement the examples described herein in the encoder.

[0236] Insert syntax elements into the signaling, for example, to enable the decoder to recognize instructions associated with executing any of the examples described herein.

[0237] Insert syntax elements into the signaling, for example, to enable the encoder to generate or encode instructions associated with performing any of the examples described herein.

[0238] Bitstreams or signals may include one or more syntax elements from the described syntax elements or variations thereof associated with performing any of the examples described herein.

[0239] A method, process, apparatus, medium for storing instructions, medium for storing data, or signal for creating and / or transmitting and / or receiving and / or decoding a bit stream or signal comprising one or more of the said syntax elements or variations thereof.

[0240] A method, process, apparatus, medium for storing instructions, medium for storing data, or signal for creating and / or transmitting and / or receiving and / or decoding according to any of the examples described herein.

[0241] TVs, set-top boxes, cellular phones, tablets, or other electronic devices that perform adaptive streaming of geometry-based point clouds (such as point cloud component substreams) in point cloud streaming services according to any of the examples described herein.

[0242] TVs, set-top boxes, cellular phones, tablets, or other electronic devices that perform adaptive streaming of geometry-based point clouds (such as point cloud component substreams) according to any of the examples described herein, and display (e.g., using a monitor, screen, or other type of display) the resulting images in a point cloud streaming service.

[0243] TVs, set-top boxes, cellular phones, tablets, or other electronic devices that select (e.g., using a tuner) channels to receive signals including encoded images, and perform adaptive streaming of geometry-based point clouds (such as point cloud component substreams) in point cloud streaming services according to any of the examples described herein.

[0244] TVs, set-top boxes, cellular phones, tablets, or other electronic devices that receive (e.g., using an antenna) signals including encoded images via radio, and perform adaptive streaming of geometry-based point clouds (such as point cloud component substreams) in point cloud streaming services according to any of the examples described herein.

[0245] Although features and elements have been described above in specific combinations, those skilled in the art will understand that each feature or element may be used alone or in any combination with other features and elements. Furthermore, the methods described herein may be implemented in a computer program, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted via a wired or wireless connection) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media (such as internal hard disks and removable disks), magneto-optical media, and optical media (such as CD-ROM disks and digital versatile optical discs (DVDs)). A processor associated with the software may be used to implement a radio frequency transceiver for a WTRU, UE, terminal, base station, RNC, or any host computer.

Claims

1. An apparatus comprising: The processor is configured as follows: Receive Media Presentation Description (MPD) files from the content server; Identify scheme identifiers from MPD files that are associated with spatial regions of geometry-based point cloud compression (G-PCC) content; Based on the scheme identifier, a G-PCC region descriptor associated with the G-PCC adaptive set is determined, wherein the G-PCC region descriptor at least indicates the mapping between the spatial region and a set of G-PCC tiles of the G-PCC content; Based on the G-PCC region descriptor, determine the region identifier of the spatial region and the G-PCC tile identifier of the G-PCC tile in the group of G-PCC tiles in the spatial region; Use the G-PCC tile identifier associated with the G-PCC tile to request G-PCC content; and Receive the requested G-PCC content.

2. The device according to claim 1, wherein, The G-PCC region descriptor includes at least indications of three coordinates of a reference point of the bounding box associated with the spatial region, and indications of the corresponding lengths of the bounding box along three axes.

3. The device according to claim 1, wherein, The G-PCC region descriptor is signaled at the adaptive set level or representation level within the GPCC adaptive set.

4. The device according to claim 3, wherein, The G-PCC adaptive set includes one or more representations.

5. The device according to claim 4, wherein, One or more representations include at least one of bit rate, resolution, number of channels, or quality level.

6. The device according to claim 1, wherein, The G-PCC region descriptor uses preselected elements to signal at the preselection level of G-PCC content.

7. The device according to claim 1, wherein, The spatial region is a dynamic spatial region, wherein the position and size of the spatial region are signaled in an adaptive set having a single representation and are associated with the main G-PCC adaptive set using an association identifier attribute and an association type value.

8. The device according to claim 7, wherein, The location and size of the spatial region are signaled in the timed metadata track.

9. The device according to claim 7, wherein, The location and size of the spatial area are signaled in the display timeline.

10. The device according to claim 1, wherein, The spatial region is associated with the viewport.

11. A method comprising: Receive Media Presentation Description (MPD) files from the content server; Identify scheme identifiers from MPD files that are associated with spatial regions of geometry-based point cloud compression (G-PCC) content; Based on the scheme identifier, a G-PCC region descriptor associated with the G-PCC adaptive set is determined, wherein the G-PCC region descriptor at least indicates the mapping between the spatial region and a set of G-PCC tiles of the G-PCC content; Based on the G-PCC region descriptor, determine the region identifier of the spatial region and the G-PCC tile identifier of the G-PCC tile in the group of G-PCC tiles in the spatial region; Use the G-PCC tile identifier associated with the G-PCC tile to request G-PCC content; and Receive the requested G-PCC content.

12. The method according to claim 11, wherein, The G-PCC region descriptor includes at least indications of three coordinates of a reference point of the bounding box associated with the spatial region, and indications of the corresponding lengths of the bounding box along three axes.

13. The method according to claim 11, wherein, The G-PCC region descriptor is signaled at the adaptive set level or representation level within the GPCC adaptive set.

14. The method according to claim 13, wherein, The G-PCC adaptive set includes one or more representations.

15. The method according to claim 14, wherein, One or more representations include at least one of bit rate, resolution, number of channels, or quality level.

16. The method according to claim 11, wherein, The G-PCC region descriptor uses preselected elements to signal at the preselection level of G-PCC content.

17. The method according to claim 11, wherein, The spatial region is a dynamic spatial region, wherein the position and size of the spatial region are signaled in an adaptive set having a single representation and are associated with the main G-PCC adaptive set using an association identifier attribute and an association type value.

18. The method according to claim 17, wherein, The location and size of the spatial region are signaled in the timed metadata track.

19. The method of claim 17, wherein, The location and size of the spatial area are signaled in the display timeline.

20. The method according to claim 11, wherein, The spatial region is associated with the viewport.

Images