Adaptive streaming of geometry-based point clouds

Abstract

Systems, methods, and means for receiving content related to geometry-based point clouds are disclosed. A client device may receive a Media Presentation Description (MPD) file from a content server. A set of pre-selected elements may be identified from the MPD file. One or more adaptation sets associated with at least one pre-selected element among the set of pre-selected elements may be identified. The adaptation set(s) may be represented by an attribute associated with one of the pre-selected elements. A geometry-based point cloud compression (GPCC) tile identifier associated with a viewport may be determined. The GPCC tile identifier may be determined based on a first descriptor received from the MPD file. One or more adaptation sets associated with the GPCC tile identifier may be selected using a second descriptor. A point cloud component associated with the selected one or more adaptation sets may be requested. The point cloud component may be received.

Description

Technology Field

[0001] Cross-reference regarding related applications

[0002] This application claims the benefit of U.S. Provisional Application No. 63 / 042,481 filed June 22, 2020, and U.S. Provisional Application No. 63 / 084,758 filed September 29, 2020, the disclosures of which are incorporated herein by reference in their entirety. Background Technology

[0003] Video coding systems can be used to compress digital video signals, for example, to reduce the storage and / or transmission bandwidth required for such signals. Video coding systems may include block-based systems, such as wavelet-based systems, object-based systems, and / or block-based hybrid video coding systems. Video coding systems can support the coding and storage of point clouds. However, systems may lack suitable mechanisms to support point cloud data streaming over a network.

[0004] Systems, methods, and means for adaptive streaming of visual media content, such as geometry-based point clouds, are disclosed. Elements, attributes, and metadata associated with point cloud components may be signaled to enable, for example, streaming clients to identify point cloud streams and their component substreams in a media presentation descriptor (MPD), and to enable streaming clients to select versions of the point cloud and / or its components based, for example, on client support. In examples, a streaming client may utilize guidance (e.g., a signal signaled in an MPD file) to make decisions regarding different representations of the point cloud content. For example, the signal may indicate which set of representations across different point cloud components constitutes a certain quality level. Components of the point cloud content may be divided into a number of tiles or tile portions. Clients may stream specific tile portions of geometric components (e.g., selected tile portions) based, for example, on bandwidth availability (e.g., instead of streaming all point cloud data). Point cloud component tile bitstreams may be available in different adaptation sets, for example, in which an adaptation set (e.g., each adaptation set) can represent point cloud component tiles.

[0005] Geometry-based point cloud compression (G-PCC) components can be signaled over dynamic streaming over HTTP (DASH). For example, G-PCC components can be signaled using a DASH manifest file or an MPD file. In these examples, G-PCC components (e.g., each G-PCC component) can be represented in a DASH MPD file as an adaptation set (e.g., a separate adaptation set). An adaptation set (e.g., a main adaptation set) can serve as the main access point for G-PCC content. In these examples, an adaptation set (e.g., a single adaptation set) can be signaled for each component per resolution.

[0006] G-PCC component descriptors can be signaled to enable, for example, a streaming client to identify the type of point cloud component in an adaptation set and / or representation. G-PCC descriptors can enable a streaming client to distinguish different point cloud streams present in an MPD file. The streaming client can identify component streams for each point cloud stream.

[0007] G-PCC preselection can be signaled (e.g., in MPD) with a list of IDs including, for example, the identifier (ID) of the main adaptation set for volumetric media and the IDs of adaptation sets corresponding to G-PCC components. Preselection can be signaled, for example, using a preselection element within a period element and / or using a preselection descriptor at the adaptation set level.

[0008] Multiple versions of a G-PCC medium can be signaled. Multiple versions of the same point cloud medium can be signaled, for example, using distinct preselections. Preselections representing alternative versions of the same geometry-based point cloud medium may, for example, include G-PCC descriptors having the same attribute values.

[0009] One or more G-PCC tiles can be signaled. For example, if multiple tiles exist within a geometry-based point cloud, tile bounding box information can be signaled. The client can select a tile ID from the tile inventory bounding box information (e.g., in MPD) to stream, for example, tiled G-PCC component data.

[0010] Clients can identify tile IDs for point cloud components in an adaptation set by, for example, checking G-PCC component descriptors. G-PCC tile ID descriptors can be signaled to enable, for example, a streaming client to distinguish G-PCC tile streams.

[0011] For example, in geometry-based volumetric media content, when 3D spatial regions are static, the properties of the spatial regions and / or mappings between the regions and G-PCC tiles can be signaled. For example, when 3D spatial regions are static and / or tile inventory information is not available, the properties of the spatial regions and / or mappings between the regions and corresponding adaptation sets of G-PCC components can be signaled (e.g., using G-PCC 3D region descriptors). Mappings between spatial regions and corresponding adaptation sets of G-PCC components can be signaled (e.g., by G-PCC region ID descriptors or G-PCC component descriptors).

[0012] A timed-metadata track (e.g., representing the positions and / or dimensions of 3D regions on a presentation timeline) can be signaled, for example, in an adaptation set associated with the main G-PCC adaptation set(s) and having a representation (e.g., for dynamic spatial regions).

[0013] Streaming client behavior can be based on signaling. DASH clients can be guided, for example, by information provided to the MPD.

[0014] Systems, methods, and means for receiving content related to geometric point clouds are disclosed. In examples, a Media Presentation Description (MPD) file may be received, for example, from a content server. A set of preselected elements may be identified from the MPD file. One or more adaptation sets associated with at least one preselected element among the set of preselected elements may be identified. For example, one or more adaptation sets may be represented by an attribute associated with one of the preselected elements.

[0015] A geometry-based point cloud compression (GPCC) tile identifier associated with a viewport can be determined. For example, the GPCC tile identifier can be determined based on a first descriptor received from an MPD file. In examples, the first descriptor may be a three-dimensional (3D) region descriptor. The 3D region descriptor may include a region location associated with the 3D region, one or more region dimensions, and / or a set of tiles.

[0016] One or more adaptation sets associated with a GPCC tile identifier may be selected using a second descriptor. In examples, the second descriptor may be a component descriptor. A component descriptor may include a component type, attribute type, index, and / or set of tiles associated with a bitstream. A point cloud component associated with one or more selected adaptation sets may be requested. In examples, a point cloud component may be received.

[0017] Each feature disclosed anywhere in this specification may be described and implemented separately / individually and in any combination with any other feature disclosed in this specification and / or any feature(s) disclosed elsewhere that may be implicitly or explicitly referenced in this specification or otherwise fall within the scope of the subject matter disclosed in this specification. Brief explanation of the drawing

[0018] FIG. 1a is a system diagram illustrating an exemplary communication system in which one or more disclosed embodiments may be implemented. FIG. 1b is a system diagram illustrating an exemplary wireless transmit / receive unit (WTRU) that can be used within the communication system illustrated in FIG. 1a according to an embodiment. FIG. 1c is a system diagram illustrating an exemplary radio access network (RAN) and an exemplary core network (CN) that can be used within the communication system illustrated in FIG. 1a according to an embodiment. FIG. 1d is a system diagram illustrating additional exemplary RAN and additional exemplary CN that can be used within the communication system illustrated in FIG. 1a according to an embodiment. Figure 2 is a drawing illustrating an exemplary video encoder. Figure 3 is a diagram illustrating an example of a video decoder. FIG. 4 is a diagram illustrating an example of a system in which various suns and examples can be implemented. Figure 5 illustrates an example of a bitstream structure for geometry-based point cloud compression (G-PCC). Figure 6 illustrates an example of a sample structure when G-PCC geometry and attribute bitstreams are stored on a single track. Figure 7 illustrates an example of a multi-track G-PCC container. Figure 8 illustrates an exemplary Media Presentation Description (MPD) hierarchical data model. Figure 9 illustrates an example of grouping G-PCC components in MPD using preselection. Figure 10 illustrates an example of grouping multiple versions of G-PCC components in MPD using presets. FIG. 11 illustrates an example of G-PCC content having multiple tile tracks. Specific details for implementing the invention

[0019] Now, a detailed description of exemplary embodiments will be given with reference to the various drawings. While this description provides detailed examples of possible implementations, it should be noted that the details are intended to be illustrative and do not limit the scope of this application in any way.

[0020] FIG. 1a is a drawing illustrating an exemplary communication system (100) in which one or more disclosed embodiments may be implemented. The communication system (100) may be a multiple access system that provides content such as voice, data, video, messaging, broadcasting, etc. to multiple wireless users. The communication system (100) may enable multiple wireless users to access such content through the sharing of system resources, including wireless bandwidth. For example, the communication systems (100) may employ one or more channel access methods such as CDMA (code division multiple access), TDMA (time division multiple access), FDMA (frequency division multiple access), OFDMA (orthogonal FDMA), SC-FDMA (single-carrier FDMA), ZT UW DTS-s OFDM (zero-tail unique-word DFT-Spread OFDM), UW-OFDM (unique word OFDM), resource block filtered OFDM, FBMC (filter bank multicarrier), etc.

[0021] As illustrated in FIG. 1a, the communication system (100) may include wireless transceiver units (WTRUs) (102a, 102b, 102c, 102d), a RAN (104 / 113), a CN (106 / 115), a public switched telephone network (PSTN) (108), the Internet (110), and other networks (112), but it will be understood that the disclosed embodiments consider any number of WTRUs, base stations, networks, and / or network elements. Each of the WTRUs (102a, 102b, 102c, 102d) may be any type of device configured to operate and / or communicate in a wireless environment. For example, WTRUs (102a, 102b, 102c, 102d)—any of which may be referred to as a “station” and / or a “STA”—may be configured to transmit and / or receive wireless signals and may include user equipment (UE), mobile stations, fixed or mobile subscriber units, subscription-based units, pagers, cellular phones, personal digital assistants (PDAs), smartphones, laptops, netbooks, personal computers, wireless sensors, hotspots or Mi-Fi devices, Internet of Things (IoT) devices, watches or other wearables, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in industrial and / or automated processing chain contexts), consumer electronics devices, devices operating on commercial and / or industrial wireless networks, etc. Any of the WTRUs (102a, 102b, 102c, and 102d) may be interchangeably referred to as UE.

[0022] Communication systems (100) may also include a base station (114a) and / or a base station (114b). Each of the base stations (114a, 114b) may be any type of device configured to wirelessly interface with at least one of the WTRUs (102a, 102b, 102c, 102d) to facilitate access to one or more communication networks, such as CN (106 / 115), the Internet (110), and / or other networks (112). For example, the base stations (114a, 114b) may be a base transceiver station (BTS), Node B, eNode B, Home Node B, Home eNode B, gNB, NR Node B, site controller, access point (AP), wireless router, etc. Base stations (114a, 114b) are each depicted as a single element, but it will be understood that base stations (114a, 114b) may include any number of interconnected base stations and / or network elements.

[0023] A base station (114a) may be part of a RAN (104 / 113) which may also include other base stations and / or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), and relay nodes. A base station (114a) and / or a base station (114b) may be configured to transmit and / or receive radio signals on one or more carrier frequencies, which may be referred to as a cell (not shown). These frequencies may be within the licensed spectrum and the unlicensed spectrum, or a combination of the licensed spectrum and the unlicensed spectrum. A cell may provide coverage for radio services for a specific geographical area that may be relatively fixed or may change over time. A cell may be further subdivided into cell sectors. For example, a cell associated with a base station (114a) may be subdivided into three sectors. Accordingly, in one embodiment, the base station (114a) may include three transceivers, that is, one for each sector of the cell. In an embodiment, the base station (114a) may employ MIMO (multiple-input multiple-output) technology and may use multiple transceivers for each sector of the cell. For example, beamforming may be used to transmit and / or receive signals in desired spatial directions.

[0024] Base stations (114a, 114b) can communicate with one or more of WTRUs (102a, 102b, 102c, 102d) through an air interface (116) which may be any suitable radio communication link (e.g., RF (radio frequency), microwave, centimeter wave, micrometer wave, IR (infrared), UV (ultraviolet), visible light, etc.). The air interface (116) may be established using any suitable radio access technology (RAT).

[0025] More specifically, as described above, the communication system (100) may be a multiple access system and may employ one or more channel access schemes such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, base stations (114a) and WTRUs (102a, 102b, 102c) within a RAN (104 / 113) may implement wireless technology such as Universal Mobile Telecommunications System (UMTS) terrestrial radio access (UTRA) that can establish air interfaces (115 / 116 / 117) using wideband CDMA (WCDMA). WCDMA may include communication protocols such as High-Speed ​​Packet Access (HSPA) and / or advanced HSPA (HSPA+). HSPA may include High Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High Speed ​​Uplink (UL) Packet Access (HSUPA).

[0026] In an embodiment, the base station (114a) and WTRUs (102a, 102b, 102c) can implement wireless technology such as E-UTRA (Evolved UMTS Terrestrial Radio Access) which can establish an air interface (116) using, for example, LTE (Long Term Evolution) and / or LTE-A (LTE-Advanced) and / or LTE-A Pro (LTE-Advanced Pro).

[0027] In an embodiment, the base station (114a) and WTRUs (102a, 102b, 102c) can implement wireless technology such as NR wireless access that can establish an air interface (116) using New Radio (NR).

[0028] In an embodiment, the base station (114a) and WTRUs (102a, 102b, 102c) may implement a number of wireless access technologies. For example, the base station (114a) and WTRUs (102a, 102b, 102c) may implement LTE wireless access and NR wireless access together, for example, using dual connectivity (DC) principles. Accordingly, the air interface used by the WTRUs (102a, 102b, 102c) may be characterized by a number of types of wireless access technologies and / or transmitters transmitted to / from a number of types of base stations (e.g., eNB and gNB).

[0029] In another embodiment, the base station (114a) and WTRUs (102a, 102b, 102c) can implement wireless technologies such as IEEE 802.11 (i.e., WiFi (Wireless Fidelity)), IEEE 802.16 (i.e., WiMAX (Worldwide Interoperability for Microwave Access)), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, IS-2000 (Interim Standard 2000), IS-95 (Interim Standard 95), IS-856 (Interim Standard 856), GSM (Global System for Mobile communications), EDGE (Enhanced Data rates for GSM Evolution), GERAN (GSM EDGE), etc.

[0030] The base station (114b) of FIG. 1a may be, for example, a wireless router, a home Node B, a home eNode B, or an access point, and any suitable RAT may be used to facilitate wireless connectivity in a localized area such as, for example, a workplace, a home, a vehicle, a campus, an industrial facility, an air corridor (e.g., for use by drones), a road, etc. In one embodiment, the base station (114b) and WTRUs (102c, 102d) may establish a wireless local area network (WLAN) by implementing a wireless technology such as IEEE 802.11. In one embodiment, the base station (114b) and WTRUs (102c, 102d) may implement a wireless technology such as IEEE 802.15 to establish a wireless personal area network (WPAN). In another embodiment, the base station (114b) and WTRUs (102c, 102d) may utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.) to establish a picocell or femtocell. As illustrated in FIG. 1a, the base station (114b) may have direct access to the Internet (110). Thus, the base station (114b) may not be required to access the Internet (110) via the CN (106 / 115).

[0031] The RAN (104 / 113) may communicate with a CN (106 / 115), which may be any type of network configured to provide voice, data, applications, and / or VoIP (voice over internet protocol) services to one or more of the WTRUs (102a, 102b, 102c, 102d). The data may have various quality of service (QoS) requirements, such as different throughput requirements, latency requirements, error tolerance limit requirements, reliability requirements, data throughput requirements, mobility requirements, etc. The CN (106 / 115) may provide call control, billing services, mobile location-based services, prepaid calls, internet connectivity, video distribution, etc., and / or perform high-level security functions such as user authentication. Although not illustrated in FIG. 1a, it will be understood that the RAN (104 / 113) and / or CN (106 / 115) may communicate directly or indirectly with other RANs employing the same RAT or a different RAT as the RAN (104 / 113). For example, in addition to being connected to the RAN (104 / 113) which may be using NR wireless technology, the CN (106 / 115) may also communicate with another RAN (not illustrated) employing GSM, UMTS, CDMA 2000, WiMAX, E-UTRA, or WiFi wireless technology.

[0032] CN (106 / 115) may also serve as a gateway for WTRUs (102a, 102b, 102c, 102d) to access the PSTN (108), the Internet (110), and / or other networks (112). The PSTN (108) may include circuit-switched telephone networks that provide plain old telephone service (POTS). The Internet (110) may include a global system of interconnected computer networks and devices using common communication protocols such as TCP, User Datagram Protocol (UDP), and / or IP in the transmission control protocol / internet protocol (TCP / IP) suite. The networks (112) may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, networks (112) may include another CN connected to one or more RANs that can employ the same RAT or a different RAT as the RAN (104 / 113).

[0033] Some or all of the WTRUs (102a, 102b, 102c, 102d) within the communication system (100) may include multi-mode capabilities (e.g., the WTRUs (102a, 102b, 102c, 102d) may include multiple transceivers to communicate with different wireless networks via different wireless links). For example, the WTRU (102c) illustrated in FIG. 1a may be configured to communicate with a base station (114a) capable of employing cellular-based wireless technology and a base station (114b) capable of employing IEEE 802 wireless technology.

[0034] FIG. 1b is a system diagram illustrating an exemplary WTRU (102). As illustrated in FIG. 1b, the WTRU (102) may include, in particular, a processor (118), a transceiver (120), a transceiver element (122), a speaker / microphone (124), a keypad (126), a display / touchpad (128), non-removable memory (130), removable memory (132), a power supply (134), a GPS (global positioning system) chipset (136), and / or other peripherals (138). It will be understood that the WTRU (102) may include any sub-combination of the aforementioned elements while remaining consistent with the embodiment.

[0035] The processor (118) may be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. The processor (118) may perform signal coding, data processing, power control, input / output processing, and / or any other function that enables the WTRU (102) to operate in a wireless environment. The processor (118) may be coupled to a transceiver (120) that may be coupled to a transceiver element (122). FIG. 1b illustrates the processor (118) and the transceiver (120) as separate components, but it will be known that the processor (118) and the transceiver (120) can be integrated together within an electronic package or chip.

[0036] The transceiver element (122) may be configured to transmit a signal to a base station (e.g., base station (114a)) or receive a signal therefrom through the air interface (116). For example, in one embodiment, the transceiver element (122) may be an antenna configured to transmit and / or receive an RF signal. In an embodiment, the transceiver element (122) may be an emitter / detector configured to transmit and / or receive an IR, UV, or visible light signal, for example. In another embodiment, the transceiver element (122) may be configured to transmit and / or receive both an RF signal and an optical signal. It will be known that the transceiver element (122) may be configured to transmit and / or receive any combination of radio signals.

[0037] Although the transmitting and receiving element (122) is illustrated as a single element in FIG. 1b, the WTRU (102) may include any number of transmitting and receiving elements (122). More specifically, the WTRU (102) may employ MIMO technology. Accordingly, in one embodiment, the WTRU (102) may include two or more transmitting and receiving elements (122) (e.g., multiple antennas) for transmitting and receiving wireless signals through the air interface (116).

[0038] The transceiver (120) may be configured to modulate a signal to be transmitted by the transceiver element (122) and to demodulate a signal received by the transceiver element (122). As previously described, the WTRU (102) may have multimode capabilities. Accordingly, the transceiver (120) may include multiple transceivers to enable the WTRU (102) to communicate through multiple RATs, such as NR and IEEE 802.11, for example.

[0039] The processor (118) of the WTRU (102) may be coupled to a speaker / microphone (124), a keypad (126), and / or a display / touch pad (128) (e.g., a liquid crystal display (LCD) display unit or an organic light-emitting diode (OLED) display unit) and may receive user input data from them. The processor (118) may also output user data to the speaker / microphone (124), the keypad (126), and / or the display / touch pad (128). Additionally, the processor (118) may access information from any type of suitable memory, such as non-removable memory (130) and / or removable memory (132), and store data therein. Non-removable memory (130) may include random-access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory (132) may include a subscriber identification module (SIM) card, a memory stick, a secure digital (SD) memory card, etc. In other embodiments, the processor (118) may access information from memory that is not physically located on the WTRU (102), such as a server or home computer (not shown), and store data therein.

[0040] The processor (118) can receive power from the power source (134) and can be configured to distribute and / or control the power to other components within the WTRU (102). The power source (134) may be any suitable device for supplying power to the WTRU (102). For example, the power source (134) may include one or more batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0041] The processor (118) may also be coupled to a GPS chipset (136) which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU (102). In addition to or instead of information from the GPS chipset (136), the WTRU (102) may receive location information from a base station (e.g., base stations (114a, 114b)) via the air interface (116) and / or determine its location based on the timing of signals received from two or more nearby base stations. The WTRU (102) will know that it may obtain location information by any suitable location determination method while still being consistent with the embodiment.

[0042] The processor (118) may be further coupled to other peripherals (138) which may include one or more software and / or hardware modules that provide additional features, functions and / or wired or wireless connectivity. For example, peripherals (138) may include an accelerometer, an electronic compass, a satellite transceiver, a digital camera (for photos and / or videos), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands-free headset, a Bluetooth® module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an internet browser, a virtual reality and / or augmented reality (VR / AR) device, an activity tracker, etc. Peripherals (138) may include one or more sensors, which may include a gyroscope, an accelerometer, a Hall effect sensor, a magnetometer, an orientation sensor, a proximity sensor, a temperature sensor, a time sensor; a geolocation sensor; It may be one or more of an altimeter, optical sensor, touch sensor, magnetometer, barometer, gesture sensor, biometric sensor, and / or humidity sensor.

[0043] The WTRU (102) may include a full-duplex radio in which the transmission and reception of some or all of the signals (associated with specific subframes for both the UL (e.g., for transmission) and the downlink (e.g., for reception) may be concomitant and / or simultaneous. The full-duplex radio may include an interference management unit that reduces and / or substantially eliminates self-interference through hardware (e.g., a choke) or through signal processing through a processor (e.g., a separate processor (not shown) or processor (118)). In an embodiment, the WTRU (102) may include a half-duplex radio for the transmission and reception of some or all of the signals (associated with specific subframes for the UL (e.g., for transmission) or the downlink (e.g., for reception).

[0044] FIG. 1c is a system diagram illustrating a RAN (104) and a CN (106) according to an embodiment. As previously described, the RAN (104) may employ E-UTRA radio technology to communicate with WTRUs (102a, 102b, 102c) through an air interface (116). The RAN (104) may also communicate with the CN (106).

[0045] The RAN (104) may include eNode-Bs (160a, 160b, 160c), but it will be recognized that the RAN (104) may include any number of eNode-Bs while remaining consistent with the embodiment. Each of the eNode-Bs (160a, 160b, 160c) may include one or more transceivers to communicate with WTRUs (102a, 102b, 102c) via the air interface (116). In one embodiment, the eNode-Bs (160a, 160b, 160c) may implement MIMO technology. Thus, the eNode-B (160a) may use multiple antennas to transmit radio signals to the WTRU (102a), for example, and / or receive radio signals from it.

[0046] Each of the eNode-Bs (160a, 160b, 160c) may be associated with a specific cell (not shown) and may be configured to handle wireless resource management decisions, handover decisions, scheduling of users in the UL and / or DL, etc. As shown in FIG. 1c, the eNodeBs (160a, 160b, 160c) may communicate with each other via an X2 interface.

[0047] The CN (106) illustrated in FIG. 1c may include a mobility management entity (MME) (162), a serving gateway (SGW) (164), and a packet data network (PDN) gateway (or PGW) (166). Although each of the aforementioned elements is depicted as part of the CN (106), it will be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0048] The MME (162) can be connected to each of the eNode-Bs (162a, 162b, 162c) within the RAN (104) via the S1 interface and can act as a control node. For example, the MME (162) may be responsible for authenticating users of the WTRUs (102a, 102b, 102c), enabling / disabling bearers, and selecting a specific serving gateway during the initial attach of the WTRUs (102a, 102b, 102c). The MME (162) may provide control plane functions for switching between the RAN (104) and other RANs (not shown) using other wireless technologies such as GSM and / or WCDMA.

[0049] The SGW (164) can be connected to each of the eNode Bs (160a, 160b, 160c) within the RAN (104) via the S1 interface. The SGW (164) can generally route and forward user data packets to and from the WTRUs (102a, 102b, 102c). The SGW (164) can perform other functions such as anchoring user planes during inter-eNode B handovers, triggering paging when DL data is available for the WTRUs (102a, 102b, 102c), and managing and storing the context of the WTRUs (102a, 102b, 102c).

[0050] The SGW (164) can be connected to a PGW (166) that can provide access to packet-switched networks, such as the Internet (110), to the WTRUs (102a, 102b, 102c) to facilitate communication between the WTRUs (102a, 102b, 102c) and IP-enabled devices.

[0051] CN (106) can facilitate communication with other networks. For example, CN (106) can provide WTRUs (102a, 102b, 102c) with access to circuit-switched networks, such as PSTN (108), to facilitate communication between WTRUs (102a, 102b, 102c) and traditional ground communication devices. For example, CN (106) may include or communicate with an IP gateway (e.g., an IMS (IP multimedia subsystem) server) that acts as an interface between CN (106) and PSTN (108). Additionally, CN (106) can provide WTRUs (102a, 102b, 102c) with access to other networks (112), which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0052] Although the WTRU is described as a wireless terminal in FIGS. 1a through 1d, it is considered that in certain representative embodiments, such a terminal may use wired communication interfaces with a communication network (e.g., temporarily or permanently).

[0053] In a typical embodiment, another network (112) may be a WLAN.

[0054] A WLAN in Basic Service Set (BSS) mode may have an Access Point (AP) for the BSS and one or more Stations (STAs) associated with the AP. The AP may have access or an interface to a Distribution System (DS) or another type of wired / wireless network that carries traffic to and / or from the BSS. Traffic originating from outside the BSS to the STAs may arrive through the AP and be forwarded to the STAs. Traffic originating from the STAs to destinations outside the BSS may be transmitted to the AP to be forwarded to their respective destinations. Traffic between STAs within the BSS may be transmitted through the AP; for example, a source STA may send traffic to the AP, and the AP may forward the traffic to the destination STA. Traffic between STAs within the BSS may be considered and / or referred to as peer-to-peer traffic. Peer-to-peer traffic can be transmitted between a source STA and a destination STA (e.g., directly between them) using a direct link setup (DLS). In certain representative embodiments, the DLS may use 802.11e DLS or 802.11z TDLS (tunneled DLS). A WLAN using Independent BSS (IBSS) mode may not have an AP, and STAs within or using IBSS (e.g., all STAs) may communicate directly with each other. The IBSS communication mode may sometimes be referred to herein as an "ad-hoc" communication mode.

[0055] When using an 802.11ac infrastructure operating mode or a similar operating mode, the AP can transmit a beacon on a fixed channel, such as the primary channel. The primary channel may have a fixed width (e.g., a bandwidth of 20 MHz) or a width dynamically set through signaling. The primary channel may be the operating channel of the BSS and may be used by STAs to establish a connection with the AP. In certain representative embodiments, Carrier Sense Multiple Access with Collision Avoidance (CSMA / CA) may be implemented, for example, in 802.11 systems. In the case of CSMA / CA, STAs including the AP (e.g., all STAs) can sense the primary channel. If the primary channel is sensed / detected and / or determined to be in use by a specific STA, that specific STA may be backed off. Only one STA (e.g., only one station) may transmit at any given time in a given BSS.

[0056] High Throughput (HT) STAs can use a 40 MHz wide channel for communication, for example, by combining an adjacent or non-adjacent 20 MHz channel and a main 20 MHz channel to form a 40 MHz wide channel.

[0057] Very High Throughput (VHT) STAs can support channels with widths of 20 MHz, 40 MHz, 80 MHz, and / or 160 MHz. 40 MHz and / or 80 MHz channels can be formed by combining adjacent 20 MHz channels. A 160 MHz channel can be formed by combining eight adjacent 20 MHz channels, or by combining two non-adjacent 80 MHz channels, which may be referred to as an 80+80 configuration. In the case of an 80+80 configuration, data can be passed through a segment parser capable of splitting the data into two streams after channel encoding. Inverse Fast Fourier Transform (IFFT) processing and time domain processing can be performed individually for each stream. The streams can be mapped to two 80 MHz channels, and the data can be transmitted by the transmitting STA. At the receiver of the receiving STA, the aforementioned operation for the 80+80 configuration can be reversed, and the combined data can be transmitted to the Medium Access Control (MAC).

[0058] Sub-1 GHz operating modes are supported by 802.11af and 802.11ah. Channel operating bandwidths and carriers are reduced in 802.11af and 802.11ah compared to those used in 802.11n and 802.11ac. 802.11af supports 5 MHz, 10 MHz, and 20 MHz bandwidths in the TV White Space (TVWS) spectrum, and 802.11ah supports 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz bandwidths using the non-TVWS spectrum. According to a representative embodiment, 802.11ah can support Meter Type Control / Machine-Type Communications, such as MTC devices within a macro coverage area. MTC devices may have limited capabilities, such as support for specific and / or limited bandwidths (e.g., support only for them). MTC devices may include batteries having a battery life exceeding a threshold (e.g., to maintain a very long battery life).

[0059] WLAN systems capable of supporting multiple channels and channel bandwidths, such as 802.11n, 802.11ac, 802.11af, and 802.11ah, include a channel that can be designated as the main channel. The main channel may have a bandwidth equal to the largest common operating bandwidth supported by all STAs within the BSS. The bandwidth of the main channel may be set and / or limited by the STA that supports the smallest bandwidth operating mode among all STAs operating in the BSS. In the example of 802.11ah, the main channel may be 1 MHz wide for STAs (e.g., MTC type devices) that support the 1 MHz mode (e.g., only that mode), even if the AP and other STAs within the BSS support 2 MHz, 4 MHz, 8 MHz, 16 MHz, and / or other channel bandwidth operating modes. Carrier sensing and / or Network Allocation Vector (NAV) settings may depend on the state of the main channel. If the main channel is in use, for example, due to transmission to an AP of a STA (which supports only 1 MHz operation mode), the entire available frequency band may be considered in use and available even if most of the frequency bands remain idle.

[0060] In the United States, the available frequency bands available for use by 802.11ah are 902 MHz to 928 MHz. In Korea, the available frequency bands are 917.5 MHz to 923.5 MHz. In Japan, the available frequency bands are 916.5 MHz to 927.5 MHz. The total available bandwidth for 802.11ah is 6 MHz to 26 MHz, depending on the country code.

[0061] FIG. 1d is a system diagram illustrating a RAN (113) and a CN (115) according to an embodiment. As mentioned above, the RAN (113) may employ NR radio technology to communicate with WTRUs (102a, 102b, 102c) through an air interface (116). The RAN (113) may also communicate with the CN (115).

[0062] It will be understood that the RAN (113) may include gNBs (180a, 180b, 180c), but the RAN (113) may include any number of gNBs while remaining consistent with the embodiment. Each of the gNBs (180a, 180b, 180c) may include one or more transceivers for communicating with the WTRUs (102a, 102b, 102c) via the air interface (116). In one embodiment, the gNBs (180a, 180b, 180c) may implement MIMO technology. For example, the gNBs (180a, 108b) may use beamforming to transmit signals to the gNBs (180a, 180b, 180c) and / or receive signals from them. Accordingly, the gNB (180a) may use multiple antennas to transmit radio signals to, for example, the WTRU (102a) and / or receive radio signals from it. In an embodiment, the gNBs (180a, 180b, 180c) may implement carrier aggregation technology. For example, the gNB (180a) may transmit multiple component carriers to the WTRU (102a) (not shown). A subset of these component carriers may be on unlicensed spectrum, while the remaining component carriers may be on licensed spectrum. In an embodiment, the gNBs (180a, 180b, 180c) may implement Coordinated Multi-Point (CoMP) technology. For example, the WTRU (102a) may receive coordinated transmissions from the gNB (180a) and the gNB (180b) (and / or the gNB (180c)).

[0063] WTRUs (102a, 102b, 102c) can communicate with gNBs (180a, 180b, 180c) using transmits associated with scalable numerology. For example, OFDM symbol intervals and / or OFDM subcarrier intervals may vary for different transmits, different cells, and / or different parts of the radio transmission spectrum. WTRUs (102a, 102b, 102c) can communicate with gNBs (180a, 180b, 180c) using subframes or transmission time intervals (TTIs) of various or scalable lengths (e.g., including varying numbers of OFDM symbols and / or lasting, varying absolute time lengths).

[0064] gNBs (180a, 180b, 180c) may be configured to communicate with WTRUs (102a, 102b, 102c) in a standalone configuration and / or a non-standalone configuration. In a standalone configuration, WTRUs (102a, 102b, 102c) may communicate with gNBs (180a, 180b, 180c) without also accessing other RANs (e.g., eNodeBs (160a, 160b, 160c)). In a standalone configuration, WTRUs (102a, 102b, 102c) may use one or more of the gNBs (180a, 180b, 180c) as mobility anchor points. In a standalone configuration, WTRUs (102a, 102b, 102c) can communicate with gNBs (180a, 180b, 180c) using signals within the unlicensed band. In a non-standalone configuration, WTRUs (102a, 102b, 102c) can also communicate with gNBs (180a, 180b, 180c) while communicating with another RAN, such as eNode-Bs (160a, 160b, 160c). For example, WTRUs (102a, 102b, 102c) can implement DC principles to communicate substantially simultaneously with one or more gNBs (180a, 180b, 180c) and one or more eNode-Bs (160a, 160b, 160c). In a non-standalone configuration, eNode-Bs (160a, 160b, 160c) can serve as mobility anchors for WTRUs (102a, 102b, 102c), and gNBs (180a, 180b, 180c) can provide additional coverage and / or throughput to service WTRUs (102a, 102b, 102c).

[0065] Each of the gNBs (180a, 180b, 180c) may be associated with a specific cell (not shown) and may be configured to handle wireless resource management decisions, handover decisions, scheduling of users in UL and / or DL, support for network slicing, duplex connectivity, interoperability between NR and E-UTRA, routing of user plane data to User Plane Functions (UPF) (184a, 184b), routing of control plane information to Access and Mobility Management Functions (AMF) (182a, 182b), etc. As shown in FIG. 1d, the gNBs (180a, 180b, 180c) may communicate with each other via an Xn interface.

[0066] The CN (115) illustrated in FIG. 1d may include at least one AMF (182a, 182b), at least one UPF (184a, 184b), at least one Session Management Function (SMF) (183a, 183b), and possibly a Data Network (DN) (185a, 185b). Although each of the aforementioned elements is depicted as part of the CN (115), it will be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0067] The AMF (182a, 182b) can be connected to one or more of the gNBs (180a, 180b, 180c) within the RAN (113) via the N2 interface and can act as a control node. For example, the AMF (182a, 182b) can be responsible for the authentication of users of the WTRUs (102a, 102b, 102c), support for network slicing (e.g., handling different PDU sessions with different requirements), selection of a specific SMF (183a, 183b), management of the registration area, termination of NAS signaling, mobility management, etc. Network slicing may be used by the AMF (182a, 182b) to customize CN support for WTRUs (102a, 102b, 102c) based on the types of services used by WTRUs (102a, 102b, 102c). For example, different network slices may be established for different use cases, such as services relying on URLLC (ultra-reliable low latency) access, services relying on eMBB (enhanced massive mobile broadband) access, services for machine type communication (MTC) access, etc. The AMF (162) may provide control plane functions for switching between the RAN (113) and other RANs (not shown) employing other radio technologies, such as LTE, LTE-A, LTE-A Pro and / or non-3GPP access technologies such as WiFi.

[0068] SMF (183a, 183b) can be connected to AMF (182a, 182b) within CN (115) via the N11 interface. SMF (183a, 183b) can also be connected to UPF (184a, 184b) within CN (115) via the N4 interface. SMF (183a, 183b) can select and control UPF (184a, 184b) and configure the routing of traffic through UPF (184a, 184b). SMF (183a, 183b) can perform other functions such as managing and assigning UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, and providing downlink data notifications. PDU session types can be IP-based, non-IP-based, Ethernet-based, etc.

[0069] UPF (184a, 184b) may be connected to one or more of gNBs (180a, 180b, 180c) in the RAN (113) via an N3 interface that can provide access to packet-switched networks, such as the Internet (110), to WTRUs (102a, 102b, 102c) to facilitate communication between WTRUs (102a, 102b, 102c) and IP-enabled devices. UPF (184, 184b) may perform other functions such as routing and forwarding packets, enforcing user plane policies, supporting multi-hom PDU sessions, handling user plane QoS, buffering downlink packets, and providing mobility anchoring.

[0070] CN (115) can facilitate communication with other networks. For example, CN (115) may include or communicate with an IP gateway (e.g., an IP multimedia subsystem (IMS) server) that acts as an interface between CN (115) and PSTN (108). Additionally, CN (115) may provide WTRUs (102a, 102b, 102c) with access to other networks (112), which may include other wired and / or wireless networks owned and / or operated by other service providers. In one embodiment, WTRUs (102a, 102b, 102c) can be connected to the local DN (185a, 185b) through the UPF (184a, 184b) via an N3 interface to the UPF (184a, 184b) and an N6 interface between the UPF (184a, 184b) and the local data network (DN) (185a, 185b).

[0071] In reference to FIGS. 1a through 1d and the corresponding description of FIGS. 1a through 1d, one or more or all of the functions described herein in connection with one or more of WTRU (102a through 102d), base station (114a, 114b), eNode-B (160a through 160c), MME (162), SGW (164), PGW (166), gNB (180a through 180c), AMF (182a, 182b), UPF (184a, 184b), SMF (183a, 183b), DN (185a, 185b) and / or any other device(s) described herein may be performed by one or more emulation devices (not shown). Emulation devices may be one or more devices configured to emulate one or all of the functions described herein. For example, emulation devices may be used to test other devices and / or simulate network and / or WTRU functions.

[0072] Emulation devices may be designed to implement one or more tests of other devices in laboratory environments and / or operator network environments. For example, one or more emulation devices may perform one or more or all functions while being fully or partially implemented and / or deployed as part of a wired and / or wireless communication network to test other devices within a communication network. One or more emulation devices may perform one or all functions while being temporarily implemented / deployed as part of a wired and / or wireless communication network. Emulation devices may be directly coupled to other devices for testing and / or may perform tests using over-the-air (OTA) wireless communications.

[0073] One or more emulation devices may perform one or more functions including all functions without being implemented or deployed as part of a wired and / or wireless communication network. For example, emulation devices may be used in test scenarios in a test laboratory and / or in a wired and / or wireless communication network that is not deployed (e.g., test) to implement testing of one or more components. One or more emulation devices may be test equipment. Direct RF coupling and / or wireless communication through an RF circuit (e.g., which may include one or more antennas) may be used by the emulation devices to transmit and / or receive data.

[0074] This application describes various aspects including tools, features, examples or embodiments, models, approaches, etc. Many of these aspects are described specifically and are often described in a manner that may sound restrictive in order to show at least individual characteristics. However, this is for the sake of clarity of description and does not limit the application or scope of the aspects. In practice, all different aspects may be combined and interchanged to provide additional aspects. Furthermore, aspects may also be combined and interchanged with aspects described in prior applications.

[0075] The embodiments described and considered in this application may be implemented in many different forms. While FIGS. 5 through 8 described herein may provide some embodiments, other embodiments are considered. The discussion of FIGS. 5 through 8 does not limit the scope of the embodiments. At least one of the embodiments generally relates to video encoding and decoding, and at least one other embodiment generally relates to transmitting a generated or encoded bitstream. These and other embodiments may be implemented as a method, an apparatus, a computer-readable storage medium storing instructions for encoding or decoding video data according to any of the described methods, and / or a computer-readable storage medium storing a bitstream generated according to any of the described methods.

[0076] In this application, the terms "reconstructed" and "decoded" may be used interchangeably, the terms "pixel" and "sample" may be used interchangeably, and the terms "image," "picture," and "frame" may be used interchangeably.

[0077] Various methods are described herein, and each method comprises one or more steps or actions to achieve the described method. Unless a specific order of steps or actions is required for the proper operation of the method, the order and / or use of specific steps and / or actions may be modified or combined. Additionally, terms such as “first,” “second,” etc., may be used in various embodiments to modify elements, components, steps, actions, etc., such as “first decoding” and “second decoding.” The use of such terms does not imply a sequencing of modified actions unless specifically required. Thus, in these examples, the first decoding does not need to be performed before the second decoding, and may occur, for example, before, during, or overlapping with the second decoding.

[0078] Various methods and other embodiments described in this application may modify modules of a video encoder (200) and a video decoder (300), e.g., pre-encoding processing (201), intra-prediction (260), entropy coding (245), and / or entropy decoding modules (330), intra-prediction (360), and post-decoding processing (385), as illustrated in FIGS. 2 and FIGS. 3, respectively (e.g., may be used to modify them). Furthermore, the subject matter disclosed herein presents embodiments not limited to VVC or HEVC and may be applicable to any type, format, or version of video coding, e.g., existing or future-developed, whether described as a standard or a recommendation, and to any extension of such standards and recommendations (e.g., including VVC and HEVC). Unless otherwise indicated or technically excluded, the embodiments described in this application may be used individually or in combination.

[0079] For example, various numeric values ​​such as minimum value ranges and maximum value ranges (e.g., 0 to 1, 0 to N, or 0 to 255), bit values ​​for indications or decisions, default values, ID numbers (e.g., in the case of adaptive IDs), etc. are used in the examples described in this application. These and other specific values ​​are for illustrative purposes only and the described embodiments are not limited to these specific values.

[0080] FIG. 2 is a drawing illustrating an exemplary video encoder. Variations of the exemplary encoder (200) are considered, but the encoder (200) is described below for clarity without describing all expected variations.

[0081] Before encoding, the video sequence may undergo pre-encoding processing (201), for example, applying a color conversion to the input color picture (e.g., converting from RGB 4:4:4 to YCbCr 4:2:0), or remapping the input picture components to obtain a signal distribution more resilient to compression (e.g., using histogram equalization of one of the color components). Metadata may be associated with the pre-processing and attached to the bitstream.

[0082] In the encoder (200), the picture is encoded by encoder elements as described below. The picture to be encoded is partitioned (202) and processed, for example, into units of coding units (CUs). Each unit is encoded, for example, using an intra or inter mode. When a unit is encoded in intra mode, it performs intra prediction (260). In inter mode, motion estimation (275) and compensation (270) are performed. The encoder determines whether to use intra mode or inter mode to encode the unit (205), and indicates the intra / inter decision, for example, by a prediction mode flag. Prediction residuals are calculated, for example, by subtracting the predicted block from the original image block (210).

[0083] Next, the predicted residuals are transformed (225) and quantized (230). The quantized transformation coefficients, as well as motion vectors and other syntax elements, are entropy-coded (245) to output a bitstream. The encoder can skip the transformation and apply quantization directly to the untransformed residual signal. The encoder can skip both the transformation and the quantization, that is, the residuals are coded directly without the application of transformation or quantization processes.

[0084] The encoder decodes the encoded block to provide a reference for additional predictions. Quantized transform coefficients are inversely quantized (240) and inversely transformed (250) to decode the prediction residuals. An image block is reconstructed by combining the decoded prediction residuals and the predicted block (255). In-loop filters (265) are applied to the reconstructed picture to perform deblocking / SAO (sample adaptive offset) filtering, for example, to reduce encoding artifacts. The filtered image is stored in a reference picture buffer (280).

[0085] FIG. 3 is a diagram illustrating an example of a video decoder. In an exemplary decoder (300), a bitstream is decoded by decoder elements as described below. The video decoder (300) generally performs a decoding pass opposite to the encoding pass as described in FIG. 2. The encoder (200) may also perform video decoding generally as part of encoding video data. For example, the encoder (200) may perform one or more of the video decoding steps presented herein. The encoder reconstructs the decoded images, for example, to maintain synchronization with the decoder with respect to one or more of the following: reference pictures, entropy coding contexts, and other decoder-related state variables.

[0086] In particular, the input to the decoder includes a video bitstream that can be generated by a video encoder (200). The bitstream is first entropy decoded to obtain transform coefficients, motion vectors, and other coded information (330). Picture partition information indicates how the picture is partitioned. Thus, the decoder can partition the picture according to the decoded picture partitioning information (335). The transform coefficients are inversely quantized (340) and inversely transformed (350) to decode the prediction residuals. An image block is reconstructed by combining the decoded prediction residuals and the predicted block (355). The predicted block can be obtained from an intra prediction (360) or a motion-compensated prediction (i.e., an inter prediction) (375) (370). In-loop filters (365) are applied to the reconstructed image. The filtered image is stored in a reference picture buffer (380).

[0087] The decoded picture may further undergo post-decoding processing (385), for example, inverse remapping or inverse color conversion (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) which performs the inverse of the remapping process performed in pre-encoding processing (201). Post-decoding processing may use metadata derived from pre-encoding processing and signaled in the bitstream.

[0088] FIG. 4 is a drawing illustrating an example of a system in which the various embodiments and examples described herein may be implemented. The system (400) may be implemented as a device comprising various components described below and configured to perform one or more of the embodiments described herein. Examples of such devices include, but are not limited to, various electronic devices, such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital TV receivers, personal video recording systems, connected home appliances, and servers. The elements of the system (400) may be implemented alone or in combination in a single integrated circuit (IC), multiple ICs, and / or separate components. For example, in at least one example, the processing and encoder / decoder elements of the system (400) are distributed across multiple ICs and / or separate components. In various embodiments, the system (400) is communically coupled to one or more other systems or other electronic devices, for example, via a communication bus or via dedicated input and / or output ports. In various embodiments, the system (400) is configured to implement one or more of the embodiments described in this document.

[0089] The system (400) includes at least one processor (410) configured to execute instructions loaded therein to implement, for example, the various embodiments described in this document. The processor (410) may include an embedded memory, an input / output interface, and various other circuit components as known in the art. The system (400) includes at least one memory (420) (e.g., a volatile memory device and / or a non-volatile memory device). The system (400) includes a storage device (440) that may include non-volatile memory and / or volatile memory, including but not limited to electrically eraseable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash, magnetic disk drive and / or optical disk drive. The storage device (440) may include, as non-limiting examples, an internal storage device, an attached storage device (including a removable storage device and a non-removable storage device), and / or a network-accessible storage device.

[0090] The system (400) includes, for example, an encoder / decoder module (430) configured to process data to provide encoded video or decoded video, and the encoder / decoder module (430) may include its own processor and memory. The encoder / decoder module (430) represents a module(s) that may be included in a device to perform encoding and / or decoding functions. As is known, the device may include one or both of the encoding and decoding modules. Additionally, the encoder / decoder module (430) may be implemented as a separate element of the system (400) or may be integrated within the processor (410) as a combination of hardware and software, as is known to those skilled in the art.

[0091] Program code to be loaded into a processor (410) or an encoder / decoder (430) to perform the various aspects described in this document may be stored in a storage device (440) and subsequently loaded into memory (420) for execution by the processor (410). According to various embodiments, one or more of the processor (410), memory (420), storage device (440), and encoder / decoder module (430) may store one or more of the various items during the execution of the processes described in this document. These stored items may include, but are not limited to, input video, decoded video or parts of decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of expressions, formulas, operations, and operation logic.

[0092] In some embodiments, memory within the processor (410) and / or encoder / decoder module (430) is used to store instructions and to provide working memory for processing required during encoding or decoding. However, in other embodiments, memory outside the processing device (e.g., the processing device may be either the processor (410) or the encoder / decoder module (430)) is used for one or more of these functions. The external memory may be memory (420) and / or storage device (440), e.g., dynamic volatile memory and / or non-volatile flash memory. In some embodiments, the external non-volatile flash memory is used, for example, to store the operating system of the television. In at least one embodiment, high-speed external dynamic volatile memory, such as RAM, is used as working memory for video coding and decoding operations, such as, for example, MPEG-2 (MPEG refers to Moving Picture Experts Group, MPEG-2 is also referred to as ISO / IEC 13818, 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC refers to High Efficiency Video Coding, also known as H.265 and MPEG-H Part 2), or VVC (Versatile Video Coding, a new standard being developed by the Joint Video Experts Team (JVET)).

[0093] Inputs to the elements of the system (400) may be provided through various input devices as shown in block (445). Such input devices include, but are not limited to, (i) an RF portion that receives a radio frequency (RF) signal transmitted wirelessly by, for example, a broadcaster, (ii) a component (COMP) input terminal (or a set of COMP input terminals), (iii) a Universal Serial Bus (USB) input terminal, and / or (iv) a High Definition Multimedia Interface (HDMI) input terminal. Other examples not shown in FIG. 4 include composite video.

[0094] In various embodiments, the input devices of block (445) have their respective associated input processing elements as known in the art. For example, the RF portion may be associated with elements suitable for (i) selecting a desired frequency (also referred to as selecting a signal or band-limiting a signal to a band of frequencies), (ii) down-converting the selected signal, (iii) band-limiting it again to a narrower band of frequencies to select a signal frequency band that may be referred to as a channel in some embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired stream of data packets. The RF portion of various embodiments includes one or more elements for performing these functions, such as frequency selectors, signal selectors, band-limiters, channel selectors, filters, down-converters, demodulators, error correctors, and demultiplexers. The RF section may include a tuner that performs these various functions, for example, including down-converting the received signal to a lower frequency (e.g., intermediate frequency or near-field baseband frequency) or to the baseband. In one set-top box embodiment, the RF section and its associated input processing element perform frequency selection by receiving, filtering, down-converting, and filtering the RF signal transmitted through a wired (e.g., cable) medium to a desired frequency band. Various embodiments rearrange the order of the aforementioned (and other) elements, remove some of these elements and / or add other elements that perform similar or different functions. Adding elements may include inserting elements between existing elements, such as inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0095] Additionally, USB and / or HDMI terminals may include their own interface processors for connecting the system (400) to other electronic devices via the USB and / or HDMI connections. It should be understood that various aspects of input processing, for example, Reed-Solomon error correction, may be implemented, for example, within a separate input processing IC or within the processor (410) as needed. Similarly, aspects of USB or HDMI interface processing may be implemented, for example, within a separate interface IC or within the processor (410). Demodulation, error correction, and demultiplexed streams are provided to various processing elements, including a processor (410) and an encoder / decoder (430), which operate in combination with memory and storage elements to process data streams as needed, for example, for presentation on an output device.

[0096] Various elements of the system (400) may be provided within an integrated housing. Within the integrated housing, the various elements may be interconnected using an internal bus as known in the art, which includes a suitable connection arrangement (425), for example, an inter-IC (I2C) bus, wiring, and a printed circuit board, and may transmit data between them.

[0097] The system (400) includes a communication interface (450) that enables communication with other devices through a communication channel (460). The communication interface (450) may include, but is not limited to, a transceiver configured to transmit and receive data through the communication channel (460). The communication interface (450) may include, but is not limited to, a modem or a network card, and the communication channel (460) may be implemented, for example, within a wired and / or wireless medium.

[0098] In various embodiments, data is streamed to or otherwise provided to the system (400) using a wireless network, such as a Wi-Fi network, e.g., IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). Wi-Fi signals in these embodiments are received through a communication channel (460) and a communication interface (450) adapted for Wi-Fi communications. The communication channel (460) in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top communications. Other embodiments provide the streamed data to the system (400) using a set-top box that transmits data via the HDMI connection of the input block (445). Still other embodiments provide the streamed data to the system (400) using the RF connection of the input block (445). As described above, various embodiments provide data in a non-streaming manner. In addition, various embodiments use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0099] The system (400) can provide output signals to various output devices, including a display (475), speakers (485), and other peripheral devices (495). In various embodiments, the display (475) includes one or more of, for example, a touchscreen display, an organic light-emitting diode (OLED) display, a curved display, and / or a foldable display. The display (475) may be for a television, tablet, laptop, mobile phone, or other device. The display (475) may also be integrated with other components (for example, as in a smartphone) or separate (for example, an external monitor for a laptop). In various examples of embodiments, other peripheral devices (495) include one or more of a standalone digital video disc (or digital multifunction disc) (for both terms, a DVR), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices (495) that provide functions based on the output of the system (400). For example, a disc player performs the function of playing the output of the system (400).

[0100] In various embodiments, control signals are communicated between the system (400) and the display (475), speakers (485), or other peripheral devices (495) using signaling such as AV.Link, Consumer Electronics Control (CEC), or other communication protocols that enable device-to-device control with or without user intervention. Output devices may be communicably coupled to the system (400) through dedicated connections via their respective interfaces (470, 480, 490). Alternatively, output devices may be connected to the system (400) using a communication channel (460) via a communication interface (450). The display (475) and speakers (485) may be integrated into a single unit with other components of the system (400) in an electronic device, for example, such as a television. In various embodiments, the display interface (470) includes a display driver, for example, a timing controller (T Con) chip.

[0101] The display (475) and speakers (485) may alternatively be separate from one or more of the other components, for example, if the RF portion of the input (445) is part of a separate set-top box. In various embodiments where the display (475) and speakers (485) are external components, the output signal may be provided through dedicated output connections, such as HDMI ports, USB ports, or COMP outputs.

[0102] The embodiments may be performed by computer software implemented by the processor (410), by hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory (420) may be any type suitable for the technical environment and, as a non-limiting example, may be implemented using any suitable data storage technology such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. The processor (410) may be any type suitable for the technical environment and, as a non-limiting example, may include one or more of microprocessors, general-purpose computers, special-purpose computers, and multi-core architecture-based processors.

[0103] Various embodiments involve decoding. "Decoding" as used in this application may include all or part of processes performed on a received encoded sequence to produce a final output suitable for display, for example. In various embodiments, such processes may include one or more of the processes typically performed by a decoder, for example, entropy decoding, inverse quantization, inverse transformation, and differential decoding.In various embodiments, such processes are also, or alternatively, processes performed by a decoder of the various embodiments described in this application, such as receiving, decoding, and interpreting signals (e.g., as described herein) representing elements, attributes, and metadata associated with point cloud components; identifying point cloud streams and their component substreams within a Media Presentation Descriptor (MPD); identifying versions of point clouds and / or their components; decoding the MPD to identify a main adaptation set and other adaptation sets to identify G-PCC components in geometry-based point cloud compression (G-PCC) content; decoding the MPD to identify types of point cloud components in an adaptation set or representation; decoding the MPD to identify one or more preselections; decoding the MPD to identify one or more versions of a G-PCC medium; decoding the MPD to identify one or more G-PCC tile groups; Decoding the MPD to identify one or more tile IDs for G-PCC components in an adaptation set; decoding the MPD to identify one or more properties of spatial regions and mappings between regions and G-PCC tiles, properties of spatial regions and mappings between regions and corresponding adaptation sets of G-PCC components, and / or mappings between spatial regions and corresponding adaptation sets of G-PCC components; decoding the MPD to identify timed metadata tracks for dynamic spatial regions; etc.

[0104] As additional embodiments, in one example, decoding may refer to entropy decoding, in another embodiment, decoding may refer to differential decoding, and in yet another embodiment, decoding may refer to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to specifically indicate a subset of operations or to generally indicate a broader decoding process will be clear based on the context of the specific descriptions and is expected to be well understood by those skilled in the art.

[0105] Various embodiments may involve encoding. In a manner similar to the above discussion regarding decoding, encoding as used in this application may include all or part of processes performed, for example, on an input video sequence to generate an encoded bitstream. In various embodiments, these processes typically include one or more of processes performed by an encoder, for example, partitioning, differential encoding, transformation, quantization, and entropy encoding.In various embodiments, such processes are also, or alternatively, processes performed by an encoder of the various embodiments described in this application, such as generating, encoding, and transmitting signals (e.g., as described herein) representing elements, attributes, and metadata associated with point cloud components; encoding an MPD to represent point cloud streams and their component substreams; encoding an MPD to represent a main adaptation set and other adaptation sets to support the identification of G-PCC components in geometry-based point cloud compression (G-PCC) content; encoding an MPD to support the identification of types of point cloud components in an adaptation set or representation; encoding an MPD to identify one or more preselections; encoding an MPD to support the identification of one or more versions of a G-PCC medium; encoding an MPD to support the identification of one or more G-PCC tile groups; encoding an MPD to support the identification of one or more tile IDs for G-PCC components in an adaptation set; Encoding the MPD to support the identification of one or more characteristics of spatial regions and mappings between regions and G-PCC tiles, mappings between characteristics of spatial regions and regions and corresponding adaptation sets of G-PCC components, and / or mappings between spatial regions and corresponding adaptation sets of G-PCC components; decoding the MPD to identify timed metadata tracks for dynamic spatial regions; etc.

[0106] As additional examples, in one embodiment, encoding may refer to entropy encoding, in another embodiment, encoding may refer to differential encoding, and in yet another embodiment, encoding may refer to a combination of differential encoding and entropy encoding. Whether the phrase “encoding process” is intended to specifically refer to a subset of operations or to generally refer to an encoding process in a broader sense will be clear based on the context of the specific descriptions and is expected to be well understood by those skilled in the art.

[0107] Note that syntax elements as used herein, such as those shown in Tables 1 through 23 and otherwise shown in the discussions or drawings presented herein, are descriptive terms. As such, they do not exclude the use of other syntax element names.

[0108] It must be understood that when a drawing is presented as a flowchart, it also provides a block diagram of the corresponding device. Similarly, it must be understood that when a drawing is presented as a block diagram, it also provides a flowchart of the corresponding method / process.

[0109] During the encoding process, a balance or trade-off between rate and distortion is generally considered, given the often-present constraints on computational complexity. Rate-distortion optimization is typically formulated as minimizing the rate-distortion function, which is a weighted sum of rate and distortion. There are different approaches to solving the rate-distortion optimization problem. For example, approaches may be based on extensive testing of all encoding options, including all considered modes or coding parameter values, utilizing a complete evaluation of the coding cost and associated distortion of the reconstructed signal after coding and decoding. Faster approaches may be used to reduce encoding complexity, particularly by utilizing the calculation of approximated distortion based on predicted or predicted residual signals rather than the reconstructed ones. For instance, a combination of these two approaches may be used by applying approximated distortion for some of the possible encoding options and full distortion for others. Other approaches may evaluate a subset of the possible encoding options. More generally, many approaches use any of the various techniques to perform optimization, but optimization is not necessarily a complete evaluation of both coding cost-related distortions.

[0110] The embodiments and aspects described herein may be implemented, for example, in a method or process, a device, a software program, a data stream, or a signal. Even if discussed in the context of a single form of the embodiment (e.g., discussed only as a method), embodiments of the features discussed may be implemented in other forms (e.g., a device or a program). A device may be implemented, for example, in suitable hardware, software, and firmware. For example, the methods may be implemented in a processor, which generally refers to processing devices, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, computers, mobile phones, portable / personal digital assistants (PDAs), and other devices that facilitate the communication of information between end users.

[0111] References to "one embodiment," "one example," "one embodiment," or "one embodiment," as well as other variations thereof, imply that specific features, structures, characteristics, etc. described in relation to the embodiment are included in at least one embodiment. Accordingly, the appearance of phrases such as "in one embodiment," "in one embodiment," "in one example," "in one embodiment," or "in one embodiment," as well as any other variations appearing in various places throughout this application, do not necessarily all refer to the same embodiment or example.

[0112] Additionally, the present application may refer to "determining" various types of information. Determining information may include, for example, estimating information, calculating information, predicting information, or retrieving information from memory. Acquiring may include receiving, extracting, configuring, generating, and / or determining.

[0113] Additionally, the present application may refer to "accessing" various information. Accessing information may include, for example, receiving information, retrieving information (for example, from memory), storing information, moving information, copying information, calculating information, determining information, predicting information, or estimating information.

[0114] Additionally, the present application may refer to various types of information as "receiving." Receiving is intended to be a broad term, similar to "accessing." Receiving information may include, for example, accessing information, or retrieving information (for example, from memory). Furthermore, "receiving" is typically involved in some way during, for example, an operation of storing information, an operation of processing information, an operation of transmitting information, an operation of moving information, an operation of copying information, an operation of erasing information, an operation of calculating information, an operation of determining information, an operation of predicting information, or an operation of estimating information.

[0115] For example, it should be understood that in the following cases of “A / B”, “A and / or B”, and “at least one of A and B”, the use of any of “ / ”, “and / or”, and “at least one” is intended to include the selection of the first enumerated option (A) alone, or the selection of the second enumerated option (B) alone, or the selection of both options (A and B). As another example, in the cases of “A, B and / or C” and “at least one of A, B and C”, these phrases are intended to include the selection of the first enumerated option (A) alone, or the selection of the second enumerated option (B) alone, or the selection of the third enumerated option (C) alone, or the selection of the first and second enumerated options (A and B) alone, or the selection of the first and third enumerated options (A and C) alone, or the selection of the second and third enumerated options (B and C) alone, or the selection of all three options (A, B and C). This can be extended to many items as described in this specification, as will be apparent to those skilled in the art and to this specification.

[0116] Additionally, as used herein, the term “signal” refers to indicating something to a corresponding decoder in particular. For example, in some embodiments, the encoder signals (e.g., to the decoder) MPD, adaptation set, representation, preselection, G-PCC components, among descriptors, in particular G-PCCComponent descriptor, G-PCC descriptor or essential attribute descriptor, supplementary attribute descriptor, G-PCC tile inventory descriptor, G-PCC static space region descriptor, GPCCTileId descriptor, GPCC3DRegionID descriptor, elements and attributes, metadata, schemas, etc. (e.g., as disclosed herein, including Tables 1 through 23). In this way, in one embodiment, the same parameter may be used on both the encoder side and the decoder side. Accordingly, for example, the encoder may transmit a specific parameter to the decoder (explicit signaling) so that the decoder can use the same specific parameter. Conversely, if the decoder already possesses not only the specific parameter but also others, signaling may be used without transmission so that the decoder simply knows and selects the specific parameter (implicit signaling). By avoiding the transmission of any actual functions, bit saving is realized in various embodiments. It should be understood that signaling can be achieved in various ways. For example, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder in various embodiments. The preceding relates to the verb form of the word "signal," but the word "signal" may be used herein as a noun.

[0117] As is apparent to those skilled in the art, embodiments may generate various signals formatted to convey information that may be stored or transmitted, for example. For example, the information may include instructions for performing a method or data generated by one of the described embodiments. For example, a signal may be formatted to convey a bitstream of the described embodiment. Such a signal may be formatted, for example, as an electromagnetic wave (e.g., using the radio frequency portion of the spectrum) or as a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information conveyed by the signal may be, for example, analog or digital information. As is known, the signal may be transmitted over various different wired or wireless links. The signal may be stored on a processor-readable medium.

[0118] 3D point clouds can represent immersive media (e.g., can be used to represent them). A point cloud may contain a set of points represented in three-dimensional (3D) space. In examples, a point (e.g., each point) may be associated with one or more coordinates representing the point's location and / or one or more attributes (e.g., point color, transparency, acquisition time, laser reflectivity, material properties, etc.). Point clouds may be captured or placed using, for example, one or more cameras, depth sensors, and / or light detection and ranging (LiDAR) laser scanners. Point clouds may contain multiple points. In examples, a point (e.g., each point) may be represented by a set of coordinates (e.g., x, y, z coordinates) mapped in 3D space. Points may be generated based on the sampling of objects. In examples, the number of points within a point cloud may be millions or billions. Point clouds can be used to reconstruct one or more objects and / or scenes. Point clouds can be represented and / or compressed, for example, to store and / or transmit point cloud data (e.g., to store and / or transmit efficiently). Point cloud compression can support lossy and / or lossless coding (e.g., encoding or decoding) of the geometric coordinates and / or attributes of the point cloud. Point clouds can be deployed to support various applications (e.g., telepresence, virtual reality (VR), and / or large-scale dynamic 3D maps).For example, libraries for mesh and point cloud compression can support the compression of vertex positions, normal values, colors, texture coordinates, and other common vertex attributes to improve, for example, the efficiency and speed of transmitting 3D content. An example of such a library is DRACO™ developed by GOOGLE™.

[0119] FIG. 5 illustrates an example of a bitstream structure for geometry-based point cloud compression (G-PCC). A G-PCC bitstream may include a set of G-PCC units, which may be referred to, for example, as type-length-value (TLV) encapsulation structures, as illustrated in FIG. 5. G-PCC and GPCC may be used interchangeably herein. As illustrated in FIG. 5, a G-PCC unit may include information regarding a G-PCC tlv_type and a G-PCC tlv unit payload. FIG. 5 illustrates various tlv unit payload types. Table 1 shows an example of G-PCC TLV unit syntax. In the examples, a G-PCC TLV unit (e.g., each G-PCC TLV unit) may include a TLV type, a G-PCC TLV unit payload length, and / or a G-PCC TLV unit payload. A TLV type (e.g., tlv_type as shown in Table 1) can represent a G-PCC unit type. Table 2 shows an example of TLV types (e.g., tlv_type as shown in Table 1) and associated data unit descriptions. For example, a G-PCC TLV unit with unit type 2 may be a geometric data unit, and a G-PCC TLV unit with unit type 4 may be an attribute data unit. A point cloud may be reconstructed, for example, based on a geometric data unit and an attribute data unit. Geometric and / or attribute G-PCC unit payloads may correspond to media data units (e.g., TLV units) that can be decoded by a G-PCC decoder, for example. Geometric and attribute parameter set G-PCC units may specify a G-PCC decoder to decode the corresponding TLV units.G-PCC bitstream high-level syntax (HLS) can support slices and / or tile groups for geometric and attribute data. A frame may be partitioned into multiple tiles and slices. A slice may be a set of points that can be encoded or decoded (e.g., independently encoded or decoded). In examples, a slice may include a geometric data unit and zero or more attribute data units. Attribute data units may depend on a corresponding geometric data unit, for example, within the same slice. Within a slice, a geometric data unit may appear prior to any associated attribute units. Data units of a slice may be adjacent. The ordering of slices within a frame may not be specified. A group of slices may be identified by a common tile identifier. A tile inventory describing the bounding box for a tile (e.g., each tile) may be implemented. A tile may overlap with another tile within its bounding box. Each slice may include an index identifying which tile the slice belongs to. Table 1 shows an example of G-PCC TLV encapsulation unit payload syntax, Table 2 shows an example of G-PCC TLV types and data unit descriptions, and Table 3 shows an example of G-PCC TLV encapsulation unit payload syntax.

[0120]

[0121]

[0122]

[0123] Examples of elements, attributes, syntax, and semantics are shown and described in this specification. Elements are distinguished from attributes. Attributes may be identified by an "@" preceding the attribute. Examples of element usage value ranges may be provided in the following format:<minimum> ... <maximum>, where the value of N may indicate that the value is not limited. The elements, attributes, syntax and semantics described herein are non-limiting examples that may or may not be implemented alone or in various combinations, with or without exemplary examples, in various embodiments.

[0124] A G-PCC container file format may be implemented. FIG. 6 illustrates an example of a sample structure when G-PCC geometry and attribute bitstreams can be stored in a single track. A video coding device may require that, for example, when a G-PCC bitstream is returned in a single track, the G-PCC encoded bitstream be represented by a single-track declaration. Single-track encapsulation of G-PCC data may utilize simple encapsulation, for example, ISO base media file format (ISOBMFF) encapsulation, by storing the G-PCC bitstream in a single track without processing (e.g., additional processing). Samples within a single track (e.g., each sample) may contain one or more G-PCC components. Each sample may contain one or more TLV encapsulation structures.

[0125] FIG. 7 illustrates an example of a multi-track (e.g., ISOBMFF) G-PCC container structure. If the coded G-PCC geometry bitstream and the coded G-PCC attribute bitstream(s) are stored in separate tracks, each sample within the track may include at least one TLV encapsulation structure that carries G-PCC component data.

[0126] A multi-track G-PCC ISOBMFF container may include G-PCC tracks containing geometric bitstream samples that return geometric data TLV units, a set of geometric parameters, a set of sequence parameters, and / or geometric bitstream samples. G-PCC tracks may include track references to other tracks that return payloads of G-PCC attribute component(s). A multi-track G-PCC ISOBMFF container may include zero or more G-PCC tracks, each of which includes a set of attribute parameters for its respective attribute and attribute bitstream samples that return attribute data TLV units.

[0127] When a G-PCC bitstream is returned from multiple tracks, a track reference tool may be used to link between G-PCC component tracks. For example, TrackReferenceTypeBoxes may be added to the TrackReferenceBox within the TrackBox of a G-PCC track. A TrackReferenceTypeBox may contain, for example, an array of track_IDs specifying the tracks referenced by the G-PCC track. To link a G-PCC geometry track to a G-PCC attribute track, the reference_type of the TrackReferenceTypeBox in the G-PCC geometry track may be implemented to identify the associated attribute tracks. The 4CCs of the track reference types may be 'gpca'. In examples, the referenced track(s) may contain a coded bitstream of G-PCC attribute data.

[0128] When a geometry stream of a G-PCC bitstream contains multiple tiles, each tile or group of tiles may be encapsulated in a separate track, such as a geometry tile track. In examples, a geometry tile track may carry TLV units of one or more geometry tiles, which may enable direct access to the tiles. An attribute stream(s) of a G-PCC bitstream containing multiple tiles may be carried out on multiple attribute tile tracks.

[0129] G-PCC tile(s) data can be returned from attribute tile tracks within a container and from separate geometry. Partial access to G-PCC coded streams in ISOBMFF containers may be supported. Tiles corresponding to spatial regions within a point cloud scene may be signaled from samples in timed metadata tracks, such as tracks with Dynamic3DSpatialRegionSampleEntry, or from GPCCSpatialRegionInfoBox boxes. This may enable players and / or streaming clients to retrieve a set of tile tracks that return information necessary to render specific spatial regions or tiles within a point cloud scene.

[0130] G-PCC base tracks can return TLV encapsulated structures. TLV encapsulated structures may include a sequence parameter set (SPS), a geometry parameter set (GPS), an attribute parameter set (APS), and tile inventory information (e.g., may include only these). To link G-PCC base tracks to geometry tile tracks, a track reference having a new track reference type may be defined using a four-character code (4CC), namely 'gccg'. Track references of the new type may be used to link G-PCC base tracks to geometry tile tracks (e.g., each geometry tile track).

[0131] Geometric tile tracks (e.g., each geometric tile track) can be linked to other attribute(s) G-PCC tile tracks that return attribute information for each tile or tile group using a track reference tool. The 4CCs of these track reference types can be 'gpca'.

[0132] Track alternatives may be represented by an alternative track mechanism (e.g., the alternate_group field of the TrackHeaderBox). In the examples, G-PCC component tile tracks containing the same alternate_group value may be different encoded versions of the same G-PCC component. Volumetric visual scenes may be coded in the alternatives. In such cases, for example, G-PCC tracks that are alternatives to each other may contain the same alternate_group value in their TrackHeaderBox.

[0133] G-PCC component tile tracks may include alternative examples. In such cases, G-PCC component tile tracks belonging to an alternative group (e.g., all G-PCC component tile tracks) may be referenced by a G-PCC base track and / or their respective G-PCC geometry tile tracks. In the examples, G-PCC component tile tracks that are alternative examples of each other may use an alternative grouping mechanism.

[0134] MPEG Dynamic Adaptive Streaming over HTTP (MPEG-DASH) is a delivery format that can dynamically adapt to changes in network delivery conditions to provide, for example, a video experience (e.g., a better video experience) to end users.

[0135] Dynamic HTTP streaming can deliver multimedia content at one or more bitrates available on the server. Multimedia content may include multiple media components (e.g., audio, video, and / or text media components). Different media components may include different characteristics. One or more characteristics of the media components may be described, for example, by a Media Presentation Description (MPD).

[0136] FIG. 8 illustrates an exemplary MPD hierarchical data model. As illustrated in FIG. 8, an MPD can describe a sequence of periods (e.g., time intervals). For example, a set of encoded versions of media content components may not change for a certain period. A period (e.g., each period) may have a start time and an associated duration. A period (e.g., each period) may include one or more adaptation sets (e.g., AdaptationSet, such as Adaptation Set 1 as illustrated in FIG. 8). Adaptation set, adaptation set, AdaptationSet, and adaptationset may be used interchangeably herein. In one example, a DASH streaming client may be a WTRU, for example, as described herein in relation to FIG. 1a through 1d. In another example, a DASH streaming client may include a head-mounted device, a head-mounted projector, and / or a head-up display. In another example, the DASH streaming client may include 3D television. In another example, the DASH streaming client may include one or more cameras (e.g., advanced cameras).

[0137] An adaptation set (e.g., adaptation set, adaptation set, AdaptationSet, or adaptationset) may represent a set of encoded versions of one or more media content components that share one or more attributes (e.g., the same attributes), such as one or more of language, media type, picture aspect ratio, role, accessibility, viewpoint, rating attributes, and / or the like. In examples, the AdaptationSet may include different bitrates for geometric and / or attribute components of multimedia content (e.g., G-PCC content). The AdaptationSet may include different bitrates for audio components (e.g., lower quality stereo and / or higher quality surround sound) of multimedia content (e.g., the same multimedia content). In examples, the AdaptationSet (e.g., each adaptation set) may include multiple representations.

[0138] A representation may describe a transmittable encoded version of one or more media components. The terms representation and description may be used interchangeably herein. A representation may differ from other representations by, for example, bitrate, resolution, number of channels, and / or other characteristics. A representation (e.g., each representation) may include one or more segments. Attributes of a representation element (e.g., @id, @bandwidth, @qualityRanking, and / or @dependencyId) may specify one or more characteristics of the representation (e.g., may be used to specify them).

[0139] Segments can be retrieved via an HTTP request. Segments (e.g., each segment) may include a URL (e.g., an addressable location on a server). In examples, segments may be downloaded using, for example, an HTTP GET or an HTTP GET with byte ranges.

[0140] A DASH client can parse MPD XML documents. For example, a DASH client can select a set of AdaptationSets (e.g., suitable for the DASH client's environment) based on the elements of an AdaptationSet (e.g., information provided for each element of the AdaptationSet). The client can select a representation for an AdaptationSet (e.g., within each AdaptationSet). The client can select a representation based, for example, the value of the @bandwidth attribute, client coding ability, and / or client rendering ability. The client can download the initial segment of the selected representation. The client can access the content (e.g., by requesting entire segments or segments of byte ranges). The client can consume media content, for example, if a presentation has started or during a presentation. The client can request media segments and / or parts of media segments (e.g., sequentially) during a presentation. The client can play the content according to the media presentation timeline. The client may switch from a first representation to a second representation based, for example, on updated information from the client's environment. The client may, for example, continuously play the content over one or more periods. A media presentation (e.g., being consumed by the client in segments) may end, a period may begin, and / or the MPD may be re-fetched, for example, around the time the media presented in the representation ends.

[0141] MPEG-DASH descriptors can provide application-specific information regarding media content. Descriptor element structures may be similar. A descriptor element may include, for example, an @schemeIdUri attribute that can provide a URI to identify a scheme, an attribute @value, and / or an attribute @id. Element semantics may be specific to the scheme being adopted. The URI identifying the scheme may be, for example, a URN or a URL. MPDs may provide information on how to use the elements. Applications adopting DASH formats may, for example, instantiate description elements using scheme information. DASH applications using elements (e.g., descriptor elements) may define a scheme identifier (e.g., first defined) (e.g., in the form of a URI) and define a value space for the element (e.g., when the scheme identifier is used). In the examples, extension elements and / or attributes may be defined in a separate namespace, for instance, for structured data. Descriptors may appear at multiple levels within the MPD. For instance, the presence of an element at the MPD level may indicate that the element is a child of the MPD element. For instance, the presence of an element at the AdaptationSet level may indicate that the element is a child of the AdaptationSet element. For instance, the presence of an element at the Presentation level may indicate that the element is a child of the Presentation element.

[0142] (e.g., in MPEG-DASH) a bundle may be a set of media components that can be jointly consumed by a decoder instance (e.g., a single decoder instance). A bundle (e.g., each bundle) may include a media component (e.g., a main media component) that may contain decoder-specific information and / or bootstrap the decoder. A preselection may refer to, / or identify, / or define a subset of media components within a bundle that can be jointly consumed (e.g., expected to be jointly consumed).

[0143] An AdaptationSet containing a main media component may be referred to as the main adaptation set. The main adaptation set, primary adaptation set, and primary adaptation set (e.g., and any variations based on the capitalization or compound word of adaptation set (e.g., adaptationset)) may be used interchangeably herein. The main media component may be included in a preselection that may be associated with a bundle. A bundle (e.g., each bundle) may include one or more partial AdaptationSets. Partial AdaptationSets may be processed in combination with the main AdaptationSet.

[0144] Table 4 shows an example of preselection element semantics. Preselections can be defined, for example, through preselection elements, as shown in Table 4. In the examples, the selection of preselections can be based on attributes and / or elements that may be included in the preselection elements.

[0145]

[0146] Multimedia applications, such as virtual reality (VR) and immersive 3D graphics, can be implemented or represented by 3D point clouds, which can enable interaction of updated forms and / or communication with one or more virtual worlds. Static and dynamic point clouds can generate large amounts of information. Efficient coding algorithms can be used to compress point cloud information, for example, to reduce storage and / or transmission resource utilization by point cloud information. For example, a bitstream of compressed dynamic point cloud information can utilize fewer transmission resources than a bitstream of uncompressed information.

[0147] Point cloud applications may utilize coding, storage, and / or network resources (e.g., streaming point cloud data over a network). In examples, point cloud applications may perform live or on-demand streaming of point cloud content, for example, depending on how the content can be generated. Point cloud applications may generate, / or process, or transmit or receive large amounts of information representing point clouds. Point cloud applications may support adaptive streaming techniques, for example, to avoid network overload and / or to provide optimized observation experiences in relation to variable network capacity and / or other operating conditions.

[0148] MPEG-DASH can provide adaptive delivery of point clouds (e.g., can be used to provide this). MPEG-DASH can be implemented as signaling to support point cloud media, including, for example, point cloud streams. Signaling elements can represent or enable streaming clients to identify point cloud streams and their component substreams within an MPD file. Signaling elements can represent or enable streaming clients to identify one or more types of metadata that may be associated with point cloud components, for example, to enable streaming clients to select version(s) (e.g., best versions) of a point cloud or point cloud component that may be configured or configurable to be supported by the streaming client.

[0149] Components of point cloud content may be available in different representations. In examples, multiple representations (e.g., each of multiple representations) may represent different quality levels. A streaming client may utilize guidance regarding different representations (e.g., indications signaled in an MPD file). For example, the indication may indicate which set of representations across different components constitutes a certain quality level (e.g., to perform appropriate quality degradation). Components of point cloud content may be divided into multiple tiles. Clients may stream specific tile portions of geometric components (e.g., selected tile portions) (e.g., instead of streaming all point cloud data) based, for example, on bandwidth availability. G-PCC component tile bitstreams may be available in different adaptation sets, for example, in which an adaptation set (e.g., each adaptation set) can represent G-PCC component tiles.

[0150] G-PCC media content may include multiple components, such as geometric structures and / or attributes. Components (e.g., each component among the multiple components) may be encoded individually, for example, as substreams of a G-PCC bitstream. Components such as geometric structures and attributes may be encoded, for example, using a G-PCC encoder. The substreams may be decoded collectively (e.g., along with metadata) to render a point cloud, for example.

[0151] Elements and / or attributes may be defined, for example, as XML elements and / or XML attributes. XML elements may be defined, for example, in a separate namespace (e.g., "urn:mpeg:mpegI:gpcc:2020"). The namespace designer "gpcc:" may be used herein, for example, to refer to a separate namespace.

[0152] G-PCC components can be signaled in the DASH MPD. In examples, a G-PCC component (e.g., each G-PCC component) can be represented in a DASH manifest file (e.g., MPD file) as a separate AdaptationSet, which can be referred to, for example, as a component adaptation set. An adaptation set containing geometric information can be a main adaptation set that can serve, for example, as an access point (e.g., main access point) for G-PCC content. In examples, an adaptation set (e.g., a single adaptation set) can be signaled for each component per resolution. In examples, the main adaptation set can include an @codecs attribute set to 'gpc1'.

[0153] The EssentialProperty descriptor can be used with the @schemeIdUri attribute set to "urn:mpeg:mpegI:gpcc:2020:component" to identify, for example, the type of G-PCC component(s) in a component adaptation set. The EssentialProperty descriptor can be referred to, for example, as the GPCCComponent descriptor.

[0154] In examples (e.g., at the adaptation set level), a GPCCComponent descriptor (e.g., one GPCCComponent descriptor) can be signaled for a point cloud component (e.g., each point cloud component) present in the representations of the adaptation set.

[0155] Table 5 illustrates an example of elements and attributes for a GPCCComponent descriptor. In the examples, the @value attribute of the GPCCComponent descriptor may not exist. A GPCCComponent descriptor may include the attributes defined in Table 5.

[0156]

[0157] Table 6 shows an example of an XML schema for a GPCCComponent descriptor corresponding to Table 5.

[0158]

[0159] The main adaptation set may include, for example, an initialization segment (e.g., a single initialization segment) at the adaptation set level or multiple initialization segments (e.g., one initialization segment for each representation) at the representation level. The initialization segments may include G-PCC parameter sets, which can initialize a G-PCC decoder (e.g., can be used to initialize it). G-PCC parameter sets for one or more representations (e.g., all representations) may be included in the initialization segment, for example, if an initialization segment (e.g., a single initialization segment) exists.

[0160] In the examples, the initialization segment for a representation (e.g., each representation) may include G-PCC parameter sets for the representation and geometric data for the representation, for example, if more than one representation is signaled in the main adaptation set. Representation(s) of other component adaptation sets (e.g., other component adaptation sets of a point cloud) may enumerate corresponding representation identifiers from the main adaptation set, for example, using the @dependencyId attribute. Representations in the main adaptation set may be mapped to corresponding representations in the G-PCC component AdaptationSets. Media segments for representations in the main adaptation set may include, for example, one or more track fragments of a G-PCC track. Media segments for representations in the component AdaptationSets may include, for example, one or more track fragments of a corresponding component track (e.g., at the file format level).

[0161] In the examples, the role descriptor element may be used with values ​​that can be defined for G-PCC components. For example, one or more geometry components may contain corresponding value(s) of gpcc-geometry, or one or more attribute components may contain corresponding value(s) of gpcc-attributes. The EssentialProperty descriptor element (e.g., similar to the EssentialProperty descriptor element described for the example shown in Table 5) may be signaled at the adaptation set level. In the examples, the EssentialProperty descriptor element may be signaled excluding the component_type attribute (e.g., at the adaptation set level). The EssentialProperty descriptor element may be signaled to identify, for example, geometry component(s) and / or attribute component(s).

[0162] In examples, certain versions (e.g., each version) among multiple versions of a G-PCC component may be signaled in a separate AdaptationSet with the value of the @codecs attribute set according to the media codec used, for example, when multiple versions of a G-PCC component are encoded using different codecs. Switching (e.g., seamless switching) between representations across the AdaptationSets of multiple versions of a G-PCC component may be supported. Each of the multiple adaptation sets may include a SupplementalProperty descriptor having an @value containing a comma-separated list of AdaptationSet IDs corresponding to the @schemeIdURI and / or other available versions set to, for example, urn:mpeg:dash:adaptation-set-switching:2016, to indicate that seamless switching between representations across the AdaptationSets of multiple versions of a G-PCC component is supported. In the examples, one or more rules may be applied to support switching across adaptation sets.

[0163] G-PCC tile tracks can be signaled. When multiple tile tracks exist in a G-PCC container, the main adaptation set may include parameter sets and tile inventory information from the G-PCC base track (e.g., may include only these). Geometric data and / or attribute data may not exist in the main adaptation set and its representations. In examples, the @codecs attribute for the main adaptation set may be set to 'gpcb', indicating, for example, that the adaptation set includes base track data containing (e.g., only these) the SPS, GPS, APS, and tile inventory information of the G-PCC content.

[0164] Component tile tracks (e.g., each component tile track) can be signaled in a separate adaptation set. A separate adaptation set can be referred to as a tile component adaptation set. When multiple versions of a component exist for the same tile (e.g., or a set of the same tiles) and / or are returned from separate tile tracks, each version can be signaled in the representation of the tile component adaptation set. The @codecs attribute for tile component adaptation sets representing component tile tracks for G-PCC media content can be set to 'gpt1'.

[0165] At the tile component adaptation set level, a GPCCComponent descriptor may be signaled. In examples, the GPCCComponent descriptor may include an attribute (e.g., an additional attribute) @tile_ids, which indicates, for example, a list of tiles present in the tile bitstream. The GPCCComponent descriptor may include an XML attribute @attr_index (e.g., conditionally included) if, for example, the component represented by the enclosing adaptation set is a G-PCC attribute component. The @attr_index attribute may signal the order of G-PCC attribute components in the SPS and / or may enable distinguishing G-PCC attribute components if, for example, multiple G-PCC attribute components having the same attribute type (e.g., more than one color attribute) exist in the G-PCC content. The GPCCComponent descriptor existing at the tile component adaptation set level may include elements and / or attributes as defined in Table 7.

[0166]

[0167] Table 8 shows an example of an XML schema for a GPCCComponent descriptor corresponding to Table 7.

[0168]

[0169] In examples, when multiple tile tracks exist in a container, each representation in the geometry tile component adaptation set can refer to a corresponding representation in the main adaptation set, for example, using the @dependencyId attribute. Each representation in the property tile component adaptation set can refer to a corresponding representation in the geometry tile component adaptation set, for example, using the @dependencyId attribute.

[0170] G-PCC component tile tracks containing the same alternate_group value can be signaled in MPD, for example, as representations of a tile component adaptation set.

[0171] G-PCC descriptors can be signaled. A streaming client can identify the type of point cloud component in an AdaptationSet and / or representation by, for example, checking the GPCCComponent descriptor within the corresponding element (e.g., it may be possible to identify it or may be configured to identify it). A streaming client can distinguish different geometric point cloud streams present in an MPD file.

[0172] A G-PCC descriptor may include, for example, a SupplementalProperty element having the @schemeIdUri attribute identical to, for example, "urn:mpeg:mpegI:gpcc:2020:gpc". Table 9 shows an example of attributes for a G-PCC descriptor. In the examples, one or more (e.g., at most one) G-PCC descriptors may exist at the adaptation set level for the main adaptation set of the G-PCC medium.

[0173]

[0174] Data types for attributes may be, for example, as defined in an XML schema. Table 10 shows an example of an XML schema for a G-PCC descriptor. The schema may be represented, for example, in an XML schema with the namespace urn:mpeg:mpegI:gpcc:2020.

[0175]

[0176] The GPCCTileId descriptor can be signaled. A streaming client can identify tile IDs or tile IDs present in a G-PCC tile component AdaptationSet, for example, by checking the GPCCComponent descriptor. In examples, one or more components of a G-PCC tile (e.g., all components) may be stored in a single track. For example, the GPCCComponent descriptor may not be signaled in the AdaptationSet associated with such a track. In examples, a streaming client can distinguish different G-PCC tile tracks that may exist in an MPD file. For example, a streaming client can distinguish different G-PCC tile tracks by identifying their respective tile streams.

[0177] In examples, a SupplementalProperty element having the @schemeIdUri attribute identical to "urn:mpeg:mpegI:gpcc:2020:tileID" may be a GPCCTileId descriptor (e.g., may be referred to as such). A GPCCTileId descriptor may be used to distinguish different G-PCC tile streams. In examples, one (e.g., at most one) GPCCTileId descriptor may be signaled at the adaptation set level for the G-PCC tile medium and / or may exist therein. For example, if a GPCCComponent descriptor is not available at the adaptation set level (e.g., when all G-PCC component data for a tile or group of tiles are on a single track), one (e.g., at most one) GPCCTileId descriptor may be signaled at the adaptation set level or may exist therein.

[0178] In the examples, the @value attribute of the GPCCTileId descriptor may not exist. The GPCCTileId descriptor may include one or more attributes as exemplified in Table 11.

[0179]

[0180] Data types for attributes may be as provided in the XML schema. For example, the XML schema for the GPCCTileId descriptor may be as exemplified in the exemplary schema below. The schema may be represented as an XML schema containing the namespace urn:mpeg:mpegI:gpcc:2020 and may be specified as exemplified in Table 12 below.

[0181]

[0182] G-PCC preselection can be signaled. G-PCC preselection can be signaled in MPD using a preselection element (e.g., as defined in DASH), which has a list of identifiers (IDs) for the @preselectionComponents attribute, for example, including the ID of the main adaptation set for the volumetric medium and the IDs of the AdaptationSets corresponding to (e.g., subsequent) G-PCC components. In the examples, the @codecs attribute for the preselection can be set to 'gpc1', which may indicate, for example, that the preselection medium is a geometry-based point cloud. Preselection can be signaled using a preselection element within a duration element and / or a preselection descriptor at the adaptation set level.

[0183] FIG. 9 illustrates an example of grouping G-PCC components in an MPD using a preset. FIG. 9 illustrates an exemplary DASH configuration for grouping G-PCC components that may belong to a volumetric medium (e.g., a single volumetric medium) within an MPEG-DASH MPD file.

[0184] Multiple versions of a G-PCC medium may be signaled. In examples, multiple versions of the same point cloud medium may be signaled, for example, using distinct preselections. Preselections representing alternative versions of the same geometry-based point cloud medium may, for example, include G-PCC descriptors having the same @gpcId value. One or more (e.g., at most one) G-PCC descriptors may exist, for example, at the preselection level. Preselections may be selectable alternatives. The ID list of the @preselectionComponents attribute may include, for example, the ID of the main adaptation set and the IDs of the subsequent remaining component adaptation sets when the @codecs attribute is set to 'gpc1'.

[0185] FIG. 10 illustrates an example of grouping multiple versions of G-PCC components in an MPD using preselections. FIG. 10 illustrates an example of a DASH configuration for grouping multiple versions of G-PCC components that may belong to a single point cloud within an MPEG-DASH MPD file. Grouping / association can be signaled, for example, using preselection descriptors. Table 13 illustrates an example of signaling multiple versions of G-PCC components in an MPD using preselections.

[0186]

[0187]

[0188]

[0189] In the examples, the G-PCC components of the point cloud, the AdaptationSets, or the representation(s) of the adaptation sets may enumerate the identifiers of the main AdaptationSets and / or representations, for example, using the @dependencyId attribute. For example, if segments from the main adaptation set are decoded together with segments from the AdaptationSets of the point cloud components to reconstruct the point cloud, dependencies (e.g., intrinsic dependencies) may exist.

[0190] In examples, G-PCC tile presets can be implemented. When G-PCC content is returned using multiple tile tracks, the main adaptation set can signal G-PCC base track data. Tile component adaptation sets can signal G-PCC geometry and / or attribute tile track data.

[0191] G-PCC tile preselection can be signaled in MPD using preselection elements as described herein. In one example, the @codecs attribute of the preselection can be set to 'gpt1', for example, to indicate that the preselection medium is a set of geometry-based point cloud tiles. The preselection can be signaled using preselection elements within a period element as described herein. The preselection can be signaled using preselection descriptors at the tile component adaptation set level.

[0192] Preselection elements may include a list of IDs for the @preselectionComponents attribute. The list of IDs for the @preselectionComponents attribute for a G-PCC tile preselection may include the ID of the geometry tile component adaptation set and the subsequent corresponding attribute tile component adaptation set ID(s). The representation of the main adaptation set corresponding to the representation of the selected geometry tile component adaptation set may be identified, for example, using the @dendencyId attribute signaled in the representation(s) of the adaptation set.

[0193] A G-PCC tile preselection (e.g., each G-PCC tile preselection) may include one or more GPCCTileId descriptor(s). This may allow identification of the tiles referenced in each preselection. When a GPCCTileId descriptor does not exist, tiles belonging to a G-PCC tile preselection can be identified by finding a geometric tile component adaptation set from the list of IDs of the @preselectionComponents attribute and checking the list of tile IDs from the GPCCComponent descriptors present in the geometric tile component adaptation set.

[0194] FIG. 11 illustrates an example of G-PCC content having multiple tile tracks. FIG. 11 may be an exemplary DASH configuration. The G-PCC content may include geometric components and one or more attribute components (e.g., three attribute components). In the example, the G-PCC bitstream includes six tiles grouped into two tile sets. The first tile set includes tiles 1, 2, and 3, and the second tile set includes tiles 4, 5, and 6. The components of each tile set may be available in two different versions (e.g., encoded in different qualities). The component versions for the tile sets (e.g., each component version) may be returned from separate G-PCC tile tracks within an ISOBMFF container file. The MPD file may include a set of tile component adaptations for each component of the two tile sets. A tile component adaptation set (e.g., each tile component adaptation set) may include two representations (e.g., one for each version of the component). Two presets may be used in MPD to signal two tile sets present in the G-PCC bitstream.

[0195] Table 14 illustrates an example of a DASH MPD file that signals G-PCC content having multiple tile tracks with a pre-selected descriptor.

[0196]

[0197]

[0198]

[0199]

[0200]

[0201] In examples, media data may be signaled using distinct preselections, for example, where multiple point cloud media are available. Preselections representing geometry-based point cloud media data may include G-PCC descriptors having unique @gpcId values. One or more (e.g., at most one) G-PCC descriptors may exist, for example, at the preselection level. There may be an ID of the main adaptation set, an ID within the list of adaptation set IDs for the @preselectionComponents attribute (e.g., the first ID), and / or IDs of AdaptationSets corresponding to (e.g., subsequent) point cloud components. Point clouds may be identified, for example, using the unique value of the @gpcId attribute, as may be defined in the G-PCC descriptor.

[0202] G-PCC tile groups can be signaled. In examples, where multiple tiles exist within a geometry-based point cloud, tile bounding box information can be signaled (e.g., using the GPCCTileInventory descriptor). The GPCCTileInventory descriptor can be a SupplementalProperty element having the @schemeIdUri attribute (e.g., set to "urn:mpeg:mpegI:gpcc:2020:gptl"). The GPCCTileInventory descriptor can exist at the adaptation set level for the main adaptation set of the G-PCC medium, for example, when the G-PCC medium is tiled. Table 15 shows an example of elements and attributes for the GPCCTileInventory descriptor.

[0203]

[0204] Table 16 illustrates an example of an XML schema for a GPCCTileInventory descriptor. Data types for the various elements and attributes of a GPCCTileInventory descriptor can be defined according to an XML schema, for example, the exemplary schema shown in Table 16.

[0205]

[0206] For example, when a client is about to stream data of tiled G-PCC components from a server, the client can select a tile ID from the tile inventory bounding box information present in the MPD (e.g., selected first). In the examples, G-PCC components with the selected tile_id can be streamed to the client.

[0207] Dynamic G-PCC tile inventory information may be signaled. When parameter set data and / or tile inventory information change dynamically, information regarding such changes may be returned from samples of the G-PCC base track. In examples, when multiple tiles exist within a geometry-based point cloud and the tile bounding box information is changing dynamically, the tile bounding box information (along with parameter set data, e.g.) may be returned from the media segments of the representations of the main adaptation set.

[0208] Spatial regions may be static. For example, if 3D spatial regions are static, the properties of the spatial regions and / or mappings between the regions and G-PCC tiles may be signaled (e.g., using GPCC3DRegions descriptors). 3D spatial regions may be static, for example, if the position and dimensions of the regions (e.g., each region) do not change over presentation time. A GPCC3DRegions descriptor may be a SupplementalProperty element having the @schemeIdUri attribute identical to "urn:mpeg:mpegI:gpcc:2020:gpsr". A GPCC3DRegions descriptor (e.g., a single GPCC3DRegions descriptor) may exist, for example, at the adaptation set level and / or the presentation level in the main G-PCC track, or at the preselection level for geometry-based volumetric media content.

[0209] The @value attribute of the GPCC3DRegions descriptor may not exist. The GPCC3DRegions descriptor may include elements and / or attributes (e.g., as specified in Table 17). Table 17 shows an example of the elements and attributes associated with the GPCC3DRegions descriptor.

[0210]

[0211] Data types for various elements and attributes of the GPCC3DRegions descriptor can be defined by a schema such as the XML schema shown in Table 18. Table 18 shows an example of an XML schema for the GPCC3DRegions descriptor.

[0212]

[0213] In examples, for instance, where 3D spatial regions are static and tile inventory information is not available, the properties of the spatial regions and / or mappings between the spatial regions and the corresponding AdaptationSets of G-PCC components can be signaled (e.g., using the GPCC3DRegions descriptor). The GPCC3DRegions descriptor may be, for instance, a SupplementalProperty element having the @schemeIdUri attribute identical to "urn:mpeg:mpegI:gpcc:2020:gpsr". The GPCC3DRegions descriptor (e.g., a single GPCC3DRegions descriptor) may exist, for instance, at the adaptation set level and / or the representation level in the main G-PCC track, or at the preselection level for geometry-based volumetric media content.

[0214] The @value attribute of the GPCC3DRegions descriptor may not exist. The GPCC3DRegions descriptor may include elements and attributes (e.g., as specified in Table 19). Table 19 shows an example of elements and attributes for the GPCC3DRegions descriptor.

[0215]

[0216] Data types for various elements and attributes of the GPCC3DRegions descriptor can be defined according to a schema, for example, an XML schema. Table 20 shows an example of an XML schema for the GPCC3DRegions descriptor.

[0217]

[0218] In examples, for instance, when 3D spatial regions are static, the mapping between the spatial region and the corresponding AdaptationSets of G-PCC components can be signaled using the GPCC3DRegionId descriptor. The descriptor can be a SupplementalProperty element with the @schemeIdUri attribute identical to "urn:mpeg:mpegI:gpcc:2020:gp3rid". A single GPCC3DRegionId descriptor can exist at the adaptation set level of a G-PCC component (e.g., each G-PCC component). The GPCC3DRegionId may not exist, for instance, if the gpsr.spatialRegion@asIds attribute exists in the GPCC3DRegions descriptor.

[0219] The @value attribute of the GPCC3DRegionId descriptor may not exist. The GPCC3DRegionId descriptor may include one or more attributes as shown in Table 21.

[0220]

[0221] The data types for the attributes may be as provided in the XML schema. The XML schema for the GPCC3DRegionID descriptor is shown below. The schema can be represented as an XML schema containing the namespace urn:mpeg:mpegI:gpcc:2020 and is specified in Table 22.

[0222]

[0223] In examples, for instance, when the 3D spatial region(s) are static, the mapping between the spatial region and the corresponding AdaptationSets of the G-PCC components can be signaled using the GPCCComponents descriptor. The GPCCComponent descriptor may include elements and attributes as defined in Table 23. The GPCC3DRegonID descriptor may not exist, for instance, if the @region_Id attribute exists in the GPCCComponents descriptor.

[0224]

[0225] An example of an XML schema for a GPCCComponent descriptor is shown in Table 24 below.

[0226]

[0227] In one example, one or more spatial regions may be dynamic. For 3D partitions that may be dynamic, a timed metadata track for signaling the positions and / or dimensions of the 3D regions (e.g., each 3D region) for the presentation timeline may be returned from a separate AdaptationSet having a single representation. The timed metadata track may be associated (e.g., linked) with the main G-PCC adaptation set(s). The attributes used may include an @associationId attribute containing the 4CC 'gpdr' for the corresponding AdaptationSet or representation and an @associationType value.

[0228] Streaming client behavior may be based on signaling, for example, the signaling of one or more descriptors. A DASH client may be guided, for example, by information provided in the MPD. The following is an example of client behavior for streaming geometry-based point cloud compressed content, using, for example, the signaling examples disclosed herein. The exemplary client behavior may assume, for example, that the association of component AdaptationSets with respect to a main point cloud AdaptationSet is signaled using G-PCC descriptors.

[0229] A streaming client can publish an HTTP request (e.g., publish first) with the destination set to a content server, for example. The streaming client can download an MPD file from the content server. The client can parse the MPD file, for example, to generate corresponding in-memory representations of XML elements in the MPD file.

[0230] A streaming client can check pre-selected elements at the period level (e.g., having the @codecs attribute set to 'gpc1' or 'gpt1') to identify G-PCC media content available during a certain period.

[0231] AdaptationSets (e.g., all AdaptationSets) belonging to the point cloud content represented by the preselection element can be identified, for example, by checking the list of IDs in the preselection's @preselectionComponents attribute. The main adaptation set may contain the same @id value as that of the first ID in the list.

[0232] A streaming client can identify the number of unique point clouds by checking, for example, the G-PCC descriptors of AdaptationSets and / or groups of AdaptationSets that have the same @gpcId value in their G-PCC descriptors as versions of the same content.

[0233] (For example, by checking the GPCCComponent descriptors of the remaining AdaptationSets referenced in the ID list of the @preselectionComponent attribute) the streaming client can identify the components of the point cloud and map the components (e.g., each component) to their corresponding AdaptationSet. In the examples, more than one point cloud component may exist in the AdaptationSet.

[0234] A group of AdaptationSets having @gpcId values ​​existing in G-PCC descriptors corresponding to desired content may be selected from a list of IDs of the @preselectionComponent attribute, for example, based on point cloud content that the user may be interested in streaming. The streaming client may select a group of AdaptationSets having supported versions (e.g., supported resolutions), for example, if multiple preselection descriptors exist with the same @gpcId value. For example, if there are no multiple preselection descriptors with the same @gpcId value, a unique group of AdaptationSets may be selected.

[0235] The client can start streaming the point cloud by, for example, downloading an initialization segment for the main adaptation set containing parameter sets for initializing the G-PCC decoder. Initialization segments for the coded component streams can be downloaded to memory and / or cached.

[0236] A streaming client may begin downloading time-aligned media segments from a main adaptation set and / or component adaptation sets (e.g., in parallel over HTTP). In examples, the downloaded segments may be stored in in-memory segment buffers. The time-aligned media segments may be removed from their respective buffers and / or concatenated with their respective initial segments.

[0237] A media container (e.g., ISO Basic Media File Format (ISOBMFF)) can be parsed to extract basic stream information and structure from a G-PCC bitstream, for example, and the bitstream can be passed to a G-PCC decoder.

[0238] Client behavior for streaming G-PCC media having multiple tiles may be implemented, for example, using MPD signaling as described herein. A client (e.g., a streaming client) may issue an HTTP request and / or download an MPD file from a content server. The client may parse the MPD file to generate corresponding in-memory representations of XML elements from the MPD file.

[0239] The client can check AdaptationSet elements with the @codecs attribute set to 'gpcb' and pre-selected elements with the @codecs attribute set to 'gpt1' at the period level to identify, for example, G-PCC tile-type media content available within a certain period.

[0240] When G-PCC tiled media content exists, the client can identify tiles of interest in the point cloud bitstream based, for example, the client's current viewport. The client can parse GPCC3DRegions descriptors and / or locate their respective tiles within the viewport. When 3D partitions are dynamic, media segments of a timed metadata adaptation set capable of returning the positions and / or dimensions of 3D regions (e.g., each 3D spatial region) on the presentation timeline can be downloaded. 3D regions within the viewport can be identified. Their respective tiles belonging to the regions can be identified.

[0241] If tiles of interest are found, the client can select preselect elements containing the tiles by, for example, parsing the GPCCTileId descriptor present in the preselect elements (e.g., each preselect element). The @tile_Ids attribute in the GPCCTileId descriptor can enumerate the available tiles. Preselect elements containing the tiles of interest can be selected. Preselections (e.g., other preselections) can be ignored.

[0242] When the GPCCTileId descriptor is not available, tiles present in the preselection element can be identified, for example, by finding a geometric tile component adaptation set from the list of IDs in the @preselectionComponents attribute and finding a list of tile IDs from the GPCCComponent descriptors present in the geometric tile component adaptation set. If tiles of interest exist in the preselection element, the preselection can be selected by the client.

[0243] From the selected preselections, the group of tile component adaptation sets to be used to download media segments can be identified from the list of IDs of the @preselectionComponent attribute. In examples, the @preselectionComponents list may include the geometry tile component adaptation set ID. The @preselectionComponents list may include the remaining component tile component adaptation set IDs. The ID of the main adaptation set may not exist in the @preselectionComponents list. For example, the ID of the main adaptation set can be identified using the @dependencyId attribute present in the representation of the geometry tile component adaptation set.

[0244] The client can start streaming the point cloud by, for example, downloading an initialization segment from the main adaptation set. The initialization segment may include a set of parameters for initializing the G-PCC decoder.

[0245] Initialization segments for coded component streams (e.g., if present) can be downloaded to memory and / or cached.

[0246] A streaming client can download time-aligned media segments from a geometry tile component adaptation set and / or an associated attribute tile component adaptation set. Downloads can be performed in parallel via HTTP, and the downloaded segments can be stored in an in-memory segment buffer.

[0247] Time-aligned media segments can be removed from their respective buffers and / or concatenated with their respective initialization segments.

[0248] A media container (e.g., ISOBMFF) can be parsed to extract basic stream information, for example. The media container can be structured, and the generated bitstream can be passed to a G-PCC decoder.

[0249] Many embodiments are described herein. Features of the embodiments may be provided alone or in any combination across various claim categories and types. Additionally, the embodiments may include one or more of the features, devices, or images described herein alone or in any combination across various claim categories and types, such as, for example, any of the following.

[0250] As a decoder such as an exemplary decoder (300), the decoder receives, decodes, and interprets signals (e.g., as described herein) representing elements, attributes, and metadata associated with point cloud components; identifies point cloud streams and their component substreams within a media presentation descriptor (MPD); identifies versions of point clouds and / or their components; decodes the MPD to identify a main adaptation set and other adaptation sets to identify G-PCC components in geometry-based point cloud compression (G-PCC) content; decodes the MPD to identify types of point cloud components in an adaptation set or representation; decodes the MPD to identify one or more preselections; decodes the MPD to identify one or more versions of a G-PCC medium; decodes the MPD to identify one or more G-PCC tile groups; It is configured to decode the MPD to identify one or more tile IDs for G-PCC components in an adaptation set; to decode the MPD to identify one or more attributes of spatial regions and mappings between regions and G-PCC tiles, attributes of spatial regions and mappings between regions and corresponding adaptation sets of G-PCC components, and / or mappings between spatial regions and corresponding adaptation sets of G-PCC components; to decode the MPD to identify timed metadata tracks for dynamic spatial regions; etc.

[0251] Decoding tools and techniques comprising one or more of entropy decoding, inverse quantization, inverse transform, and differential decoding used to enable the examples described in this specification in a decoder.

[0252] As an encoder such as an exemplary encoder (200), the encoder is configured to generate, encode, and transmit signals (e.g., as described herein) representing elements, attributes, and metadata associated with point cloud components; to encode an MPD to represent point cloud streams and their component substreams; to encode an MPD to represent a main adaptation set and other adaptation sets to support the identification of G-PCC components in geometry-based point cloud compression (G-PCC) content; to encode an MPD to support the identification of types of point cloud components in an adaptation set or representation; to encode an MPD to identify one or more preselections; to encode an MPD to support the identification of one or more versions of a G-PCC medium; to encode an MPD to support the identification of one or more G-PCC tile groups; to encode an MPD to support the identification of one or more tile IDs for G-PCC components in an adaptation set; It is configured to encode the MPD to support the identification of one or more characteristics of spatial regions and mappings between regions and G-PCC tiles, characteristics of spatial regions and mappings between regions and corresponding adaptation sets of G-PCC components, and / or mappings between spatial regions and corresponding adaptation sets of G-PCC components; to decode the MPD to identify timed metadata tracks for dynamic spatial regions; etc.

[0253] Encoding tools and techniques comprising one or more of quantization, entropy coding, inverse quantization, inverse transform, and differential coding used to enable the examples described in this specification in an encoder.

[0254] For example, syntax element(s) inserted into a signaling to enable a decoder to identify an indication associated with performing any of the examples described herein.

[0255] For example, syntax element(s) inserted into a signaling to enable an encoder to generate or encode a mark associated with performing any of the examples described herein.

[0256] A bitstream or signal may include one or more of the described syntax elements associated with performing any of the examples described herein, or variations thereof.

[0257] A method, process, device, medium for storing instructions, medium for storing data, or signal for generating and / or transmitting and / or receiving and / or decoding a bitstream or signal comprising one or more of the described syntax elements or variations thereof.

[0258] According to any one of the examples described herein, a method, process, apparatus, a medium for storing instructions, a medium for storing data, or a signal for generating and / or transmitting and / or receiving and / or decoding.

[0259] A TV, set-top box, mobile phone, tablet, or other electronic device that performs adaptive streaming of geometry-based point clouds, such as point cloud component substreams in point cloud streaming services, according to any one of the examples described in this specification.

[0260] A TV, set-top box, mobile phone, tablet, or other electronic device that performs adaptive streaming of geometry-based point clouds, such as point cloud component substreams in point cloud streaming services, according to any one of the examples described herein, and displays the generated image (e.g., using a monitor, screen, or other type of display).

[0261] A TV, set-top box, mobile phone, tablet, or other electronic device that selects a channel for receiving a signal including an encoded image (e.g., using a tuner) according to any one of the examples described herein, and performs adaptive streaming of geometry-based point clouds, such as point cloud component substreams in point cloud streaming services.

[0262] A TV, set-top box, mobile phone, tablet, or other electronic device that receives a signal containing an encoded image wirelessly (e.g., using an antenna) according to any one of the examples described herein, and performs adaptive streaming of geometry-based point clouds, such as point cloud component substreams in point cloud streaming services.

[0263] Although features and elements have been described above in specific combinations, those skilled in the art will understand that each feature or element may be used alone or in any combination with other features and elements. Additionally, the methods described herein may be implemented as computer programs, software, or firmware integrated into a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted via wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, magnetic media such as read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, internal hard disks, and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital multifunction disks (DVDs). A processor associated with software may be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.< / maximum> < / minimum>

Claims

Claim 1 A device comprising a processor, wherein the processor is configured to receive a media presentation description (MPD) file from a content server; to determine from the MPD file a geometry-based point cloud compression (G-PCC) adaptation set, wherein the G-PCC adaptation set is an adaptation set associated with a G-PCC component; to determine a G-PCC tile identifier associated with a tile, wherein the G-PCC tile identifier is determined from the GPCCComponent descriptor associated with the G-PCC adaptation set when the GPCCComponent descriptor is signaled at the adaptation set level, and from the GPCCTileID descriptor associated with the G-PCC adaptation set when all G-PCC components of the tile are available on a single track; to request the G-PCC component using the G-PCC tile identifier associated with the tile; and to receive the G-PCC component. Claim 2 A device according to claim 1, wherein when the G-PCC tile identifier is determined from the GPCCComponent descriptor, the GPCCComponent descriptor is signaled for each point cloud component existing in the set of representations associated with the G-PCC adaptation set. Claim 3 A device according to claim 1, wherein when the G-PCC tile identifier is determined from the GPCCComponent descriptor, the GPCCComponent descriptor includes information associated with geometric point cloud components present in the representation of the G-PCC adaptation set. Claim 4 In claim 1, when the G-PCC tile identifier is determined from the GPCCComponent descriptor, the GPCCComponent descriptor includes a component type, an attribute type, an attribute index, and a list of G-PCC tile identifiers, wherein the component type indicates whether the G-PCC component is a geometric component or an attribute component, the attribute type indicates the attribute index and the type of attribute that indicate the order of the attribute existing in a sequence parameter set (SPS), and the attribute type and the attribute index exist in the GPCCComponent descriptor if the component type indicates a G-PCC attribute component. Claim 5 In claim 1, the processor is configured to determine a G-PCC descriptor representing a point cloud stream existing in the MPD file, and the G-PCC descriptor is signaled at the adaptive set level, the device. Claim 6 In paragraph 5, the device, wherein the G-PCC descriptor is signaled at a preset level using a preset element within a period element. Claim 7 In claim 1, the device, wherein the G-PCC adaptation set comprises one or more expressions. Claim 8 In claim 7, the device, wherein one or more of the above expressions include at least one of a bitrate, resolution, number of channels, or quality level. Claim 9 A device according to claim 1, wherein when the G-PCC tile identifier is determined from the GPCCTileID descriptor, the GPCCTileID descriptor is used to distinguish different G-PCC tile streams. Claim 10 In claim 1, when the G-PCC tile identifier is determined from the GPCCTileID descriptor, the GPCCTileID descriptor is signaled at the adaptive set level or preset level, the device. Claim 11 A method implemented by a device, comprising: receiving a Media Presentation Description (MPD) file from a content server; determining a Geometric-Based Point Cloud Compression (G-PCC) adaptation set from the MPD file, wherein the G-PCC adaptation set is an adaptation set associated with a G-PCC component; determining a G-PCC tile identifier associated with a tile, wherein the G-PCC tile identifier is determined from the GPCCComponent descriptor associated with the G-PCC adaptation set when the GPCCComponent descriptor is signaled at the adaptation set level, and the GPCCTileID descriptor associated with the G-PCC adaptation set when all G-PCC components of the tile are available in one track; requesting the G-PCC component using the G-PCC tile identifier associated with the tile; and receiving the G-PCC component. Claim 12 A method according to claim 11, wherein when the G-PCC tile identifier is determined from the GPCCComponent descriptor, the GPCCComponent descriptor is signaled for each point cloud component existing in the set of representations associated with the G-PCC adaptation set. Claim 13 A method according to claim 11, wherein when the G-PCC tile identifier is determined from the GPCCComponent descriptor, the GPCCComponent descriptor includes information associated with geometric point cloud components present in the representation of the G-PCC adaptation set. Claim 14 In claim 11, when the G-PCC tile identifier is determined from the GPCCComponent descriptor, the GPCCComponent descriptor comprises a component type, an attribute type, an attribute index, and a list of G-PCC tile identifiers, wherein the component type indicates whether the G-PCC component is a geometric component or an attribute component, the attribute type indicates the attribute index and the type of attribute that indicate the order of the attribute existing in the sequence parameter set (SPS), and the attribute type and the attribute index exist in the GPCCComponent descriptor if the component type indicates a G-PCC attribute component. Claim 15 A method according to claim 11, further comprising the step of determining a G-PCC descriptor representing a point cloud stream existing in the MPD file, wherein the G-PCC descriptor is signaled at the adaptive set level. Claim 16 In paragraph 15, the above G-PCC descriptor is signaled at a pre-selection level using a pre-selection element within a period element. Claim 17 In claim 11, the method wherein the G-PCC adaptation set comprises one or more expressions. Claim 18 In claim 17, the method wherein one or more of the above expressions include at least one of bitrate, resolution, number of channels, or quality level. Claim 19 A method according to claim 11, wherein when the G-PCC tile identifier is determined from the GPCCTileID descriptor, the GPCCTileID descriptor is used to distinguish different G-PCC tile streams. Claim 20 In claim 11, a method wherein, when the G-PCC tile identifier is determined from the GPCCTileID descriptor, the GPCCTileID descriptor is signaled at the adaptive set level or the pre-selection level.

Images