Tile track for geometry-based point cloud data

Abstract

A method and apparatus include receiving a timing metadata track identifying point cloud tiles corresponding to one or more spatial regions within a point cloud scene. A decoding device determines one or more point cloud tiles to use for rendering an image. One or more geometry tile tracks corresponding to the determined one or more point cloud tiles are retrieved via a communication network. Each geometry tile track includes point cloud geometry data for a respective tile. The retrieved geometry tile tracks are processed.

Description

[0001] Cross-references to related applications

[0002] This application is a non-provisional filing of U.S. Provisional Patent Application Serial No. 63 / 063,167, entitled "Tile Tracks for Geometry-Based Point Cloud Data," filed August 7, 2020, and claiming the benefit of that patent application pursuant to 35 U.S.C. § 119(e), which is incorporated herein by reference in its entirety; and a non-provisional filing of U.S. Provisional Patent Application Serial No. 63 / 087,683, entitled "Tile Tracks for Geometry-Based Point Cloud Data," filed October 5, 2020, and claiming the benefit of that patent application pursuant to 35 U.S.C. § 119(e), which is incorporated herein by reference in its entirety; and a non-provisional filing of U.S. Provisional Patent Application Serial No. 63 / 087,683, entitled "Tile Tracks for Geometry-Based Point Cloud Data," filed March 12, 2021, entitled "Tile Tracks for Geometry-Based Point Cloud Data." The following are non-provisional filings: U.S. Provisional Patent Application Serial No. 63 / 160,223 entitled “Tile Tracks for Geometry-Based PointCloud Data”, filed on July 1, 2021, entitled “Tile Tracks for Geometry-Based PointCloud Data”, which are also claimed under 35 U.S. SC § 119(e) and are incorporated herein by reference in their entirety: Background Technology

[0003] High-quality 3D point clouds have recently emerged as a high-level representation for immersive media. A point cloud consists of a set of points represented in 3D space using coordinates indicating the location of each point and one or more attributes, such as color, transparency, laser reflectivity, or material properties associated with each point. Point clouds can be captured in a variety of ways. For example, one technique for capturing point clouds uses multiple cameras and depth sensors. Light detection and ranging (LiDAR) laser scanners are also commonly used to capture point clouds. The number of points required to realistically reconstruct objects and scenes using point clouds is approximately several million (or even billions). Therefore, efficient representation and compression are essential for storing and / or transmitting point cloud data.

[0004] Recent advancements in 3D point capture and rendering technologies have enabled novel applications in fields such as telepresence, virtual reality, and large-scale dynamic 3D mapping (N16331, "Use Cases for Point Cloud Compression (PCC)", MPEG 115, June 2016). The 3D Graphics subgroup of the ISO / IEC JTC1 / SC29 / WG11 Moving Picture Experts Group (MPEG) is currently developing two 3D point cloud compression (PCC) standards: a geometry-based compression standard for static point clouds and a video-based compression standard for dynamic point clouds. The goal of these standards is to support efficient and interoperable storage and transmission of 3D point clouds. One of the requirements of these standards is support for lossy and / or lossless encoding of point cloud geometric coordinates and attributes.

[0005] New media such as virtual reality and immersive 3D graphics have generated considerable interest. High-quality 3D point clouds have recently emerged as a high-level representation of immersive media, enabling new forms of interaction and communication with the virtual world. The vast amount of information required to represent such point clouds necessitates efficient encoding algorithms. The MPEG 3DG Working Group is currently developing the ISO / IEC 23090-9 standard (N19328, “Text of ISO / IEC DIS 23090-9 Geometry-based Point Cloud Compression”, MPEG 131, July 2020) for geometry-based compression of point clouds. Work on another standard for carrying G-PCC, ISO / IEC 23090-18 (“ISO / IEC 23090-18 Carriage of Geometry-based Point Cloud Compression Data”, MPEG 130, April 2020) is underway and is in the Draft Working (WD) stage.

[0006] The latest draft of ISO / IEC WD 23090-18 only supports carrying geometry-based point cloud compression (G-PCC data) in a single track or multiple tracks, with each track carrying G-PCC component data. Even when a user is only interested in a specific region / object within the G-PCC content, this type of support is problematic in streaming applications forced to download and decode all G-PCC component information. The latest DIS version of ISO / IEC 23090-18 (N00075, “Text of ISO / IEC DIS 23090-18 Carriage of Geometry-based Point Cloud Compression Data”, MPEG 132, October 2020) supports carrying non-timing G-PCC data, but does not provide effective partial access support for non-timing G-PCC data.

[0007] Several methods for overcoming the aforementioned shortcomings are described. Summary of the Invention

[0008] A method and apparatus include receiving timing metadata tracks that identify point cloud tiles corresponding to one or more spatial regions within a point cloud scene. A decoding device determines one or more point cloud tiles to be used for rendering an image. One or more geometric tile tracks corresponding to the determined one or more point cloud tiles are retrieved via a communication network. Each geometric tile track includes point cloud geometry data of the corresponding tile. The retrieved geometric tile tracks are processed. Attached Figure Description

[0009] In the accompanying drawings, the same reference numerals indicate the same elements, wherein:

[0010] Figure 1A This is a system diagram illustrating an exemplary communication system that can be implemented in one or more of the disclosed embodiments;

[0011] Figure 1B This illustrates that, according to one implementation scheme, it is possible to Figure 1A A system diagram of an exemplary wireless transmit / receive unit (WTRU) used within the communication system shown;

[0012] Figure 1C This illustrates that, according to one implementation scheme, it is possible to Figure 1A A system diagram of an exemplary radio access network (RAN) and an exemplary core network (CN) used within the communication system shown;

[0013] Figure 1D This illustrates that, according to one implementation scheme, it is possible to Figure 1AA system diagram of another exemplary RAN and another exemplary CN used in the communication system shown;

[0014] Figure 2 This is a system interface diagram illustrating an example set of interfaces for two servers and one client according to some implementation schemes.

[0015] Figure 3A It is an exemplary point cloud of a scene or image that can be captured and processed by some implementation schemes.

[0016] Figure 3B An exemplary point cloud of an object or image that can be captured and processed by some implementation schemes is shown.

[0017] Figure 4 shows the geometry-based point cloud compressed data file structure;

[0018] Figure 5 shows an exemplary structure of an encoded G-PCC data file stored in a single track;

[0019] Figure 6 shows the structure of a multi-track G-PCC data file container;

[0020] Figure 7 This is a diagram illustrating an example of carrying non-timing G-PCC data;

[0021] Figure 8 This is a diagram showing a G-PCC tile item that includes multiple G-PCC tiles;

[0022] Figure 9 This is a diagram illustrating the encapsulation of G-PCC data files in multiple tile tracks according to an implementation scheme;

[0023] Figure 10 This is a diagram illustrating the alternative selection and grouping of tracks for storing G-PCC content according to the implementation scheme.

[0024] Figure 11 This is a diagram showing the alternative basic tracks for G-PCC tiles, as well as the grouping of corresponding geometric tile tracks and attribute tile tracks;

[0025] Figure 12 This is a flowchart illustrating a method for decoding geometry-based point cloud tiles according to an implementation scheme;

[0026] Figure 13 This is a diagram showing the grouping of alternative attribute tracks and corresponding geometric tracks of multiple tracks according to the implementation scheme;

[0027] Figure 14 This is a diagram showing the grouping of alternative attribute tile tracks and corresponding geometric tile tracks according to the implementation scheme;

[0028] Figure 15This is a diagram illustrating the encapsulation of a G-PCC data file with track references between a base track, multiple tile tracks, and 3D spatial region timing metadata tracks according to an implementation scheme;

[0029] Figure 16 This is a diagram showing partial access to non-timing G-PCC data with N G-PCC tiles;

[0030] Figure 17 This is a diagram showing partial access to non-timed G-PCC data of a G-PCC item of type 'gpe1';

[0031] Figure 18 This is a diagram showing partial access to non-timed G-PCC data for a G-PCC item of type 'gpci'.

[0032] Exemplary system for implementing the implementation scheme

[0033] Figure 1A This is a schematic diagram illustrating an exemplary communication system 100 that can be implemented in one or more of the disclosed embodiments. Communication system 100 can be a multiple access system providing content such as voice, data, video, messaging, and broadcasting to multiple wireless users. Communication system 100 enables multiple wireless users to access such content through the sharing of system resources (including wireless bandwidth). For example, communication system 100 may employ one or more channel access methods, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal FDMA (OFDMA), Single Carrier FDMA (SC-FDMA), Zero-Tail Unique Word DFT Extended OFDM (ZT UW DTS-s OFDM), Unique Word OFDM (UW-OFDM), Resource Block Filtered OFDM, Filter Bank Multicarrier (FBMC), etc.

[0034] like Figure 1AAs shown, the communication system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, RAN 104 / 113, CN 106 / 115, Public Switched Telephone Network (PSTN) 108, Internet 110, and other networks 112. However, it should be understood that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Each of the WTRUs 102a, 102b, 102c, and 102d may be any type of device configured to operate and / or communicate in a wireless environment. As examples, WTRUs 102a, 102b, 102c, and 102d (any of which may be referred to as a “station” and / or “STA”) may be configured to transmit and / or receive wireless signals and may include user equipment (UE), mobile stations, fixed or mobile user units, subscription-based units, pagers, cellular phones, personal digital assistants (PDAs), smartphones, laptops, netbooks, personal computers, wireless sensors, hotspots or Mi-Fi devices, Internet of Things (IoT) devices, watches or other wearable devices, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in industrial and / or automated processing chain environments), consumer electronics devices, devices operating on commercial and / or industrial wireless networks, etc. Any of UEs 102a, 102b, 102c, and 102d may be interchangeably referred to as WTRUs.

[0035] The communication system 100 may also include base station 114a and / or base station 114b. Each of base stations 114a and 114b may be any type of device configured to wirelessly interface with at least one of WTRUs 102a, 102b, 102c, and 102d to facilitate access to one or more communication networks, such as CN106 / 115, Internet 110, and / or other networks 112. As an example, base stations 114a and 114b may be base transceiver stations (BTS), Node Bs, evolved Node Bs, home Node Bs, home evolved Node Bs, gNBs, NR Node Bs, site controllers, access points (APs), wireless routers, etc. Although base stations 114a and 114b are each depicted as a single element, it should be understood that base stations 114a and 114b may include any number of interconnected base stations and / or network elements.

[0036] Base station 114a may be part of RAN 104 / 113, which may also include other base stations and / or network elements (not shown), such as base station controllers (BSCs), radio network controllers (RNCs), relay nodes, etc. Base station 114a and / or base station 114b may be configured to transmit and / or receive radio signals on one or more carrier frequencies (which may be referred to as cells (not shown)). These frequencies may be in licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide coverage of radio services to a specific geographic area, which may be relatively fixed or changeable over time. A cell may be further divided into cell sectors. For example, the cell associated with base station 114a may be divided into three sectors. Therefore, in an embodiment, base station 114a may include three transceivers, i.e., one transceiver for each sector of the cell. In an embodiment, base station 114a may employ multiple-input multiple-output (MIMO) technology and may utilize multiple transceivers for each sector of the cell. For example, beamforming may be used to transmit and / or receive signals in desired spatial directions.

[0037] Base stations 114a and 114b can communicate with one or more of WTRUs 102a, 102b, 102c, and 102d via air interface 116, which can be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). Any suitable radio access technology (RAT) can be used to establish air interface 116.

[0038] More specifically, as noted above, the communication system 100 may be a multiple access system and may employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, base stations 114a and WTRUs 102a, 102b, and 102c in RAN 104 / 113 may implement radio technologies such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may use Wideband CDMA (WCDMA) to establish air interfaces 115 / 116 / 117. WCDMA may include communication protocols such as High-Speed ​​Packet Access (HSPA) and / or evolved HSPA (HSPA+). HSPA may include High-Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High-Speed ​​UL Packet Access (HSUPA).

[0039] In the implementation scheme, base station 114a and WTRUs 102a, 102b, 102c can implement radio technologies such as evolved UMTS terrestrial radio access (E-UTRA), which can use Long Term Evolution (LTE) and / or Advanced LTE (LTE-A) and / or Advanced LTE Pro (LTE-A Pro) to establish air interface 116.

[0040] In one implementation, base station 114a and WTRUs 102a, 102b, 102c can implement radio technologies such as NR radio access, which can use New Radio (NR) to establish air interface 116.

[0041] In the implementation scheme, base station 114a and WTRUs 102a, 102b, and 102c can implement multiple radio access technologies. For example, base station 114a and WTRUs 102a, 102b, and 102c can, for instance, use a dual connectivity (DC) principle to implement both LTE and NR radio access together. Therefore, the air interface used by WTRUs 102a, 102b, and 102c can be characterized by multiple types of radio access technologies and / or transmissions sent to / from multiple types of base stations (e.g., eNBs and gNBs).

[0042] In other implementations, base station 114a and WTRUs 102a, 102b, and 102c can implement radio technologies such as IEEE 802.11 (i.e., WiFi), IEEE 802.16 (i.e., WiMAX), CDMA2000, CDMA2000 1X, CDMA2000EV-DO, Provisional Standard 2000 (IS-2000), Provisional Standard 95 (IS-95), Provisional Standard 856 (IS-856), Global System for Mobile Communications (GSM), GSM Enhanced Data Rate Evolution (EDGE), and GSM EDGE (GERAN).

[0043] Figure 1ABase station 114b can be, for example, a wireless router, a home node B, a home evolution node B, or an access point, and can utilize any suitable RAT to facilitate wireless connectivity in localized areas such as commercial locations, homes, vehicles, campuses, industrial facilities, air corridors (e.g., for use by drones), roads, etc. In one embodiment, base station 114b and WTRUs 102c, 102d can implement radio technologies such as IEEE 802.11 to establish a wireless local area network (WLAN). In another embodiment, base station 114b and WTRUs 102c, 102d can implement radio technologies such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, base station 114b and WTRUs 102c, 102d can utilize cellular-based RATs (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.) to establish picocells or femtocells. Figure 1A As shown, base station 114b may have a direct connection to Internet 110. Therefore, base station 114b may not need to access Internet 110 via CN 106 / 115.

[0044] RAN 104 / 113 can communicate with CN 106 / 115, which can be any type of network configured to provide voice, data, application, and / or Voice over Internet Protocol (VoIP) services to one or more of WTRU102a, 102b, 102c, and 102d. Data can have different Quality of Service (QoS) requirements, such as different throughput requirements, latency requirements, error tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. CN 106 / 115 can provide call control, billing services, location-based services, prepaid calling, internet connectivity, video distribution, etc., and / or perform advanced security functions such as user authentication. Although not explicitly stated... Figure 1A As shown, but it should be understood that RAN 104 / 113 and / or CN 106 / 115 can communicate directly or indirectly with other RANs that use the same RAT as RAN 104 / 113 or a different RAT. For example, in addition to being connected to RAN 104 / 113 which can utilize NR radio technology, CN 106 / 115 can also communicate with another RAN (not shown) that uses GSM, UMTS, CDMA2000, WiMAX, E-UTRA, or WiFi radio technology.

[0045] CN 106 / 115 may also act as a gateway for WTRU 102a, 102b, 102c, 102d to access PSTN 108, Internet 110, and / or other networks 112. PSTN 108 may include a circuit-switched telephone network providing Common Old-Style Telephone Service (POTS). Internet 110 may include a global system of interconnected computer networks and devices using common communication protocols such as Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and / or Internet Protocol (IP) from the TCP / IP Internet Protocol suite. Network 112 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, network 112 may include another CN connected to one or more RANs, which may use the same RAT as RAN 104 / 113 or a different RAT.

[0046] Some or all of the WTRUs 102a, 102b, 102c, and 102d in the communication system 100 may include multi-mode capabilities (e.g., WTRUs 102a, 102b, 102c, and 102d may include multiple transceivers for communicating with different wireless networks via different wireless links). For example, Figure 1A The WTRU 102c shown can be configured to communicate with a base station 114a that can employ cellular-based radio technology and with a base station 114b that can employ IEEE 802 radio technology.

[0047] Figure 1B This is a system diagram illustrating an exemplary WTRU 102. (See diagram below.) Figure 1B As shown, WTRU 102 may include a processor 118, a transceiver 120, a transmitting / receiving element 122, a speaker / microphone 124, a keypad 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power supply 134, a Global Positioning System (GPS) chipset 136, and / or other peripheral devices 138, etc. It should be understood that, while remaining consistent with the implementation, WTRU 102 may include any sub-combination of the foregoing elements.

[0048] Processor 118 can be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. Processor 118 can perform signal encoding, data processing, power control, input / output processing, and / or any other functions that enable WTRU 102 to operate in a wireless environment. Processor 118 can be coupled to transceiver 120, which can be coupled to transmitting / receiving element 122. Although Figure 1B The processor 118 and transceiver 120 are depicted as separate components, but it should be understood that the processor 118 and transceiver 120 may be integrated together in an electronic package or chip.

[0049] Transmitting / receiving element 122 may be configured to transmit signals to or receive signals from a base station (e.g., base station 114a) via air interface 116. For example, in one embodiment, transmitting / receiving element 122 may be an antenna configured to transmit and / or receive RF signals. In one embodiment, transmitting / receiving element 122 may be a transmitter / detector configured to transmit and / or receive, for example, IR, UV, or visible light signals. In yet another embodiment, transmitting / receiving element 122 may be configured to transmit and / or receive RF and optical signals. It should be understood that transmitting / receiving element 122 may be configured to transmit and / or receive any combination of wireless signals.

[0050] Although the transmitting / receiving element 122 is in Figure 1B While depicted as a single element, WTRU 102 may include any number of transmitting / receiving elements 122. More specifically, WTRU 102 may employ MIMO technology. Therefore, in one embodiment, WTRU 102 may include two or more transmitting / receiving elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals via air interface 116.

[0051] Transceiver 120 can be configured to modulate signals transmitted by transmitting / receiving element 122 and demodulate signals received by transmitting / receiving element 122. As noted above, WTRU 102 may have multi-mode capability. For example, transceiver 120 may therefore include multiple transceivers to enable WTRU 102 to communicate via various RATs (such as NR and IEEE 802.11).

[0052] The processor 118 of WTRU 102 may be coupled to a speaker / microphone 124, a keypad 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) unit or an organic light-emitting diode (OLED) display unit) and may receive user input data therefrom. The processor 118 may also output user data to the speaker / microphone 124, keypad 126, and / or display / touchpad 128. Additionally, the processor 118 may access information from any type of suitable memory (such as non-removable memory 130 and / or removable memory 132) and store data in any type of suitable memory. Non-removable memory 130 may include random access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. Removable memory 132 may include a user identity module (SIM) card, memory stick, secure digital storage (SD) card, etc. In other embodiments, the processor 118 may access information from memory that is not physically located on WTRU 102 (such as on a server or home computer (not shown)) and store data in that memory.

[0053] The processor 118 may receive power from the power supply 134 and may be configured to distribute and / or control power to other components in the WTRU 102. The power supply 134 may be any suitable device for powering the WTRU 102. For example, the power supply 134 may include one or more dry cell battery packs (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0054] The processor 118 may also be coupled to a GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) about the current location of the WTRU 102. In addition to or instead of the information from the GPS chipset 136, the WTRU 102 may receive location information from base stations (e.g., base stations 114a, 114b) via air interface 116 and / or determine its location based on the timing of signals received from two or more nearby base stations. It should be understood that, while remaining consistent with the implementation, the WTRU 102 may acquire location information using any suitable location determination method.

[0055] The processor 118 may also be coupled to other peripheral devices 138, which may include one or more software and / or hardware modules providing additional features, functions, and / or wired or wireless connectivity. For example, peripheral device 138 may include an accelerometer, electronic compass, satellite transceiver, digital camera (for photos and / or video), Universal Serial Bus (USB) port, vibration device, television transceiver, hands-free headset, etc. Modules, FM radio units, digital music players, media players, video game player modules, internet browsers, virtual reality and / or augmented reality (VR / AR) devices, activity trackers, etc. Peripheral devices 138 may include one or more sensors, which may be one or more of the following: gyroscopes, accelerometers, Hall effect sensors, magnetometers, orientation sensors, proximity sensors, temperature sensors, time sensors; geolocation sensors; altimeters, light sensors, touch sensors, magnetometers, barometers, gesture sensors, biometric sensors, and / or humidity sensors.

[0056] WTRU 102 may include a full-duplex radio for which the transmission and reception of some or all signals (e.g., associated with specific subframes for UL (e.g., for transmission) and downlink (e.g., for reception)) may be concurrent and / or simultaneous. The full-duplex radio may include an interference management unit for reducing and / or substantially eliminating self-interference through signal processing via hardware (e.g., a choke) or via a processor (e.g., a separate processor (not shown) or via processor 118). In one embodiment, WTRU 102 may include a half-duplex radio for which the transmission and reception of some or all signals (e.g., associated with specific subframes for UL (e.g., for transmission) or downlink (e.g., for reception)) may be concurrent and / or simultaneous.

[0057] Figure 1C This is a system diagram illustrating RAN 104 and CN 106 according to an implementation scheme. As described above, RAN 104 can communicate with WTRUs 102a, 102b, and 102c via air interface 116 using E-UTRA radio technology. RAN 104 can also communicate with CN 106.

[0058] RAN 104 may include evolved Node Bs 160a, 160b, and 160c; however, it should be understood that RAN 104 may include any number of evolved Node Bs while remaining consistent with the implementation scheme. Each evolved Node B 160a, 160b, and 160c may include one or more transceivers for communicating with WTRUs 102a, 102b, and 102c via air interface 116. In the implementation scheme, evolved Node Bs 160a, 160b, and 160c may implement MIMO technology. Therefore, evolved Node B 160a may, for example, use multiple antennas to transmit radio signals to and / or receive radio signals from WTRU 102a.

[0059] Each of the evolved nodes B 160a, 160b, and 160c can be associated with a specific cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, and user scheduling in the UL and / or DL, etc. Figure 1C As shown, evolution nodes B 160a, 160b, and 160c can communicate with each other via the X2 interface.

[0060] Figure 1C The CN 106 shown may include a Mobility Management Entity (MME) 162, a Serving Gateway (SGW) 164, and a Packet Data Network (PDN) Gateway (or PGW) 166. While each of the foregoing elements is depicted as part of the CN 106, it should be understood that any of these elements may be owned and / or operated by an entity other than a CN operator.

[0061] The MME 162 can connect to each of the evolved nodes B 162a, 162b, and 162c in RAN 104 via the S1 interface and can be used as a control node. For example, the MME 162 can be responsible for authenticating users of WTRUs 102a, 102b, and 102c, bearer activation / deactivation, selecting a specific serving gateway during the initial attachment of WTRUs 102a, 102b, and 102c, etc. The MME 162 can provide control plane functions for handover between RAN 104 and other RANs (not shown) employing other radio technologies such as GSM and / or WCDMA.

[0062] The SGW 164 can connect to each of the evolved Nodes B 160a, 160b, and 160c in RAN 104 via the S1 interface. The SGW 164 typically routes and forwards user data packets to and from WTRUs 102a, 102b, and 102c. The SGW 164 can perform other functions such as anchoring the user plane during inter-evolved Node B handovers, triggering paging when DL data is available for WTRUs 102a, 102b, and 102c, and managing and storing the context of WTRUs 102a, 102b, and 102c.

[0063] SGW 164 can be connected to PGW 166, which provides WTRU 102a, 102b, 102c with access to packet-switched networks (such as Internet 110) to facilitate communication between WTRU 102a, 102b, 102c and IP-enabled devices.

[0064] CN 106 can facilitate communication with other networks. For example, CN 106 can provide WTRUs 102a, 102b, and 102c with access to a circuit-switched network (such as PSTN 108) to facilitate communication between WTRUs 102a, 102b, and 102c and conventional landline communication equipment. For example, CN 106 may include, or be able to communicate with, an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that serves as an interface between CN 106 and PSTN 108. Additionally, CN 106 can provide WTRUs 102a, 102b, and 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0065] Despite WTRU in Figures 1A to 1D While described as a wireless terminal, it is conceivable that in some representative implementations, such a terminal may (e.g., temporarily or permanently) use a wired communication interface with a communication network.

[0066] In a representative implementation, the other network 112 may be a WLAN.

[0067] A WLAN in Basic Services Set (BSS) mode may have an access point (AP) for the BSS and one or more stations (STAs) associated with the AP. The AP may have access or an interface to a distribution system (DS) or another type of wired / wireless network that carries traffic to and / or carries traffic out of the BSS. Traffic originating outside the BSS and destined for a STA can reach and be delivered to the STA via the AP. Traffic originating from a STA and destined for a destination outside the BSS can be sent to the AP for delivery to the appropriate destination. Traffic between STAs within the BSS can be sent via the AP, for example, where a source STA can send traffic to the AP, and the AP can deliver the traffic to the destination STA. Traffic between STAs within the BSS can be considered and / or referred to as point-to-point traffic. Point-to-point traffic can be sent between source and destination STAs (e.g., directly between them) using Direct Link Establishment (DLS). In some representative implementations, the DLS may use 802.11e DLS or 802.11z Tunneled DLS (TDLS). WLANs using the Standalone BSS (IBSS) mode may not have an access point (AP), and STAs within the IBSS or using the IBSS (e.g., all STAs) can communicate directly with each other. The IBSS communication mode may sometimes be referred to as the "ad-hoc" communication mode in this document.

[0068] When operating in 802.11ac infrastructure mode or a similar mode, the AP can transmit beacons on a fixed channel, such as the primary channel. The primary channel can be of fixed width (e.g., a 20 MHz wide bandwidth) or dynamically set via signaling. The primary channel can be the operating channel of the BSS and can be used by the STA to establish a connection with the AP. In some representative implementations, such as in an 802.11 system, Carrier Sense Multiple Access / Collision Avoidance (CSMA / CA) can be implemented. For CSMA / CA, each STA (including the AP) can listen to the primary channel. If the primary channel is listened to / detected and / or determined to be busy by a particular STA, that STA can back off. A single STA (e.g., only one station) can transmit at any given time within a given BSS.

[0069] High-throughput (HT) STAs can communicate using a 40MHz wide channel, for example, by combining a primary 20MHz channel with adjacent or non-adjacent 20MHz channels to form a 40MHz wide channel.

[0070] The Very High Throughput (VHT) STA supports channels with widths of 20MHz, 40MHz, 80MHz, and / or 160MHz. 40MHz and / or 80MHz channels can be formed by combining consecutive 20MHz channels. A 160MHz channel can be formed by combining eight consecutive 20MHz channels, or by combining two non-consecutive 80MHz channels (this can be referred to as an 80+80 configuration). For the 80+80 configuration, after channel coding, data can be processed by a segment parser that can split the data into two streams. Inverse Fast Fourier Transform (IFFT) processing and time-domain processing can be performed separately on each stream. These streams can be mapped to two 80MHz channels, and data can be transmitted via the transmitting STA. At the receiver of the receiving STA, the operations described above for the 80+80 configuration can be reversed, and the combined data can be sent to Media Access Control (MAC).

[0071] 802.11af and 802.11ah support operating modes below 1 GHz. Compared to those used in 802.11n and 802.11ac, 802.11af and 802.11ah reduce channel operating bandwidth and carrier. 802.11af supports 5 MHz, 10 MHz, and 20 MHz bandwidths in the TV white space (TVWS) spectrum, while 802.11ah supports 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz bandwidths using non-TVWS spectrum. According to representative implementations, 802.11ah may support instrument-type control / machine-type communications, such as MTC devices in macro coverage areas. MTC devices may have certain capabilities, such as limited capabilities, including support (e.g., only support) certain bandwidths and / or limited bandwidths. MTC devices may include batteries with battery life above a threshold (e.g., to maintain a very long battery life).

[0072] WLAN systems supporting multiple channels, as well as channel bandwidths such as 802.11n, 802.11ac, 802.11af, and 802.11ah, include channels that can be designated as primary channels. A primary channel can have a bandwidth equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel can be set and / or limited by STAs operating in the BSS (each supporting a minimum bandwidth operating mode). In the 802.11ah example, for STAs supporting (e.g., only supporting) a 1MHz mode (e.g., MTC type devices), the primary channel can be 1MHz wide, even if the AP and other STAs in the BSS support 2MHz, 4MHz, 8MHz, 16MHz, and / or other channel bandwidth operating modes. Carrier Sense and / or Network Allocation Vector (NAV) settings can depend on the status of the primary channel. If the primary channel is busy, for example, because an STA (supporting only the 1MHz operating mode) is transmitting to the AP, the entire available band can be considered busy even if most of the band remains idle and potentially available.

[0073] In the United States, the available frequency bands for 802.11ah are 902MHz to 928MHz. In South Korea, the available frequency bands are 917.5MHz to 923.5MHz. In Japan, the available frequency bands are 916.5MHz to 927.5MHz. The total available bandwidth for 802.11ah ranges from 6MHz to 26MHz, depending on the country code.

[0074] Figure 1D This is a system diagram illustrating RAN 113 and CN 115 according to one implementation scheme. As noted above, RAN 113 can communicate with WTRU 102a, 102b, and 102c via air interface 116 using NR radio technology. RAN 113 can also communicate with CN 115.

[0075] RAN 113 may include gNBs 180a, 180b, and 180c; however, it should be understood that RAN 113 may include any number of gNBs while remaining consistent with the implementation scheme. Each of gNBs 180a, 180b, and 180c may include one or more transceivers for communication with WTRUs 102a, 102b, and 102c via air interface 116. In the implementation scheme, gNBs 180a, 180b, and 180c may implement MIMO technology. For example, gNBs 180a and 180b may utilize beamforming to transmit signals to and / or receive signals from gNBs 180a, 180b, and 180c. Therefore, gNB 180a may, for example, use multiple antennas to transmit radio signals to and / or receive radio signals from WTRU 102a. In the implementation scheme, gNBs 180a, 180b, and 180c can implement carrier aggregation technology. For example, gNB 180a can transmit multiple component carriers to WTRU 102a (not shown). Subgroups of these component carriers can be on unlicensed spectrum, while the remaining component carriers can be on licensed spectrum. In the implementation scheme, gNBs 180a, 180b, and 180c can implement Cooperative Multipoint (CoMP) technology. For example, WTRU 102a can receive cooperative transmissions from gNBs 180a and 180b (and / or gNB 180c).

[0076] WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c using transmissions associated with an scalable set of parameters. For example, OFDM symbol spacing and / or OFDM subcarrier spacing can vary depending on different transmissions, different cells, and / or different portions of the radio transmission spectrum. WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c using subframes or transmission time intervals (TTIs) of various or scalable lengths (e.g., containing different numbers of OFDM symbols and / or continuously varying absolute time lengths).

[0077] gNBs 180a, 180b, and 180c can be configured to communicate with WTRUs 102a, 102b, and 102c in standalone and / or non-standalone configurations. In standalone configuration, WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c without accessing other RANs (e.g., evolved Node Bs 160a, 160b, and 160c). In standalone configuration, WTRUs 102a, 102b, and 102c can use one or more of gNBs 180a, 180b, and 180c as mobility anchors. In standalone configuration, WTRUs 102a, 102b, and 102c can communicate with gNBs 180a, 180b, and 180c using signals in unlicensed frequency bands. In a non-standalone configuration, WTRUs 102a, 102b, and 102c can communicate or connect to gNBs 180a, 180b, and 180c, and also communicate or connect to other RANs (such as evolved Node Bs 160a, 160b, and 160c). For example, WTRUs 102a, 102b, and 102c can implement DC principles to communicate substantially simultaneously with one or more gNBs 180a, 180b, and 180c and one or more evolved Node Bs 160a, 160b, and 160c. In a non-standalone configuration, evolved Node Bs 160a, 160b, and 160c can be used as mobility anchors for WTRUs 102a, 102b, and 102c, and gNBs 180a, 180b, and 180c can provide additional coverage and / or throughput for serving WTRUs 102a, 102b, and 102c.

[0078] Each of gNBs 180a, 180b, and 180c can be associated with a specific cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, user scheduling in UL and / or DL, network slicing support, dual connectivity, interoperability between NR and E-UTRA, routing of user plane data to User Plane Functions (UPF) 184a and 184b, routing of control plane information to Access and Mobility Management Functions (AMF) 182a and 182b, etc. Figure 1D As shown, gNB 180a, 180b, and 180c can communicate with each other via the Xn interface.

[0079] Figure 1DThe CN 115 shown may include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one Session Management Function (SMF) 183a, 183b, and possibly a Data Network (DN) 185a, 185b. Although each of the foregoing elements is depicted as part of the CN 115, it should be understood that any one of these elements may be owned and / or operated by an entity other than a CN operator.

[0080] AMF 182a and 182b can connect to one or more of gNB 180a, 180b, and 180c via the N2 interface in RAN 113 and can be used as control nodes. For example, AMF 182a and 182b can be responsible for authenticating users of WTRU 102a, 102b, and 102c, supporting network slicing (e.g., handling different PDU sessions with different requirements), selecting specific SMF 183a and 183b, managing registration areas, terminating NAS signaling, mobility management, etc. AMF 182a and 182b can use network slicing to customize CN support for WTRU 102a, 102b, and 102c based on the type of service used by WTRU 102a, 102b, and 102c. For example, different network slices can be established for different use cases, such as services that rely on Ultra-Reliable Low Latency (URLLC) access, services that rely on Enhanced Mobile Broadband (eMBB) access, services for Machine Type Communication (MTC) access, etc. The AMF162 can provide control plane functions for switching between RAN 113 and other RANs (not shown) that employ other radio technologies, such as LTE, LTE-A, LTE-A Pro, and / or non-3GPP access technologies (such as WiFi)).

[0081] SMFs 183a and 183b can connect to AMFs 182a and 182b in CN 115 via the N11 interface. SMFs 183a and 183b can also connect to UPFs 184a and 184b in CN 115 via the N4 interface. SMFs 183a and 183b can select and control UPFs 184a and 184b, and configure traffic routing through UPFs 184a and 184b. SMFs 183a and 183b can perform other functions, such as managing and allocating UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, and providing downlink data notifications. PDU session types can be IP-based, non-IP-based, Ethernet-based, etc.

[0082] UPF 184a and 184b can be connected via the N3 interface to one or more of the gNBs 180a, 180b, and 180c in RAN 113. These gNBs can provide WTRU 102a, 102b, and 102c with access to packet-switched networks (such as Internet 110) to facilitate communication between WTRU 102a, 102b, and 102c and IP-enabled devices. UPF 184 and 184b can perform other functions such as routing and forwarding packets, enforcing user plane policies, supporting multihomed PDU sessions, handling user plane QoS, buffering downlink packets, and providing mobility anchoring.

[0083] CN 115 may facilitate communication with other networks. For example, CN 115 may include, or be able to communicate with, an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) serving as an interface between CN115 and PSTN 108. Additionally, CN115 may provide WTRUs 102a, 102b, and 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers. In an embodiment, WTRUs 102a, 102b, and 102c may be connected to DNs 185a and 185b via UPFs 184a and 184b through their N3 interfaces and their N6 interfaces with local data networks (DNs) 185a and 185b.

[0084] Given Figures 1A to 1D as well as Figures 1A to 1D The corresponding descriptions herein refer to one or more of the functions described below, which may be performed by one or more emulation devices (not shown): WTRU102a-d, base station 114a-b, evolved Node B 160a-c, MME 162, SGW 164, PGW 166, gNB 180a-c, AMF182a-b, UPF184a-b, SMF 183a-b, DN 185a-b, and / or any other device described herein. An emulation device may be one or more devices configured to mimic one or more of the functions described herein. For example, an emulation device may be used to test other devices and / or simulate network and / or WTRU functions.

[0085] Simulation devices can be designed to perform one or more tests on other devices in laboratory and / or carrier network environments. For example, the one or more simulation devices may perform one or more or all functions while being fully or partially implemented and / or deployed as part of a wired and / or wireless communication network to test other devices within the communication network. The one or more simulation devices may perform one or more or all functions while being temporarily implemented / deployed as part of a wired and / or wireless communication network. Simulation devices may be directly coupled to another device for testing purposes and / or may use over-the-air wireless communication to perform tests.

[0086] The one or more simulation devices may perform one or more (including all) functions without being implemented / deployed as part of a wired and / or wireless communication network. For example, the simulation devices may be used in test scenarios within a test laboratory and / or non-deployed (e.g., testing) wired and / or wireless communication networks to perform testing of one or more components. The one or more simulation devices may be test equipment. Direct RF coupling and / or wireless communication via an RF circuit system (e.g., which may include one or more antennas) may be used by the simulation devices to transmit and / or receive data.

[0087] Figure 2 This is a system interface diagram illustrating an example set of interfaces for two servers and one client according to some embodiments. According to this example, one server 202 may be a point cloud server, and the other server 210 may be a neural network server. In some embodiments, the servers may be consistent. Both servers are connected to the Internet 110 and other networks 112. The client 218 is also connected to the Internet 110 and other networks 112, thereby enabling communication between all three nodes 202, 210, and 218. Each node 202, 210, and 218 includes processors 204, 212, and 220, non-transitory computer-readable storage media 206, 214, and 224, and executable instructions 208, 216, and 226 contained within the storage media 206, 214, and 224, which can be executed by the processors 204, 212, and 220 to perform the methods or portions thereof disclosed herein. As shown, in some embodiments, the client may include a graphics processor 222 for rendering 3D video for a display such as a head-mounted display (HMD) 228. Any one or all nodes may include WTRUs and communicate via a network, as described above. Figure 1A and Figure 1B As stated above.

[0088] In some implementations, system 200 may include a point cloud server 202, a neural network server 210, and / or a client 218, the client including one or more processors 204, 212, 220 and one or more non-transitory computer-readable media 206, 214, 224 storing instructions 208, 216, 226, which, when executed by processor 204, 212, 220, are operable to perform the methods disclosed herein. In some implementations, node 218 may include one or more graphics processors 222. In some implementations, nodes 202, 210, 218 may include one or more sensors.

[0089] Figure 3A An exemplary point cloud of a scene or image is shown, which can be captured and processed by some implementation schemes. Scene 302 includes multiple buildings at a distance and some closer objects imaged from an observer's viewpoint with a certain apparent height. The relative angles with points within the point cloud can change as the observer's viewpoint changes, such as by moving lower or closer to the buildings. The point cloud can be detected in a real-world scene, generated with virtual objects, or generated using any combination of these or other applicable techniques. Region 304 may include one or more tiles.

[0090] Figure 3B An exemplary point cloud of an object or image that can be captured and processed by some implementation schemes is shown. Figure 3B This is a two-dimensional black-and-white line drawing of a three-dimensional point cloud object 306. In a three-dimensional display environment, a point cloud object has points representing three-dimensional coordinates, in which a portion of the object has been detected to be present. Such detection can occur using, for example, 3D sensors such as light detection and ranging (LIDAR), stereo video, and RGB-D cameras. Point cloud data can include, for example, 3D position and radiometric image data or voxels. Detailed Implementation

[0091] Several methods to overcome the aforementioned shortcomings are described. Signaling methods are provided to achieve flexible partial access, such as different portions of an coded point cloud sequence encapsulated in an ISOBMFF container. Methods for providing efficient partial access to non-timing G-PCC data carried in ISOBMFF files are also described.

[0092] Figure 4 illustrates the structure of a data file used for geometry-based point cloud compression (G-PCC). In the latest draft of the international standard (DIS) version of ISO / IEC 23090-9 (N19328, “Text of ISO / IEC DIS 23090-9 Geometry-based Point Cloud Compression”, MPEG 131, July 2020), the G-PCC data file 402 includes a set of G-PCC units 404, also known as Type Length Value (TLV) encapsulation structures, as shown in Figure 4. The syntax of the G-PCC TLV units 404, as described in the latest version of the G-PCC standard's DIS draft, is given in Table 1, where each G-PCC TLV unit 404 has a G-PCC TLV type 406, a G-PCC TLV unit payload length, and a G-PCC TLV unit payload 408. Examples of G-PCC TLV unit payloads 408 include sequence parameter sets, geometric parameter sets, attribute parameter sets, geometric data, attribute data, and frame boundary markers. The tlv_type and associated G-PCC data unit descriptions are shown in Table 2. G-PCC TLV units 104 with unit types 2 and 4 are geometric and attribute data units, as defined in ISO / IEC 23090-9. These data units represent the two main components required to reconstruct the point cloud. The payloads of the geometric and attribute G-PCC units correspond to media data units, such as TLV units, that can be decoded by a G-PCC decoder specified in the corresponding geometric and attribute parameter set G-PCC unit.

[0093]

[0094] Table 1

[0095] tlv_type describe 0 Sequence parameter set 1 Geometric parameter set 2 Geometric Data Unit 3 Attribute parameter set 4 Attribute Data Unit 5 Tile inventory 6 Frame boundary flag 7 Default attribute data unit

[0096] Table 2

[0097] The G-PCC attribute types of known_attribute_label are shown in Table 3.

[0098] known_attribute_label Attribute type 0 color 1 reflectivity 2 Frame Index 3 Material ID 4 transparency 5 normal

[0099] Table 3

[0100] The attribute types of the known_attribute_label in G-PCC are shown in Table 4.

[0101]

[0102]

[0103] Table 4

[0104] The G-PCC File High-Level Syntax (HLS) supports the concept of slices and tile groups for geometric and attribute data. A frame is divided into multiple tiles and slices. A slice is a set of points that can be independently encoded or decoded. A slice consists of one geometric data unit and zero or more attribute data units. Attribute data units may depend on corresponding geometric data units within the same slice. Within a slice, the geometric data unit appears before any associated attribute unit. Advantageously, the data units in a slice are consecutive. The order of slices within a frame is not specified.

[0105] Tile groups can be identified by a common tile identifier. The ISO / IEC 23090-9 standard provides a tile inventory describing the boundary frame of each tile. Tiles can overlap with other tiles within the boundary frame. Each tile contains an index identifying the tile to which it belongs.

[0106] The ISO / IEC 14496 (MPEG-4) standard includes several parts defining file formats for storing time-based media. These formats are based on and derived from the ISO Basic Media File Format (ISOBMFF), which has a structured, media-independent definition. ISOBMFF contains structured and media data information that can be used for the timing presentation of media data such as audio, video, etc. Support for non-timing data, such as metadata at different levels within the file structure, is also provided. The logical structure of the file is a movie structure containing a set of time-parallel tracks. The temporal structure of the file includes tracks containing sample sequences of time, and those sequences are mapped to the timeline of the entire movie. ISOBMFF is based on the concept of frame-structured files. Frame-structured files consist of a series of frames (sometimes called atoms) of a certain size and type. The type can be a 32-bit value and is usually chosen as four printable characters, also known as a four-character code (4CC). Non-timing data can be contained in metadata frames (at the file level) or attached to a stream of timing data (called a track) within a movie frame or movie.

[0107] The multi-track G-PCC data file container structure according to the implementation scheme is shown in Figure 6. The top-level box 602 of ftyp identifies which specification is the 'best use' of the container (also known as the file), the minor version of this specification, and the other sets of specifications that the file follows.

[0108] Within the ISOBMFF container, the top-level box is MovieBox('moov') 604, which contains metadata about the continuous media stream present in the container or file. This metadata is signaled within the hierarchy of the MovieBox boxes (e.g., within TrackBox('trak')). A track represents a continuous media stream present in the file. The media stream itself consists of a sequence of samples, such as audio or video units of the underlying media stream, and is encapsulated within MediaDataBox('mdat') 606, which is located at the top level of the container. The metadata for each track includes a list of sample description entries, each providing the encoding or encapsulation format used in the track and initialization data for processing that format. Each sample is associated with a sample description entry for the track. ISO / IEC 14496-12 provides a tool for defining an explicit timeline mapping for each track. This tool is called an EditList and is signaled using EditListBox with the following syntax, where each entry either forms part of the timeline by mapping or by indicating a 'empty' time to represent a part of the track's timeline, e.g., rendering a timeline mapping to a media-free section, also known as an 'empty' edit. For example:

[0109]

[0110] A point cloud sequence can represent a scene with multiple tiles. In many applications, it is desirable to access individual tiles without having to decode other parts of the scene, for example, for streaming and / or rendering data. Similarly, a point cloud can represent a single object, and a user may expect to access certain parts of the object without decoding the entire point cloud.

[0111] When a G-PCC data file is carried in a single track, the G-PCC encoded data is represented by a single track declaration. Single-track encapsulation of G-PCC data can utilize simple ISOBMFF encapsulation by storing the G-PCC data file in a single track without further processing. Each sample in this track contains one or more G-PCC components. For example, each sample includes one or more TLV encapsulation structures. Figure 5 shows an example of a sample structure when G-PCC geometry and attribute data are stored in a single track. This structure includes a parameter set TLV field (if present) 502, a geometry TLV field 504, and an attribute TLV field (if present) 506.

[0112] When encoded G-PCC geometry data and encoded G-PCC attribute data are stored in separate tracks, each sample in the track contains at least one TLV encapsulation structure carrying data for a single G-PCC component. Figure 6 illustrates the structure of a multi-track ISOBMF G-PCC container according to the latest draft of the MPEG-IPart 18 (ISO / IEC 23090-18) standard (N19286, “Working Draft of ISO / IEC 23090-18 Carriage of Geometry-based Point Cloud Compression Data”, MPEG 130, April 2020). The boxes in Figure 6 map to the corresponding ISOBMFF boxes in ISO / IEC 14496-12.

[0113] Based on the structure in Figure 6, the multi-track G-PCC ISOBMFF container includes the following: (i) a G-PCC track 608 containing a set of geometric parameters, a set of sequence parameters, and a geometric data sample 610 carrying a geometric data TLV unit, the track also including track references to other tracks carrying payloads of G-PCC attribute components 612; and (ii) zero or more G-PCC tracks 614, each track containing an attribute parameter set for the corresponding attribute and an attribute data sample 616 carrying an attribute data TLV unit 618.

[0114] When G-PCC data files are carried across multiple tracks, the track referencing tool in ISO / IEC 14496-12 (“Coding of Audio-Visual Objects, Part 12: ISO BaseMedia File Format”, 2015) is used to link between G-PCC component tracks. A TrackReferenceTypeBoxes are added to the TrackReferenceBox within the TrackBox of a G-PCC track. The TrackReferenceTypeBox contains an array of track_IDs specifying the track referenced by the G-PCC track. To link a G-PCC geometry track to a G-PCC attribute track, the reference_type of the TrackReferenceTypeBox in the G-PCC geometry track identifies the associated attribute track. These track reference types have a 4CC of 'gpca': the referenced track contains an encoded data file of G-PCC attribute data.

[0115] When the 3D spatial region information and associated G-PCC tiles within a 3D spatial region in a G-PCC data file change dynamically, a timing metadata track carries this dynamically changed 3D spatial region information. This timing metadata track provides the temporal association between the 3D spatial region information and the corresponding G-PCC tile for each 3D spatial region.

[0116] The timing metadata track can contain 'cdsc' track references to the G-PCC base track. The G-PCC base track can advantageously contain new track reference types identified by the timing metadata track using 4CC 'gbsr'.

[0117] Non-timing G-PCC data is encapsulated into an ISOBMFF file using G-PCC items. Unlike the sample data described in ISO / IEC 14496-12 (“Coding of Audio-Visual Objects, Part 12: ISO Base Media File Format”, 2015), an item is a box carrying data that does not require timing processing. The carrying of non-timing G-PCC data is supported using single or multiple items with G-PCC tiles. For multiple items with G-PCC tiles, a new item of type 'gpt1', along with property items and item references, are described in N00075 (“Text of ISO / IEC DIS 23090-18 Carriage of Geometry-based Point Cloud Compression Data”, MPEG 132, October 2020) to support partial access.

[0118] Data from one or more G-PCC tiles can be carried in a single GPCC tile entry. Figure 7 A working example of carrying non-timing G-PCC data is shown. For example... Figure 7 As shown in the example, by storing each G-PCC tile in separate tile items 704, 706, and 708, the data of the GPCC item 702 for the three G-PCC tiles is carried in the three tile items. The player identifies the tile item containing the appropriate G-PCC tile by interpreting the associated spatial region item properties 710, 712, and 714.

[0119] like Figure 8As shown in the example, data for two G-PCC tiles (tile #1 and tile #2) are carried in a tile item 804 with associated spatial region item properties 806, 808. To support finer-grained indication of G-PCC tiles, even if a G-PCC tile item 804 contains multiple G-PCC tiles, subsample information 810 can be used, such as... Figure 8 As shown. For example, subsample information 810 may be adapted to indicate the identifier of a tile contained within a G-PCC tile item. Data for another G-PCC tile (tile #3) is carried in a second tile item 812 having an associated spatial region item property 814.

[0120] When the geometry stream of a G-PCC data file includes multiple tiles, each tile or group of tiles is encapsulated in an independent track, called a geometry tile track. The geometry tile track carries one or more TLV cells for the geometry tiles, thus enabling direct access to these tiles. Similarly, the attribute stream of a G-PCC data file containing multiple tiles can be carried in multiple attribute tile tracks. Therefore, a G-PCC tile track for tiles includes a geometry tile track and optionally one or more attribute tile tracks, the geometry tile track containing the geometric information of the tiles carried in the track, and the one or more attribute tile tracks containing the attribute information (such as TLV cells) of the tiles carried in the track.

[0121] G-PCC tile data is carried in separate geometry and attribute tile tracks within the container. For example, each tile may be carried in a geometry tile track dedicated to that tile and one or more attribute tile tracks dedicated to that tile. To support partial access within the ISOBMFF container for the G-PCC encoded stream, tiles corresponding to spatial regions within the point cloud scene are signaled in samples on timing metadata tracks (such as tracks with Dynamic3DSpatialRegionSampleEntry, as described in ISO / IEC 23090-18 or in the GPCCSpatialRegionInfoBox box described in ISO / IEC 23090-18). The timing metadata track is a separate track present within the container. The timing metadata track contains information identifying the spatial regions present in the G-PCC scene. The timing metadata track also contains tile identifiers (IDs) associated with the tiles present in each spatial region. When a user wants to stream content involving a specific or selected spatial region, the player application parses the tile IDs present in the selected spatial region and downloads and / or extracts the tile data present in the corresponding G-PCC tile tracks involving those tile IDs. A tile track sample entry contains a list of tiles present in that track. This allows the player and streaming client to retrieve only the set of tile tracks that carry the information needed to render certain spatial regions or tiles within the point cloud scene.

[0122] Figure 9 The diagram illustrates the architecture of an example formatted container including G-PCC base track 902, G-PCC geometric tile tracks 904, 906, and G-PCC attribute tile tracks 908, 910, 912, 914. The G-PCC base track 902 carries a TLV package structure that contains, for example, only SPS, GPS, APS, and tile inventory information, as described in ISO / IEC 23090-9. Base track 902 carries initialization information to facilitate decoding at the start of each tile. To link the G-PCC base track 902 to the geometric tile tracks 904, 906, track references with a new track reference type are identified using a four-character code (4CC) 'gpbt'. The new track reference type 'gpbt' is used to link or associate the G-PCC base track 'gpcb' 902 with each of the geometric tile tracks 904, 906, such as the G-PCC geometric tile tracks 904, 906 'gpt1' for tiles 0 to N-1. Figure 9 As shown.

[0123] For example, using the track reference tool of ISO / IEC 14496-12, each geometric tile track 904, 906 is linked to G-PCC attribute tile tracks 908, 910, 912, 914 carrying attribute information for the corresponding tile or tile group. These track reference types of 4CC can be referred to as 'gpca', as described in ISO / IEC 23090-18. Figure 9 As shown, each geometric tile track 'gpt1' 904, 906 from tile 0 to tile N-1 is linked or associated with one or more attribute G-PCC tile tracks 908, 910, 912, 914 corresponding to tiles 0 to N-1 and carrying attribute information of the corresponding tile or tile group through the 'gpca' track reference type.

[0124] In another implementation, when the G-PCC data file contains multiple tiles and the tiles are carried in a geometric tile track and an attribute tile track, the G-PCC base track can use a GPCCSampleEntry with a sample entry type of 'gpcb'.

[0125] For example, the G-PCC basic track sample entry contains a GPCCConfigurationBox as described in ISO / IEC 23090-18. Under the 'gpcb' sample entry, all parameter sets as described in ISO / IEC 23090-9 may exist in the setupUnit array or in the data file. Under the 'gpcb' sample entry, there is no GPCCComponentTypeBox.

[0126] In another implementation, when parameter set data and tile inventory information are frequently changed, the parameter set data and tile inventory information can be carried in the base track as part of the G-PCC sample, as described in ISO / IEC 23090-18. The GPCC sample in the base track can carry only SPS, GPS, APS, and tile inventory information TLV_encapsulation cells, and can advantageously not include any geometry or attribute data TLV encapsulation cells.

[0127] The rendering time of a sample is used to identify and decode a G-PCC base track sample (carrying a parameter set and tile inventory data) of a G-PCC tile track sample. The rendering time of the corresponding base track sample is, for example, equal to or less than the rendering time of the tile track sample. When the rendering times of the base track and the tile track sample do not match precisely, the base track sample with a rendering time closer to that of the tile track sample is used to decode such tile track samples or to identify the tile inventory information of the sample. The rendering time of a G-PCC sample (base track or tile track) can be derived from the corresponding track by parsing the 'stts' table in the TimeToSampleBox and the 'ctts' table in the CompositionOffsetBox, as described in ISO / IEC 14496-12.

[0128] In another implementation, when tile inventory information is available in the G-PCC data file and the information does not change over time, the tile inventory information described in ISO / IEC 23090-9 may exist in the setupUnit array of the basic track sample entries for the tiles or in the sample itself.

[0129] G-PCC tile tracks are identified by GPCCTileSampleEntry sample descriptions. Sample entries for G-PCC geometric or attribute tile tracks are of type 'gpt1'. A GPCCTileSampleEntry can be described as follows:

[0130]

[0131]

[0132]

[0133] The above sample entries describe media samples of G-PCC component tile tracks.

[0134] An example of the semantics of fields in GPCCTileSampleEntry is as follows:

[0135] The compressorname in the base class VolumetricVisualSampleEntry indicates the name of the compressor used, with the recommended value being "\013GPCC encoding"; the first byte is the count of the remaining bytes, represented here by \013, which is 11 (decimal) (13 in octal), that is, the number of bytes in the remaining part of the string;

[0136] The config file contains G-PCC tile group configuration records.

[0137] type is an instance of GPCCComponentTypeBox, which indicates the type of G-PCC component carried in the corresponding track; this box does not exist when the data of all components are carried together.

[0138] num_tiles_in_track indicates the number of tiles carried in the corresponding track;

[0139] The `dynamic_tile_id_flag` flag indicates whether the `tile_id` changes within the data file; a value of 0 indicates that the `tile_id` value does not change throughout the data file; a value of 1 indicates that the `tile_id` value changes within the data file; when this flag is set to 1, the tile index is used instead of the tile ID to represent this specific tile; the default value of this flag is 0.

[0140] `tile_id` indicates the unique identifier of a specific tile in the tile inventory; when `dynamic_tile_id_flag` is set to 0, `tile_id` represents a tile ID value existing in the tile inventory; when `dynamic_tile_id_flag` is set to 1, `tile_id` represents the tile index in the tile inventory.

[0141] In another implementation, the G-PCC tile track advantageously indicates all tile identifiers present in the sample of the corresponding tile track. The tile identifiers present in the tile track are signaled in the GPCCTileSampleEntry. The tile identifiers present in a tile track sample should not overlap with tile identifiers present in other tile track samples. The GPCCTileSampleEntry is described below:

[0142]

[0143]

[0144] The above sample entries describe media samples of G-PCC component tile tracks.

[0145] An example of the semantics of fields in GPCCTileSampleEntry is as follows:

[0146] The compressorname in the base class VolumetricVisualSampleEntry indicates the name of the compressor used, with the recommended value being "\013GPCC encoding"; the first byte is the count of the remaining bytes, represented here by \013, which is 11 (decimal) (13 in octal), that is, the number of bytes in the remaining part of the string.

[0147] The config file contains G-PCC tile group configuration records.

[0148] `type` is an instance of `GPCCComponentTypeBox`, which indicates the type of G-PCC component carried in the corresponding track. This box does not exist when the data for all components is carried together.

[0149] The `dynamic_tile_id_flag` indicates whether the number of tiles or tile identifiers present in a tile track sample changes dynamically in the stream. A value of 0 indicates that all samples in the tile track contain the same number of tiles and that the tile identifiers of those tiles have not changed. A value of 1 indicates that the number of tiles present in the tile track sample changes or that the tile identifiers in the tile track sample change in the stream.

[0150] `max_num_tile_ids_in_track` indicates the maximum number of unique G-PCC tile identifiers present in a sample of the corresponding track. When `dynamic_num_tiles_flag` is 0, each sample in the tile track should contain `max_num_tile_ids_in_track` tiles, and the tile identifiers of those tiles remain unchanged throughout the stream. When `dynamic_num_tiles_flag` is 1, each sample in the tile track can contain at most `max_num_tile_ids_in_track` tiles, and the tile identifiers of those tiles can change between samples.

[0151] tile_id indicates the identifier of a specific G-PCC tile corresponding to a sample in the corresponding track.

[0152] Samples in the geometry and attribute tile tracks may have the same sample format as described in ISO / IEC WD 23090-18. The flag value in the codec_specific_parameters field of the SubsampleInformationBox is advantageously set to 1. Each G-PCC sample in the tile track corresponding to a single-point cloud frame contains one or more TLV encapsulation structures belonging to the same rendering time. All TLV encapsulation structures present in the sample advantageously have the same rendering time. Each TLV encapsulation structure contains a single type of G-PCC payload, such as a geometry data unit or an attribute data unit. In an implementation, when parameter set information and tile inventory information are carried in the G-PCC base track, the parameter set information and tile inventory information are not carried in the G-PCC tile track sample.

[0153] G-PCC base tracks use track references to link to geometry tile tracks. The new track reference type uses a four-character code (4CC) 'gpbt' to link G-PCC base tracks with geometry tile tracks.

[0154] Using the 'gpca' track reference type as described in ISO / IEC WD 23090-18, each geometric tile track is linked to other G-PCC tile tracks that carry attribute information of the tiles in the geometric tile track.

[0155] When all G-PCC components of a tile or a group of tiles are carried in a single tile track, a G-PCC sample includes multiple subsamples.

[0156] In another implementation, when all G-PCC components of a tile or group of tiles are carried in a single tile track, the sample entry type 'gptm' is used to indicate that the G-PCC sample contains a representation of two or more temporally interleaved GPCC component data.

[0157] The 'gptm' sample entry can be described as follows:

[0158]

[0159] The presence of a 'gptm' sample entry type indicates the use of a temporally interleaved component encapsulation arrangement. The synthesis time of consecutive samples is equal to the value of the first sample in the sample group within the interleaved component encapsulation arrangement. The syntax can be as follows:

[0160]

[0161] In semantics, component_count_minus1+1 indicates the number of G-PCC component samples that exist in the orbit as consecutive samples.

[0162] In another implementation, the number and layout of tiles in the G-PCC frame are fixed throughout the entire duration of the encoded point cloud sequence to avoid bursts in the number of tracks in the container file.

[0163] The alternative orbitals and their groupings are visualized, such as... Figure 10As shown. Track alternatives can be indicated by alternative track mechanisms as described in ISO / IEC 14496-12, such as the alternate_group field of the TrackHeaderBox. Geometry and attributes are G-PCC components. G-PCC component tile tracks include geometry tile tracks and attribute tile tracks. G-PCC component tile tracks 1004, 1006, 1010, 1012, 1014, 1016, 1018, 1020, 1022, and 1024 with the same alternate_group value are the same G-PCC components in different encoded versions. G-PCC scenes can be encoded in alternative tracks. When a G-PCC scene is encoded in an alternative track, G-PCC component tile tracks that are alternatives to each other have the same alternate_group value in their TrackHeaderBox.

[0164] G-PCC component tile tracks 1004, 1006, 1010, 1012, 1014, 1016, 1018, 1020, 1022, and 1024 may have alternative tracks. In such cases, all G-PCC component tile tracks 1004, 1006, 1010, 1012, 1014, 1016, 1018, 1020, 1022, and 1024 belonging to the alternative group are referenced by the G-PCC basic track 1002 or the corresponding G-PCC geometric tile tracks 1004 and 1006. G-PCC component tile tracks 1004, 1006, 1010, 1012, 1014, 1016, 1018, 1020, 1022, and 1024, which are alternatives to each other, use alternative grouping mechanisms, such as those described in ISO / IEC 14496-12.

[0165] The alternatively selected basic tile tracks 1102 and 1104, along with the corresponding geometric tile tracks 1106, 1108, 1110, and 1112, and attribute tile tracks 1114, 1116, 1118, 1120, 1122, 1124, 1126, and 1128, are grouped in... Figure 11 As shown in the diagram, the volumetric visual scene can be encoded in an alternative track. In another implementation, when the same G-PCC geometric components of different encoded versions are available and each version of the geometric components is signaled in one tile base track and one or more G-PCC tile tracks, the corresponding G-PCC tile base tracks advantageously have the same alternate_group value. In such cases, G-PCC tile base tracks that are alternatives to each other can have the same alternate_group value in their TrackHeaderBox.

[0166] Figure 12The diagram below illustrates a method for decoding tiles of geometry-based point cloud data. The method includes receiving 1202 a formatted container comprising geometry-based point cloud data, which includes a plurality of tiles. Obtaining 1204 a timing metadata track from the formatted container. The timing metadata track includes a plurality of tile identifiers. Each tile identifier corresponds to a corresponding tile 304 among the plurality of tiles. Selecting 1206 at least one selected tile from the plurality of tiles. The at least one selected tile corresponds to at least one tile identifier. Identifying 1208 at least one geometric tile track associated with the at least one tile identifier from the formatted container. Identifying 1210 a base track comprising initialization data of the at least one selected tile from the formatted container using a first track reference type associated with the at least one geometric tile track. Identifying 1212 at least one attribute tile track associated with the at least one selected tile (if present) from the formatted container using a second track reference type associated with the at least one geometric tile track. Decoding 1214 the at least one selected tile into at least one decoded tile using the at least one geometric tile track, the at least one attribute tile track (if present), and the initialization data. Advantageously, decoding can be performed without decoding all geometry-based point cloud data.

[0167] Figure 12 The method can be applied to devices, such as head-mounted displays, smartphones, or other WTRUs, for example. The device identifies region 304 from region 308 of a scene or object 306 of a point cloud 302 to be displayed. Each region 302, 308 may include one or more tiles. Selected decoded tiles are rendered on the display. Alternatively, the selected decoded tiles may be streamed or utilized in other ways, such as in dynamic video streaming applications capable of decoding geometry-based point cloud content. Tiles can be streamed from a server and decoded on a receiving client / UE / device (hereinafter referred to as the "client"). Streaming can be performed using any streaming or transport protocol, such as Dynamic Adaptive Streaming over HTTP (DASH)

[0168] The volumetric visual scene can be encoded in an alternative track. In another implementation, when the same G-PCC attribute components of different encoded versions are available and each version of the attribute components is signaled in a single track or one or more G-PCC tile tracks, the corresponding G-PCC attribute tracks can have the same alternate_group value. G-PCC attribute tracks that are alternatives to each other advantageously have the same alternate_group value in their TrackHeaderBox. G-PCC attribute tile tracks that are alternatives to each other advantageously have the same alternate_group value in their TrackHeaderBox. A diagram illustrating the grouping of alternative attribute tracks and corresponding geometric tracks for multiple tracks is shown in [the figure]. Figure 13 As shown in the figure. G-PCC property tracks 1304 and 1308 may each have alternative tracks such as G-PCC property tracks 1306 and 1310, respectively. All G-PCC property tracks 1304, 1306, 1308, and 1310 belonging to the alternative group are advantageously referenced by the corresponding G-PCC geometric track 1302. G-PCC property tracks 1304 and 1306, which are alternatives to each other, may use alternative grouping mechanisms, such as those described in ISO / IEC 14496-12.

[0169] A diagram showing the grouping of optional attribute tile tracks and corresponding geometric tile tracks is shown in [the diagram]. Figure 14 The diagram shows a grouping of geometric tile tracks and corresponding alternative attribute tile tracks for each of the N tiles labeled 0 to N-1 for the G-PCC basic track. G-PCC attribute tile tracks 1014 and 1022 may have alternative tracks such as G-PCC attribute tile tracks 1016 and 1024, respectively. All G-PCC attribute tile tracks 1014, 1016, 1022, and 1024 belonging to the alternative group are advantageously referenced by the G-PCC basic track 1002 or the corresponding G-PCC geometric tile tracks 1004 and 1006. Alternative grouping mechanisms, such as those described in ISO / IEC 14496-12, can be used for G-PCC attribute tile tracks 1014, 1016, 1022, and 1024 that are alternatives to each other.

[0170] In another implementation, to link static 3D spatial region information to the G-PCC base track, a GPCCSpatialRegionInfoBox can be added to the base track. The base track carries parameter sets such as SPS, GPS, APS, tile inventory information TLV units, and the GPCCSpatialRegionInfoBox.

[0171] In another implementation, when the 3D spatial region information changes dynamically, the G-PCC base track is linked to a timing metadata track 1502 carrying the dynamically changed 3D spatial region information using the track reference tool of ISO / IEC 14496-12. The timing metadata track 1502 may advantageously contain a 'cdsc' track reference to the G-PCC base track. The G-PCC base track may advantageously contain a new track reference type described in the timing metadata track using 4CC 'gb3d'.

[0172] The overall architecture of the G-PCC basic track, G-PCC tile track, 3D spatial region timing metadata track 1502, and track references between the basic track 902 and the 3D spatial region timing metadata track 1502 is as follows: Figure 15 As shown in the image.

[0173] The GPCCComponentTypeBox described in ISO / IEC 23090-18 represents the type of G-PCC component, such as geometry or attribute. In another embodiment, in order to represent the type of attribute component present in the data file and to distinguish the various attribute components present in the G-PCC data file, the GPCCComponentInfoBox is described as replacing the GPCCComponentTypeBox as described in ISO / IEC 23090-18.

[0174] The GPCCComponentInfoBox uses signals to transmit information about G-PCC components. When this box is present in a sample entry for a track carrying G-PCC component data, it indicates the type of G-PCC component carried by the corresponding track. When the corresponding track carries G-PCC attribute components, this box also provides the attribute type and index. The attr_index variable in the GPCCComponentInfoBox distinguishes various attribute components with the same attr_type value as specified in Table 8 of ISO / IEC 23090-9. This box is advantageously not present in a sample entry when the G-PCC data file is stored in a single track.

[0175] An example syntax can be as follows:

[0176]

[0177]

[0178] The semantics can be as follows: attr_type identifies the type of attribute component as specified in Table 8 of ISO / IEC 23090-9, and attr_index identifies the order of attributes in the SPS.

[0179] In another implementation, when the corresponding track carries a G-PCC attribute component, the GPCCComponentInfoBox also provides the attribute name, index, and optional attribute type or attribute object identifier.

[0180] The syntax of the GPCCComponentInfoBox box is shown below.

[0181]

[0182] The semantics of the GPCCComponentInfoBox can be as follows.

[0183] attr_index identifies the order of attributes in SPS.

[0184] The `attr_type_present` parameter indicates that attribute type information exists in the `GPCCComponentInfoBox`. A value of 1 indicates that attribute type information is sent via a signal in this box. A value of 0 indicates that attribute type information is not sent via a signal.

[0185] The known_attribute_label_flag indicates whether the attribute is identified by the value of attr_type or by the international object identifier attribute_label_oid.

[0186] attr_type identifies the type of attribute component as specified in Table 8 of ISO / IEC 23090-9.

[0187] The attribute_label_oid identifies international object identifiers as specified in Recommendation ITU-T X.660 ISO / IEC 9834-1. The syntax of the object identifier is described in subclause 9.6.5.1 of ISO / IEC 23090-9.

[0188] attr_name specifies a human-readable name for the type of G-PCC attribute component.

[0189] In another implementation, when the G-PCC data file contains 3D objects, 3DObjectInfoStruct provides the bounding box information of the 3D objects.

[0190] 3DObjectInfoStruct provides bounding box information for 3D objects, including the X, Y, and Z coordinates of the anchor points and the dimensions of the bounding box relative to the anchor points along the X, Y, and Z axes.

[0191] An example syntax can be as follows:

[0192]

[0193]

[0194] The semantics can be as follows:

[0195] anchor_included indicates whether the X, Y, Z coordinates of the origin of a 3D object are included in the structure;

[0196] A dimension_included value of 1 indicates that the dimensions of a 3D object are signaled within the structure. A dimension_included value of 0 indicates that the dimensions of a 3D object are not signaled within the structure.

[0197] 3d_object_id indicates the identifier of a 3D object;

[0198] anchor_x, anchor_y, and anchor_z indicate the x, y, and z offsets of the anchor point of a 3D object in Cartesian coordinates, respectively. When an anchor point does not exist in the structure, it can be inferred that the anchor point is equal to (0, 0, 0).

[0199] object_dx, object_dy, and object_dz indicate the dimensions of a 3D object in Cartesian coordinates relative to its anchor point along the x, y, and z axes, respectively, and also indicate the width, height, and depth of the 3D object in Cartesian coordinates.

[0200] In another implementation, when the 3D objects present in the G-PCC are static, the GPCC3DObjectsInfoBox present in the G-PCC base track provides the bounding box information of each 3D object and the associated G-PCC tile.

[0201] The GPCC3DObjectsInfoBox provides information about the 3D objects present in the G-PCC data file, including bounding box information such as the X, Y, and Z coordinates of the anchor point and the dimensions of the 3D object's bounding box relative to the anchor point along the X, Y, and Z axes. This box also provides a mapping to a tile set for each object and allows you to enable or disable the object.

[0202] The GPCC3DObjectsInfoBox can optionally exist in a sample entry of the G-PCC base track. When the GPCC3DObjectsInfoBox exists in a sample entry of the G-PCC base track, it indicates information about static 3D objects present in G-PCC.

[0203] An example syntax can be as follows:

[0204]

[0205]

[0206] The semantics can be as follows:

[0207] num_objects indicates the number of 3D objects present in the point cloud;

[0208] 3DObjectInfoStruct provides spatial information about a 3D object indicated by its anchor point, as well as the dimensions of the 3D object relative to the anchor point along the X, Y, and Z axes.

[0209] num_tiles[i] indicates the number of G-PCC tiles associated with the i-th 3D object;

[0210] `object_enabled` equal to 1 indicates that the 3D object is active in the scene. `object_enabled` equal to 0 indicates that the 3D object does not exist in the first frame of the G-PCC data file;

[0211] tile_id[j] identifies the j-th G-PCC tile associated with the i-th 3D object.

[0212] In another implementation, when the 3D objects containing bounding box information in the G-PCC data file and the G-PCC tiles associated with those 3D objects change dynamically, a timing metadata track carries the dynamically changed 3D object information. This 3D object information timing metadata track provides the temporal association between the 3D object information and the corresponding G-PCC tile for each 3D object.

[0213] Timing metadata track 1502 may advantageously contain a 'cdsc' track reference to the G-PCC base track. The G-PCC base track may advantageously contain a new track reference type described in timing metadata track 1502 using 4CC 'gb3d'.

[0214] Synchronization samples in the timed metadata track can advantageously carry the dimensions and associated tile mapping information of all 3D objects, regardless of whether 3D objects are enabled. For synchronization samples, the values ​​of the `dynamic_dimension_flag` and `dynamic_tile_mapping_flag` flags for each 3D object are set to 1. When the object is active in this synchronization sample, the `object_enabled` flag is set to 1; otherwise, the `object_enabled` flag is set to 0.

[0215] The asynchronous samples in this timing metadata track can advantageously carry only updated 3D object information that references the most recently available 3D object information in the presynchronous samples.

[0216] If the base track has an associated timing metadata track of sample entry type 'gpdo', the position of the associated 3D object in the point cloud data is considered dynamic.

[0217] Sample Entries

[0218]

[0219]

[0220] GPCC3DObjectsInfoBox indicates the initial position information of a 3D object;

[0221] num_objects indicates the number of 3D objects signaled in the sample entry;

[0222] 3DObjectInfoStruct provides initial information for the i-th 3D object, including the anchor point in the sample entry and its dimensions in Cartesian coordinates relative to the anchor point along the X, Y, and Z axes.

[0223] num_tiles[i] indicates the number of G-PCC tiles associated with the i-th 3D object in the sample entry;

[0224] An object_enabled value of 1 indicates that the 3D object is active; an object_enabled value of 0 indicates that the 3D object is inactive.

[0225] tile_id[j] identifies the j-th G-PCC tile associated with the i-th 3D object in the sample entry;

[0226] A dynamic_dimension_flag value of 0 indicates that the dimensions of the 3D object remain unchanged across all samples referencing this sample entry; a dynamic_dimension_flag value of 1 indicates that the dimensions of the 3D object are indicated in each sample.

[0227] A value of 0 for dynamic_tile_mapping_flag indicates that the identifier of the tile associated with the 3D object remains unchanged across all samples referencing this sample entry;

[0228] A dynamic_tile_mapping_flag value of 1 specifies that the identifier of the tile associated with the 3D object exists in each sample.

[0229] The sample syntax for this sample entry type 'gpdo' can be as follows:

[0230]

[0231]

[0232] num_objects indicates the number of 3D objects that reference the most recently synchronized sample that have been updated in the sample;

[0233] 3d_object_id indicates the updated 3D object identifier;

[0234] An object_enabled value of 0 indicates that the updated 3D object does not exist in the sample;

[0235] An object_enabled value of 1 specifies that the updated 3D object is active in the sample;

[0236] A dynamic_dimension_flag value of 0 indicates that the updated 3D object dimension references the most recent synchronization sample and has not yet changed; a dynamic_dimension_flag value of 1 indicates that the updated 3D object dimension references the most recent synchronization sample and has changed.

[0237] A dynamic_tile_mapping_flag value of 0 indicates that the tile reference associated with the updated 3D object has not changed in the most recent synchronization sample; a dynamic_tile_mapping_flag value of 1 indicates that the tile reference associated with the updated 3D object has changed in the most recent synchronization sample.

[0238] 3DObjectInfoStruct provides updated 3D object spatial information;

[0239] num_tile[i] indicates the number of G-PCC tiles associated with the i-th 3D object when this sample is applied;

[0240] tile_id[j] identifies the G-PCC tile associated with the i-th 3D object when this sample is applied.

[0241] In another implementation, the synchronization samples in the 3D spatial region information timing metadata track advantageously carry the dimensions and associated tile mapping information of all 3D spatial regions. For the synchronization samples, the values ​​of the dynamic_dimension_flag and dynamic_tile_id_flag flags for each 3D spatial region are set to 1.

[0242] In another implementation, asynchronous samples in the timing metadata track advantageously carry only updated 3D spatial region information that references the 3D spatial region information most recently available in the presynchronous samples.

[0243] In another implementation, the system advantageously sets samples in the 3D spatial region information timing metadata track as either synchronous or asynchronous samples. Advantageously, for a specific number of samples (keyframe distance) or for a specific time interval (keyframe time), there exists a synchronous sample. The keyframe distance or keyframe time is advantageously specified by the system.

[0244] In another implementation, for synchronized samples, the value of the dynamic_dimension_flag flag is set to 1, and the dynamic_tile_id_flag flag is set to 1 when tile inventory information exists in the G-PCC data file, and the canceled_region_flag flag is set to 0 for each 3D spatial region.

[0245] In another implementation, asynchronous samples may optionally be signaled only with respect to 3D spatial regions that have changed relative to the most recent previous synchronous sample, including updated dimensions or associated 3D tiles and any added or removed 3D spatial regions. The `cancelled_region_flag` flag is set to 1 when a 3D spatial region is removed in the previous synchronous sample. The `dynamic_dimension_flag` flag is set to 1 when the dimension of a 3D spatial region in the current sample is updated in the previous synchronous sample. The `dynamic_tile_id_flag` flag is set to 1 when the associated tile of a 3D spatial region in the current sample is updated in the previous synchronous sample.

[0246] An example syntax can be as follows:

[0247]

[0248]

[0249] Examples of semantics are:

[0250] `num_regions` indicates the number of updated 3D spatial regions referenced in the previous sample and signaled in the sample. The dimensions and / or associated 3D tiles referenced in the previous sample that update the 3D spatial regions are considered updated regions. 3D spatial regions referenced in the previous sample that are removed from this sample are also considered updated regions.

[0251] The `cancelled_region_flag` indicates whether a 3D region referenced in a previous synchronized sample is eliminated or updated in the current sample. A value of 0 indicates that the 3D region dimension and / or associated 3D tiles referenced in a previous synchronized sample are updated. A value of 1 indicates that the 3D region referenced in a previous synchronized sample is eliminated in this sample.

[0252] The dynamic_dimension_flag indicates whether the referenced pre-synchronous sample updates the dimension of this 3D region.

[0253] The dynamic_tile_id_flag indicates whether the referenced previous synchronous sample updates the associated 3D tiles for this 3D region.

[0254] 3DSpatialRegionStruct provides 3D spatial region information of the G-PCC data when this sample is applied.

[0255] num_tile indicates the number of G-PCC tiles associated with a 3D spatial region when this sample is applied.

[0256] tile_id identifies a specific G-PCC tile associated with a 3D spatial region.

[0257] 3d_region_id identifies the 3D spatial region referenced in the previous synchronous sample elimination.

[0258] The GPCCSpatialRegionInfoProperty descriptive property, described in 23090-18, associated with one or more G-PCC tile items, describes spatial region information, including identifiers, anchor points, and the dimensions of the 3D tile relative to the anchor point along the X, Y, and Z axes in Cartesian coordinates. When a client wants partial access to non-timing information, the client resolves all GPCCSpatialRegionInfoProperty property items and locates the G-PCC tile item of interest based on the user viewport and the 3D tile inventory information present in the GPCCSpatialRegionInfoProperty property items. This process is verbose on the client side.

[0259] The use of the descriptive property GPCCSpatialRegionsInfoProperty solves the above problems and provides better support for partial access.

[0260] In another implementation, each G-PCC entry of type 'gpeb' is advantageously associated with the GPCCSpatialRegionsInfoProperty property. GPCCSpatialRegionsInfoProperty advantageously indicates the 3D region identifier, offset, and size of the bounding box information for each 3D region. In another implementation, each G-PCC entry of type 'gpel' is advantageously associated with the GPCCSpatialRegionsInfoProperty property entry when 3D piece inventory information is available in the G-PCC data file. When 3D piece inventory information is not available in the G-PCC data file, the GPCCSpatialRegionsInfoProperty property entry does not exist.

[0261] In another implementation, when 3D tile inventory information is available in the G-PCC data file and the subsample item property is linked to this G-PCC item, the G-PCC item carrying the type 'gpci' of the G-PCC geometric components is advantageously associated with the GPCCSpatialRegionsInfoProperty property item. When 3D tile inventory information is not available in the G-PCC data file or the subsample item property is not linked to this G-PCC item, the GPCCSpatialRegionsInfoProperty property item does not exist.

[0262] Figure 16 This is a diagram showing partial access to a non-timing G-PCC data item 1602 with N G-PCC tiles, having spatial region item property 1604, where N is an integer. Figure 16 This is an example of carrying non-timed G-PCC data consisting of N G-PCC tiles arranged in multiple G-PCC tile items 1606 and 1608 by storing G-PCC tiles in separate items with associated tile information item properties 1610 and 1612.

[0263] Figure 17 This is a diagram showing partial access to non-timed G-PCC data of G-PCC item 1702 with type 'gpe1'. Figure 17 This is an example of carrying non-timed G-PCC data with a G-PCC item of type 'gpe1'. When 3D tile inventory information is available in the G-PCC data file, the G-PCC item is associated with G-PCC spatial region item property 1704.

[0264] Figure 18 This is a diagram showing partial access to a non-timed G-PCC of G-PCC item 1802 with type 'gpci'. Figure 18An example of a non-timed G-PCC carried by a G-PCC item 1802 of type 'gpci' is shown. When 3D tile inventory information 1806 is available in the G-PCC data file and subsample item property 1810 is associated with a G-PCC item of type 'gpci', the G-PCC item 1802 carrying geometric data is associated with G-PCC spatial region item property 1804 and includes associated component information item property 1808. Each hidden attribute 1812, 1814 is shown as having associated configuration item properties 1816, 1818, associated component information item properties 1820, 1822, and subsample item properties 1824, 1826.

[0265] In another implementation, the GPCCTileInfoProperty property describes the tile identifier information for each 3D tile present in a G-PCC tile item. Each G-PCC tile item of type 'gpt1' is advantageously associated with a GPCCTileInfoProperty property item. The GPCCTileInfoProperty property item advantageously indicates the 3D tile identifier information for each 3D tile present in a G-PCC tile item of type 'gpt1'. The G-PCC player uses the G-PCC spatial region item property associated with the G-PCC item to identify the required tile identifier based on the viewport region of interest. The tile item containing the specific G-PCC tile identifier is interpreted using the associated G-PCC tile information item property.

[0266] The properties of GPCCSpatialRegionsInfoProperty and GPCCTileInfoProperty enable partial access to non-timed G-PCC.

[0267] The properties of the G-PCC spatial region term can be described as follows.

[0268]

[0269] The GPCCSpatialRegionsInfoProperty descriptive property describes spatial region information, including 3D region identifiers, anchor points, and the dimensions of the 3D spatial regions in Cartesian coordinates relative to each 3D spatial region's anchor point along the X, Y, and Z axes. The GPCCSpatialRegionsInfoProperty property also describes the 3D tile identifier associated with each 3D spatial region.

[0270] Here is an example of the syntax:

[0271]

[0272]

[0273] Examples of semantics are as follows:

[0274] num_regions indicates the number of 3D spatial regions in the G-PCC data file.

[0275] 3d_region_id indicates the identifier of a spatial region.

[0276] anchor_x, anchor_y, and anchor_z indicate the x, y, and z offsets of the anchor point in the 3D spatial region in Cartesian coordinates, respectively.

[0277] region_dx, region_dy, and region_dz indicate the dimensions of a 3D spatial region relative to its anchor point along the x, y, and z axes in Cartesian coordinates, respectively, and also indicate the width, height, and depth of the 3D spatial region in Cartesian coordinates.

[0278] num_tile indicates the number of 3D tiles associated with a 3D spatial region.

[0279] tile_id indicates the tile identifier of a 3D tile associated with a 3D spatial region.

[0280] The properties of G-PCC tile information items can be described as follows.

[0281]

[0282] The GPCCTileInfoProperty descriptive property describes the tile identifier of the 3D tiles present in the G-PCC tile item. The GPCCTileInfoProperty property may optionally include the anchor point and the dimensions of the 3D tile in Cartesian coordinates relative to the anchor points of all 3D tiles present in the G-PCC tile item along the X, Y, and Z axes.

[0283] Here is an example of the syntax:

[0284]

[0285]

[0286] Examples of semantics are as follows:

[0287] num_tile indicates the number of 3D tiles present in a G-PCC tile item.

[0288] tile_id indicates the tile identifier of the 3D tile that exists in the G-PCC tile item.

[0289] The `tile_inventory_info_flag` indicates whether tile inventory information is available in the `GPCCTileInfoProperty` property. A value of 0 indicates that tile inventory information is not available in the `GPCCTileInfoProperty` property. A value of 1 indicates that tile inventory information is available in the `GPCCTileInfoProperty` property.

[0290] 3d_region_id indicates the identifier of a 3D tile.

[0291] anchor_x, anchor_y, and anchor_z indicate the x, y, and z offsets of the anchor point of a 3D tile in Cartesian coordinates, respectively.

[0292] region_dx, region_dy, and region_dz indicate the dimensions of the 3D tile relative to the anchor point along the x, y, and z axes in Cartesian coordinates, respectively, and also indicate the width, height, and depth of the 3D tile in Cartesian coordinates.

[0293] In another implementation, temporal scalability in G-PCC data files can be supported by partitioning G-PCC frames based on temporal layers. The system can select the maximum number of temporal layers present in the G-PCC data file to support temporal scalability. The system can distribute G-PCC frames in the data file across multiple temporal layers. For example, a G-PCC data file containing 600 frames can be distributed across three temporal layers, with the first frame assigned to temporal layer 0, the second to temporal layer 1, the third to temporal layer 3, the fourth to temporal layer 0, and so on. If the mapping between G-PCC frames and temporal layer identifier information is not signaled in the G-PCC data file, the system can identify the distribution logic of G-PCC frames to specific temporal layers. G-PCC streaming applications can stream only frames with specific temporal layer IDs, frames belonging to multiple temporal layers, or frames from all temporal layers, and then decode and render those frames to a point cloud renderer. Frames from individual temporal layers among multiple identified temporal layers can be decoded and rendered without decoding and rendering any other temporal layers.

[0294] In another implementation, the GPCCScalabilityInfoBox indicates scalability information present in the data file. When this box is present in a sample entry representing a track in the main G-PCC data, it indicates whether scalability is supported. If scalability is supported, this box provides the maximum number of time layers present in the G-PCC data file.

[0295] In another implementation, the G-PCC tile base track or master track transmits the maximum number of time layers present in the G-PCC data file via signal transmission.

[0296] Here is an example of the syntax for GPCCScalabilityInfoBox:

[0297]

[0298] The semantics of GPCCScalabilityInfoBox are shown below:

[0299] The `temporal_scalability_flag` indicates whether G-PCC frames in a data file are divided into time layers. A value of 0 indicates that time layer information is unavailable or that all time layer frames are signaled within a single time layer. A value of 1 indicates that the frames are divided into multiple time layers.

[0300] max_num_temporal_layers indicates the maximum number of temporal layers into which a G-PCC data file frame is divided.

[0301] temporal_layer_id indicates the temporal layer identifier of the existing sample.

[0302] In another implementation, the G-PCC tile track can signal the time layer identifier of the G-PCC samples present in this track. The time layer identifier information present in the tile track is signaled in the GPCCTileSampleEntry. The G-PCC tile track can signal one or more tiles belonging to one or more time layers or all time layers.

[0303] The sample entry describes a media sample of a G-PCC component tile track. The GPCCTileSampleEntry is described as follows:

[0304]

[0305] Here is an example of the syntax for GPCCTileSampleEntry:

[0306]

[0307] The semantics of the fields in GPCCTileSampleEntry can be described as follows:

[0308] The compressorname in the base class VolumetricVisualSampleEntry indicates the name of the compressor used, with the recommended value being "\013GPCC encoding"; the first byte is the count of the remaining bytes, represented here by \013, which is 11 (decimal) (13 in octal), that is, the number of bytes in the remaining part of the string.

[0309] The config file contains G-PCC tile group configuration records.

[0310] `type` is an instance of `GPCCComponentTypeBox`, which indicates the type of G-PCC component carried in the corresponding track. This box does not exist when the data for all components is carried together.

[0311] The `dynamic_tile_id_flag` indicates whether the number of tiles or tile identifiers present in a tile track sample changes dynamically in the stream. A value of 0 indicates that all samples in the tile track contain the same number of tiles and that the tile identifiers of those tiles have not changed. A value of 1 indicates that the number of tiles present in the tile track sample changes or that the tile identifiers in the tile track sample change in the stream.

[0312] The `temporal_scalability_flag` indicates whether G-PCC frames in a data file are segmented into temporal layers. A value of 0 indicates that temporal layer information is unavailable or that all temporal layer frames are transmitted via signaling in this tile track. A value of 1 indicates that temporal layer identifier information is present.

[0313] `max_num_tile_ids_in_track` indicates the maximum number of unique G-PCC tile identifiers present in a sample of the corresponding track. When `dynamic_num_tiles_flag` is 0, each sample in the tile track contains `max_num_tile_ids_in_track` tiles, and the tile identifiers of these tiles remain unchanged throughout the stream. When `dynamic_num_tiles_flag` is 1, each sample in the tile track contains at most `max_num_tile_ids_in_track` tiles, and the tile identifiers of those tiles can change between samples.

[0314] tile_id indicates the identifier of a specific G-PCC tile corresponding to a sample in the corresponding track.

[0315] num_temporal_layers indicates the number of temporal layers present in the samples of the corresponding orbit.

[0316] temporal_layer_id indicates the temporal layer identifier of the sample sent by signal in the corresponding orbit.

[0317] Sample entries for G-PCC tile base tracks or G-PCC geometry tracks may contain GPCCScalabilityInfoBox boxes. Sample entries for G-PCC tile base tracks are as follows:

[0318]

[0319]

[0320] The sample entries for G-PCC geometric orbitals are as follows:

[0321]

[0322] In another implementation, a G-PCC track of type 'gpe1' or 'gpeg' can be signaled to transmit the time layer identifier of the G-PCC samples present in that track. A GPCCScalabilityInfoBox can exist in the sample entry to signal the time layer identifier information present in that track. A G-PCC track of type 'gpe1' or 'gpeg' can be signaled to transmit all time layers present in the data file.

[0323] The following shows sample entries for G-PCC orbitals for a single orbital case.

[0324]

[0325] scalabilityInfo indicates the temporal scalability layer identifier information present in the samples of this track.

[0326] The presentation times of samples belonging to the same point cloud component in different time-level orbits should be different. For example, the presentation times of geometric component samples in time-level 0 and time-level 1 orbits should be different.

[0327] As described in ISO / IEC 23090-18, the GPCCDecoderConfigurationRecord can be expanded to indicate the number of time levels present in the file. The syntax and semantics of the expanded decoder configuration record are shown below. Decoder configuration information (such as SPS, GPS, APS, and tile inventory information) for all time level tracks can advantageously be the same. Advantageously, only the number of time levels and time level identifiers present in those tracks can be changed.

[0328] The example syntax is as follows:

[0329]

[0330] The following are examples of semantics:

[0331] `num_temporal_layers` indicates the maximum number of temporal layers present in the track. This field has a value of 1 when temporal layer information is unavailable or when all frames are signaled within a single temporal layer.

[0332] temporal_layer_id indicates the temporal layer identifier.

[0333] In another implementation, G-PCC component samples are grouped based on their time level. Time-level sample grouping ('tele') provides codec-independent sample grouping that can be used to group G-PCC samples in orbits (and potential orbit segments) according to time level, where samples at one time level do not have encoding dependencies on samples at other time levels.

[0334] In another implementation, the time-level sample group 'tele' specified in ISO / IEC 14496-12 is used to indicate the TemporalId value. When the 'tele' sample group exists in a G-PCC track carrying geometric and / or attribute data, the sample with the time-level TemporalId is mapped to the sample group description index TemporalId+1. The sample group description box is signaled in the decoder configuration record for all layers' sample group descriptions.

[0335] In another implementation, when tile inventory information is available in the G-PCC data file and is static or changes over time, the tile inventory information is signaled using a sample group with grouping_type 'gtii'. The sample group with grouping type 'gtii' is used to group G-PCC samples that use the same tile inventory information in the G-PCC geometry. The tile inventory information may exist in the sample group description entry or within the sample itself.

[0336] In another implementation, when a G-PCC data file is carried using a G-PCC track with track type 'gpc1' and tile inventory information is available in the data file, the geometric track contains sample groups of tile inventory information with grouping type 'gtii', and the tile inventory information is present in the sample group description entry. The attribute track does not contain sample groups with grouping type 'gtii'.

[0337] In another implementation, under the 'gpcg' sample entry, when tile inventory information is available in the data file, the geometry track contains a sample group of tile inventory information with grouping type 'gtii', and the tile inventory information may exist in the sample group description entry or in the sample of the G-PCC geometry track.

[0338] In another implementation, under the 'gpe1' sample entry, when tile inventory information is available in the data file, the G-PCC track contains a sample group of tile inventory information with grouping type 'gtii', and the tile inventory information exists in the sample group description entry.

[0339] In another implementation, under the 'gpeg' sample entry, when tile inventory information is available in the data file, the G-PCC track contains a sample group of tile inventory information with grouping type 'gtii', and the tile inventory information may exist in the sample group description entry or in the sample of the G-PCC track.

[0340] In another implementation, when using tile tracks to carry G-PCC data files, a basic tile track with track type 'gpcb' or 'gpeb' can contain sample groups with grouping type 'gtii', and tile inventory information is available in the basic tile track samples. Tile inventory information is not present in the 'gtii' sample group description entries. Geometric and attribute tile tracks with track type 'gpt1' do not contain sample groups with grouping type 'gtii'.

[0341] In another implementation, when using a tile track with track type 'gpt1' to carry a G-PCC data file, the geometric tile track may contain a 'gtii' sample group to signal the tile inventory information of the tiles present in the sample of this track.

[0342] Sample group of tile inventory information entries:

[0343] Group type: 'gtii'

[0344] Container: Sample group descriptor box ('sgpd')

[0345] Required fields: None

[0346] Quantity: Zero or more

[0347] The Tile Inventory Sample Group entry describes the tile inventory information for all samples using the same tile inventory information.

[0348] Here is an example of the syntax:

[0349] abstract class SampleGroupDescriptionEntry(unsigned int(32)grouping_type)

[0350] {

[0351] }

[0352] abstract class VolumetricSampleGroupEntry(unsigned int(32)grouping_type)

[0353] extends SampleGroupDescriptionEntry(grouping_type)

[0354] {

[0355] }

[0356] class TileInventoryInfoEntry()extends VolumetricSampleGroupEntry('gtii')

[0357] {

[0358] tlv_encapsulation tile_inventory_info; / / as defined in ISO / IEC 23090-9

[0359] }

[0360] Examples of semantics are as follows:

[0361] tile_inventory_info contains tile inventory information in a TLV encapsulation structure with tlv_type equal to 5, as described in ISO / IEC 23090-9.

[0362] For example, a G-PCC data file with multiple tile tracks has one geometric component and two attribute components. In this example, the G-PCC data file includes 50 tiles divided into ten tile sets. The first tile set may include tiles 1 to 5, the second tile set may include tiles 6 to 9, the third tile set may include tiles 10 to 20, and so on. The number of tiles in each set may vary between sets or may be the same. Each component of the tile set is carried in a separate G-PCC tile track within an ISOBMFF container file.

[0363] When a client wants to play back G-PCC content with a specific 3D region of interest, the client identifies the 3D region present in the G-PCC data file from the GPCCSpatialRegionInfoBox located in the G-PCC base track. The client selects the tile associated with the 3D region of interest. The client identifies the desired tile track for the selected tile based on the tile information present in the GPCCTileSampleEntry for each tile track. The GPCCTileSampleEntry specifies the list of tiles present in this tile track.

[0364] When G-PCC tile media content is present, the client identifies the tiles of interest in the point cloud data file based on the client's current viewport. The client parses the GPCCSpatialRegionInfoBoxes present in the G-PCC base track and locates the corresponding 3D regions within the current viewport. The GPCCSpatialRegionInfoBoxes are used to identify the tiles within those selected 3D regions. The client then identifies the required tile track for the selected tiles based on the tile information present in the GPCCTileSampleEntry for each tile track.

[0365] When 3D region information or tiles existing within a 3D region dynamically change in G-PCC content and the client wants to replay G-PCC content with a 3D region of interest, the client identifies the dynamically changing 3D region in the G-PCC data file from the Dynamic3DSpatialRegionSampleEntry in the timing metadata track with sample entry type 'gpdr'. The client uses Dynamic3DSpatialRegionSample type samples existing in the timing metadata track to identify tiles existing within the 3D region of interest. The client identifies the desired tile track for the selected tile based on the tile information existing in the GPCCTileSampleEntry for each tile track.

[0366] The client can also access tile track data based on the user viewport. When the 3D partitions present in the user viewport are dynamic, the client identifies dynamically changing 3D regions existing in the G-PCC data file from the Dynamic3DSpatialRegionSampleEntry, which exists in the timing metadata track with sample entry type 'gpdr'. The client uses Dynamic3DSpatialRegionSample type samples existing in the timing metadata track to identify 3D regions present in the viewport. The client uses the information available in the Dynamic3DSpatialRegionSample samples to identify tiles present in those selected 3D regions. The client identifies the desired tile track for the selected tiles based on the tile information present in the GPCCTileSampleEntry box for each tile track.

[0367] The following is an example client method for playing back G-PCC tile content.

[0368] 1. Identify viewports or user viewport information that the client is interested in;

[0369] 2. Use GPCCSpatialRegionInfoBox to identify 3D regions associated with the viewport of interest to the client or the user's viewport;

[0370] 3. When the 3D region information changes dynamically, identify the 3D spatial region information existing in the metadata track sample of the Dynamic3DSpatialRegionSample type.

[0371] 4. Based on available 3D region information, identify 3D regions associated with the viewport region of interest (e.g., regions that change dynamically in 3D);

[0372] 5. Identify tiles associated with 3D regions of interest from timed metadata track samples of 3D spatial region information;

[0373] 6. Use the information in the GPCCTileSampleEntry box present in each tile track to identify the tile tracks associated with those selected tiles;

[0374] 7. Based on the user's current viewport or viewport of interest, the selected tile track stream is extracted, decoded, and presented to the user from the G-PCC data file or bitstream.

[0375] When a client wants to replay G-PCC content containing 3D objects of interest, the client identifies the 3D objects present in the G-PCC data file from the GPCC3DObjectsInfoBox located in the G-PCC base track. The client selects the tiles to download for the 3D objects of interest. The client identifies the desired tile track for the selected tiles based on the tile information present in the GPCCTileSampleEntry for each tile track. The GPCCTileSampleEntry specifies the list of tiles present in this tile track.

[0376] When the bounding box information of a 3D object or the tiles present in a 3D object dynamically changes, and the client wants to replay G-PCC content containing the 3D object of interest, the client identifies the dynamically changed 3D object in the G-PCC data file using the sample entry type 'gpdo' from the Dynamic3DObjectsInfoSampleEntry in the timing metadata track. The client uses Dynamic3DObjectsInfoSample type samples present in the 3D object's timing metadata track to identify the tiles present in the 3D object of interest. The client identifies the desired tile track for the selected tile based on the tile information present in the GPCCTileSampleEntry for each tile track.

[0377] The following is an example client method for playing back G-PCC tile content.

[0378] 1. Identify 3D objects and viewport information that the user is interested in;

[0379] 2. Use GPCC3DObjectsInfoBox to identify tiles associated with the 3D object of interest;

[0380] 3. When the spatial information of a 3D object changes dynamically, the information present in the timed metadata item sample of the 3D object information of type Dynamic3DObjectsInfoSample is used to identify the tiles associated with the 3D object of interest;

[0381] 4. Use the information in the GPCCTileSampleEntry box present in each tile track to identify the tile tracks associated with those selected tiles;

[0382] 5. For viewports of interest, use GPCCSpatialRegionInfoBox to identify the 3D region associated with the viewport of interest or the user viewport;

[0383] 6. When the 3D region information changes dynamically, identify the 3D spatial region information existing in the timing metadata track sample of the Dynamic3DSpatialRegionSample type;

[0384] 7. Based on available 3D region information, identify 3D regions associated with the viewport region of interest (e.g., regions that change dynamically in 3D);

[0385] 8. Identify tiles associated with the 3D region of interest from timed metadata track samples of 3D spatial region information;

[0386] 9. Use the information in the GPCCTileSampleEntry box present in each tile track to identify the tile track associated with the selected tile;

[0387] 10. Based on the 3D object of the user of interest and the current viewport or viewport of interest, the tile track stream selected from steps 4 and 9 is extracted, decoded, and presented to the user from the G-PCC data file or bitstream.

[0388] An alternative method includes receiving a timing metadata track that identifies point cloud tiles corresponding to one or more spatial regions within a point cloud scene. A decoding device determines one or more point cloud tiles to be used for rendering an image. One or more geometric tile tracks corresponding to the determined one or more point cloud tiles are retrieved via a communication network. Each geometric tile track includes point cloud geometry data for the corresponding tile. The retrieved geometric tile tracks are processed. The timing metadata track may be a track having a Dynamic3DSpatialRegionSampleEntry data field or a GPCCSpatialRegionInfoBox box data field. Determining the tiles to be used for rendering the image may include obtaining the viewer's device's perspective relative to the point cloud data. The decoding device may be a player device or a streaming client, and determining one or more point clouds may include identifying a set of tile tracks carrying information required for rendering certain spatial regions or tiles within the point cloud scene. The base track may carry initialization data, which includes at least one of (i) a type-length value encapsulation structure containing only SPS, GPS, and APS types, and (ii) tile inventory information as described in ISO / IEC 23090-9. Basic tracks can be linked to geometry tile tracks using four-character codes (4CC) based on track reference type. Each geometry tile track can be linked to one or more attribute tile tracks. Geometry tile tracks can be associated with attribute tile tracks carrying attribute information for the corresponding tile or tile group using the track reference tool of ISO / IEC 14496-12. Multiple tiles and their corresponding tile data can be carried across multiple geometry tile tracks and multiple attribute tile tracks. Basic tracks can use the GPCCSampleEntry data field with a sample entry type of 'gpcb'. GPCC component tile tracks with the same alternate_group value are the same G-PCC component with different encoding versions, and alternative G-PCC component tile tracks can have the same alternate_group value, for example, in their TrackHeaderBox. G-PCC component tile tracks belonging to an alternative group can be referenced by a G-PCC basic track or the corresponding G-PCC geometry tile track. G-PCC attribute tracks that are alternatives to each other can have the same alternate_group value. G-PCC attribute tile tracks that are alternatives to each other can have the same alternate_group value.

[0389] In one implementation, a method for generating a point cloud data stream includes generating a basic track sample entry containing a GPCCConfigurationBox.

[0390] In one implementation, a method for generating a point cloud data stream includes carrying basetrack sample entries as part of a G-PCC sample as described in ISO / IEC 23090-18.

[0391] In one implementation, a method includes: receiving a timing metadata track identifying point cloud tiles corresponding to one or more spatial regions within a point cloud scene; determining one or more point cloud tiles to be used for rendering an image at a decoding device; retrieving from a communication network one or more geometric tile tracks corresponding to the determined one or more point cloud tiles, each geometric tile track including point cloud geometry data of the corresponding tile; and processing the retrieved geometric tile tracks. A set of tile tracks carrying information required for rendering certain spatial regions or tiles within a point cloud scene can be identified. Each geometric tile track can be linked to one or more attribute tile tracks. When a data file is carried using tile tracks, a base tile track can contain tile inventory information from a sample of the base tile track, and a geometric tile track contains a sample set to signal the tile inventory present in the sample of the geometric tile track. When a data file is carried using a single track or multiple tracks, and each track carries component data, the track carrying the geometry data can contain a sample set to signal the tile inventory information. G-PCC component tile tracks belonging to an alternative group can be referenced by a G-PCC base track or a corresponding G-PCC geometric tile track. The method may further include receiving a formatted container comprising geometry-based point cloud data, the geometry-based point cloud data comprising one or more point cloud tiles; obtaining a timing metadata track from the formatted container, wherein the timing metadata track comprises a plurality of tile identifiers, wherein each tile identifier corresponds to a corresponding tile in the one or more point cloud tiles; selecting at least one selected tile from the one or more point cloud tiles, wherein the at least one selected tile corresponds to at least one tile identifier; identifying at least one geometric tile track associated with the at least one tile identifier; identifying a base track comprising initialization data of the at least one selected tile using a first track reference type associated with the at least one geometric tile track; and decoding the at least one selected tile into at least one decoded tile using the at least one geometric tile track and the initialization data. The method may further include: identifying at least one attribute tile track associated with the at least one selected tile; wherein decoding the at least one selected tile comprises utilizing the at least one geometric tile track, the at least one attribute tile track, and the initialization data in the at least one decoded tile. Decoding can be performed without decoding all the geometry-based point cloud data.The method may further include: identifying the viewport of the client; identifying at least one 3D region associated with the viewport; identifying information of at least one 3D region present in a 3D spatial region information timing metadata track sample when the information of the at least one 3D region changes dynamically; identifying which of the at least one 3D region is associated with the viewport based on the available 3D region information; identifying at least one tile associated with at least one 3D region of interest from the 3D spatial region information timing metadata track sample; identifying at least one tile track associated with at least one tile associated with at least one 3D region of interest by using information present in each tile track; extracting the identified tile track from the G-PCC data file, decoding the identified tile track, and displaying the decoded tile track based on the current viewport or the viewport. The timing metadata track can configure samples as synchronous or asynchronous samples. Asynchronous samples in the timing metadata track carry only updated 3D spatial region information referencing the most recently available 3D spatial region information in the pre-synchronous sample. This updated 3D spatial region information, including updated dimensions or associated 3D tiles and any added or removed 3D spatial regions, can be signaled using multiple tile base tracks. Different encoded versions of cloud tiles can be signaled using multiple tile base tracks and have the same group identification, e.g., a single group identification. Different encoded versions of attribute component cloud tiles can be signaled using the same group identification. Frames of point cloud data can be distributed across multiple identified temporal layers, with each frame assigned to one of the multiple identified temporal layers. The geometric tile track signals at least one temporal layer identifier for the G-PCC samples present in the geometric tile track, and the samples of the G-PCC components of the geometric tile track are grouped based on the temporal level of each sample. Frames of individual time layers among multiple identified time layers can be decoded and rendered without decoding and rendering any other time layers. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method. An apparatus includes at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, are operable to cause the apparatus to: receive timing metadata tracks identifying point cloud tiles corresponding to one or more spatial regions within a point cloud scene; determine one or more point cloud tiles to be used for rendering an image at a decoding device; retrieve one or more geometric tile tracks corresponding to the determined one or more point cloud tiles from a communication network, each geometric tile track including point cloud geometry data of the corresponding tile; and process the retrieved geometric tile tracks.

[0392] In one implementation, a method includes identifying a G-PCC base track sample (carrying a parameter set and tile inventory data) required to decode a G-PCC tile using the presentation time of the sample. The presentation time of the corresponding base track sample can be equal to or less than the presentation time of the tile track sample. When the presentation times of the base track and the tile track sample do not match, the tile track sample is decoded, or the tile inventory information of the sample is identified using a base track sample whose presentation time is closer to that of the tile track sample.

[0393] Encoding selectable tiles enables the decoding of selected tiles without decoding the entire formatted container. The base track can include a parameter set and tile inventory data. The base track sample for decoding the tile track sample can be identified using the rendering time of the corresponding sample. Geometry-based point cloud data can include multiple geometry-based point cloud compression (G-PCC) units, where each G-PCC unit includes a G-PCC type length value and a G-PCC payload. A non-transitory computer-readable medium can include computer-readable instructions configured to perform any of the methods described above.

[0394] In one implementation, a method includes: receiving a formatted container comprising geometry-based point cloud data, the geometry-based point cloud data comprising a plurality of tiles; and obtaining a timing metadata track from the formatted container, wherein the timing metadata track comprises a plurality of tile identifiers, wherein each tile identifier corresponds to a corresponding tile among the plurality of tiles. At least one selected tile is selected from the plurality of tiles, wherein the at least one selected tile corresponds to at least one tile identifier. At least one geometric tile track associated with the at least one tile identifier is identified. A base track comprising initialization data of the at least one selected tile is identified using a first track reference type associated with the at least one geometric tile track. The at least one selected tile is decoded into at least one decoded tile using the at least one geometric tile track and the initialization data. The method may further include: identifying at least one attribute tile track associated with the at least one selected tile using a second track reference type associated with the at least one geometric tile track; wherein decoding the at least one selected tile includes utilizing the at least one geometric tile track, the at least one attribute tile track, and the initialization data in the at least one decoded tile. Decoding can be performed without decoding all the geometry-based point cloud data. When tile inventory information is available in a data file, it can be signaled using a tile inventory information sample group, which groups samples with the same tile inventory information within a geometric track. When tile inventory information is available in a data file, the geometric track can contain a tile inventory information sample group type, where the tile inventory information exists in the sample group description or in the samples within the geometric track. When using a tile track to carry a data file, the basic tile track can contain the tile inventory information from the basic tile track samples. When using a tile track to carry a data file, the geometric tile track can contain sample groups to signal the tile inventory of tiles present in the samples of the geometric tile track.

[0395] In one implementation, a method includes: identifying a client's viewport; identifying at least one 3D region associated with the viewport; and identifying information about at least one 3D region present in a 3D spatial region information timing metadata track sample when the information of the at least one 3D region changes dynamically; and identifying which of the at least one 3D region is associated with the viewport based on the available 3D region information. At least one tile associated with at least one 3D region of interest from the 3D spatial region information timing metadata track sample is identified. At least one tile track associated with at least one tile associated with at least one 3D region of interest is identified using information present in each tile track. The identified tile track is extracted from a G-PCC data file, the identified tile track is decoded, and the decoded tile track is displayed based on the current viewport or the viewport. The timing metadata track can be set as synchronous or asynchronous samples. Samples can exist in a specific number of samples. Samples can exist at specific time intervals. Asynchronous samples in the timing metadata track can reference 3D spatial region information recently available in the previous synchronous sample, carrying only the updated 3D spatial region information. Asynchronous samples in the timing metadata track can reference 3D spatial region information recently available in the previous synchronous sample, only sending updated 3D spatial region information via signaling. This includes updated dimensions or associated 3D tiles, and any added or removed 3D spatial regions. A dynamic tile ID flag indicates whether the associated tile references for a 3D spatial region in the current sample were updated in the previous synchronous sample. It can include an indication of the number of updated 3D spatial regions referenced in the previous synchronous sample and signaled in the current sample. The timing metadata track can include the 3D region identifier, offset, and bounding box information for each 3D region.

[0396] In one implementation, a method includes identifying a 3D object of interest and viewport information; identifying tiles associated with the 3D object of interest; identifying at least one tile associated with the 3D object of interest by using information present in a 3D object information timing metadata track sample when the spatial information of the 3D object changes dynamically; and identifying at least one tile track associated with the at least one tile using information present in each tile track. For the viewport, a 3D region associated with the viewport information is identified. When the information of the 3D region changes dynamically, 3D region information present in a 3D spatial region information timing metadata track sample is identified. Based on the available 3D region information, a 3D region associated with the viewport region is identified. Tiles associated with the 3D region of interest from the 3D spatial region information timing metadata track sample are identified. At least one tile track associated with the identified tile is identified using information present in each tile track. At least one tile track stream is extracted from a G-PCC data file, at least one tile track stream is decoded, and the decoded tile track is displayed based on the current viewport or the viewport. The viewport may be the viewport of interest.

[0397] A method includes: receiving an associated spatial region property item corresponding to one or more spatial regions within a point cloud scene; at a decoding device, determining one or more point cloud tiles for a frame to be used to render the point cloud scene; and retrieving one or more tile items corresponding to the determined one or more point cloud tiles from a communication network, each tile item including point cloud geometry data of the corresponding tile. The retrieved tile items are then processed. Tile items containing point cloud tiles are identified by interpreting associated spatial region image properties and associated tile information item properties, wherein at least some of the one or more point cloud tiles are stored in separate image items. The image items may be associated with tile information item properties or subsample information item properties suitable for indicating an identifier of a tile contained within a point cloud tile. Spatial region item properties and tile information item properties facilitate partial access to non-timing cloud tile data. Each tile item may also include attribute data.

[0398] A method includes: receiving timing metadata tracks that identify point cloud tiles corresponding to one or more spatial regions within a point cloud scene; at a decoding device, determining one or more point cloud tiles to be used for rendering an image; and retrieving from a communication network one or more geometric tile tracks corresponding to the determined one or more point cloud tiles, each geometric tile track including point cloud geometry data of the corresponding tile. The retrieved geometric tile tracks are then processed. Different encoded versions of cloud tiles are signaled within a basic tile track and have the same group identification.

[0399] In one implementation, a method includes: receiving timing metadata tracks that identify point cloud tiles corresponding to one or more spatial regions within a point cloud scene; at a decoding device, determining one or more point cloud tiles to be used for rendering an image, and retrieving from a communication network one or more geometric tile tracks corresponding to the determined one or more point cloud tiles, each geometric tile track including point cloud geometric data of the corresponding tile. The retrieved geometric tile tracks are then processed. Different encoded versions of cloud tiles can be signaled within a single tile base track and can have the same group identification. Frames of point cloud data can be distributed across multiple identified temporal layers, and each frame can be assigned to one of the multiple identified temporal layers. Frames of individual temporal layers among the multiple identified temporal layers can be decoded and rendered without decoding and rendering any other temporal layers. A maximum number of temporal layers present in a data file, including timing metadata tracks, can be identified in the data file. At least one temporal layer identifier of G-PCC samples present in the geometric tile track can be signaled. Samples of the G-PCC components of the geometric tile track can be grouped based on the temporal level of each sample.

[0400] Although features and elements have been described above in specific combinations, those skilled in the art will understand that each feature or element may be used alone or in any combination with other features and elements. Furthermore, the methods described herein may be implemented in computer-readable instructions, computer programs, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor. The computer-readable medium may be a non-transitory storage medium. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media (such as internal hard disks and removable disks), magneto-optical media, and optical media (such as CD-ROM disks and digital versatile optical discs (DVDs)). A processor associated with the software may be used to implement the radio frequency transceiver in a wireless transmit / receive unit (WTRU), user equipment (UE), terminal, base station, radio network controller (RNC), or any host.

Claims

1. A method for processing tile tracks for point cloud data, the method comprising: Receive timing metadata tracks for point cloud tiles that correspond to one or more spatial regions within a point cloud scene in a geometry-based point cloud compression (G-PCC) data file; At the decoding device, one or more point cloud tiles are determined to be used for rendering the image; Retrieve one or more geometric tile tracks corresponding to one or more identified point cloud tiles from the communication network, each geometric tile track including the point cloud geometry data of the corresponding tile; as well as Process the retrieved geometric tile tracks.

2. The method according to claim 1, further comprising identifying a set of tile tracks carrying information required for rendering certain spatial regions or tiles within the point cloud scene.

3. The method of claim 1 or 2, wherein each geometric tile track is linked to one or more attribute tile tracks.

4. The method according to claim 1 or 2, wherein when using a tile track to carry the data file: The basic tile track contains tile inventory information from the basic tile track sample, and The geometric tile track contains a sample set to signal the tile inventory of tiles present in the sample of the geometric tile track.

5. The method of claim 1 or 2, wherein the G-PCC component tile track belonging to an alternative group is referenced by the G-PCC base track or the corresponding G-PCC geometric tile track.

6. The method according to claim 1 or 2, further comprising: Receive a formatted container that includes geometry-based point cloud data, the geometry-based point cloud data including the one or more point cloud tiles; The timing metadata track is obtained from the formatted container. The timing metadata track includes multiple tile identifiers, and Each tile identifier corresponds to a specific tile in the one or more point cloud tiles; Select at least one selected tile from the one or more point cloud tiles, wherein the at least one selected tile corresponds to at least one tile identifier; Identify at least one geometric tile track associated with the at least one tile identifier; Using a first track reference type associated with the at least one geometric tile track, a basic track including initialization data of the at least one selected tile is identified; as well as The at least one selected tile is decoded into at least one decoded tile using the at least one geometric tile track and the initialization data.

7. The method according to claim 6, further comprising: Identify at least one attribute tile track associated with the at least one selected tile; Decoding the at least one selected tile includes utilizing the at least one geometric tile track, the at least one attribute tile track, and the initialization data in the at least one decoded tile.

8. The method of claim 6, wherein decoding is performed without decoding all of the geometry-based point cloud data.

9. The method according to claim 1 or 2, further comprising: Identify the client's viewport; Identify at least one 3D region associated with the viewport; When the information of the at least one 3D region changes dynamically, the information of the at least one 3D region present in the timing metadata track sample of the 3D spatial region information is identified. Based on available 3D region information, identify which of the at least one 3D region is associated with the viewport; Identify at least one tile associated with at least one 3D region of interest from the timed metadata track samples of the 3D spatial region information; At least one tile track is identified by using the information present in each tile track and associated with at least one tile of interest in at least one 3D region of interest. as well as Extract the identified tile tracks from the G-PCC data file, decode the identified tile tracks, and display the decoded tile tracks based on the current viewport or the viewport.

10. The method of claim 1 or 2, wherein the timing metadata track sets the sample as a synchronous sample or a asynchronous sample, and wherein the asynchronous sample in the timing metadata track is only signaled to send updated 3D spatial region information referencing the 3D spatial region information most recently available in the previous synchronous sample, including updated dimensions or associated 3D tiles and any added or removed 3D spatial regions.

11. The method of claim 1 or 2, wherein different encoded versions of cloud tiles are transmitted by signal using multiple tile basic tracks and have the same group identification.

12. The method according to claim 6, The point cloud data is distributed across multiple recognition time layers. Each frame is assigned to one of the multiple recognition time layers. The geometric tile track transmits at least one time-layer identifier of the G-PCC samples present in the geometric tile track via a signal, and The samples of the G-PCC component of the geometric tile track are grouped based on the time level of each sample.

13. The method of claim 12, wherein frames of individual time layers among the plurality of identified time layers are decoded and rendered without decoding and rendering any other time layers.

14. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform the method according to claim 1 or 2.

15. An apparatus for processing tile tracks for point cloud data, the apparatus comprising: At least one processor; and At least one memory storing instructions that, when executed by the at least one processor, are operable to cause the device to: Receive timing metadata tracks for point cloud tiles that correspond to one or more spatial regions within a point cloud scene in a geometry-based point cloud compression (G-PCC) data file; At the decoding device, one or more point cloud tiles are determined to be used for rendering the image; Retrieve one or more geometric tile tracks corresponding to one or more identified point cloud tiles from the communication network, each geometric tile track including the point cloud geometry data of the corresponding tile; as well as Process the retrieved geometric tile tracks.

Images