Video decoding apparatus and method for video decoding

Abstract

The invention discloses a video decoding apparatus and a method for video decoding. The method includes: receiving a media container file, where the media container file indicates an atlas track associated with an atlas frame, the atlas track including information about one or more blocks of the atlas frame; obtaining information of the one or more blocks of the atlas frame included in the atlas track based on the media container file; and decoding at least a portion of the atlas frame based on the obtained information.

Description

[0001] This application is a divisional application of the invention patent application with application number 202080072900.0 filed on September 23, 2020 and invention name “Video-based Point Cloud Stream”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims the benefit of U.S. Provisional Application No. 62 / 907,249, filed on September 27, 2019, and U.S. Provisional Application No. 62 / 958,031, filed on January 7, 2020, the disclosures of which are incorporated herein by reference in their entireties. Background Art

[0004] Video coding systems can be used to compress and / or decompress digital video signals, for example to reduce the storage and / or transmission bandwidth required for such signals. Three-dimensional (3D) point clouds have emerged as advanced representations of immersive media. These point clouds can be captured in a variety of ways, using, for example, multiple cameras, depth sensors, and / or light detection and ranging (LiDAR) laser scanners. The number of points required to realistically reconstruct objects and / or scenes in 3D space can be on the order of millions or billions. Therefore, efficient representation, compression, and / or delivery techniques are desired for storing and / or transmitting point cloud data. Summary of the invention

[0005] The present invention discloses systems, methods and tools for processing video data associated with a three-dimensional (3D) space. A video decoding device as described herein may include a processor configured to receive a media container file (e.g., an International Organization for Standardization (ISO) Base Media File Format (ISOBMFF) container file), the media container file including a video-based point cloud compression (V-PCC) bitstream. The processor may determine, based on the media container file, that a first plurality of V-PCC component tracks are associated with a first V-PCC component and a second plurality of V-PCC component tracks are associated with a second V-PCC component. Each V-PCC component track in the first plurality of V-PCC component tracks may correspond to a corresponding encoded version of the first V-PCC component and each V-PCC component track in the second plurality of V-PCC component tracks may correspond to a corresponding encoded version of the second V-PCC component. For example, a first plurality of V-PCC component tracks may include alternative encoded versions of a first V-PCC component, a second plurality of V-PCC component tracks may include alternative encoded versions of a second V-PCC component, and the first plurality of V-PCC component tracks and the second plurality of V-PCC component tracks may belong to respective first and second alternative track groups. The alternative encoded versions of the first V-PCC component may differ from each other by at least one attribute and the alternative encoded versions of the second V-PCC component may also differ from each other by at least one attribute. The processor may determine, based on the media container file, that a first V-PCC component track in the first plurality of V-PCC component tracks and a second V-PCC component track in the second plurality of V-PCC component tracks belong to the same broadcast track group, for example, based on determining that both the first V-PCC component track and the second V-PCC component track are associated with an identifier of the broadcast track group. In response to determining that the first V-PCC component track and the second V-PCC component track belong to the same broadcast track group, the processor may play the first V-PCC component track and the second V-PCC component track together as a group, for example, because the first V-PCC component track and the second V-PCC component track are encoded in a substantially similar manner with respect to at least one attribute (for example, the first V-PCC component track and the second V-PCC component track may have the same resolution).

[0006] The video decoding device as described herein may also be configured to determine, based on the media container file, that at least one of the first V-PCC component track or the second V-PCC component track includes groups of blocks corresponding to different objects in the point cloud or different parts of the same object in the point cloud. The video decoding device may be configured to decode the groups of blocks independently of each other so that a subset of the objects or parts of the point cloud may be accessed without also accessing the rest of the objects or parts. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1Ais a system diagram illustrating an exemplary communication system in which one or more disclosed embodiments may be implemented.

[0008] Figure 1B is a diagram showing that according to one embodiment Figure 1A A system diagram of an exemplary wireless transmit / receive unit (WTRU) for use within a communication system is shown.

[0009] Figure 1C is a diagram showing that according to one embodiment Figure 1A A system diagram of an exemplary radio access network (RAN) and an exemplary core network (CN) for use within a communication system is shown.

[0010] Figure 1D is a diagram showing that according to one embodiment Figure 1A System diagram of an additional exemplary RAN and an additional exemplary CN for use within the illustrated communication system.

[0011] Figure 2 An exemplary video-based point cloud compression (V-PCC) bitstream structure including multiple V-PCC units is shown.

[0012] Figure 3 An exemplary media container structure is shown.

[0013] Figure 4 An exemplary constraint is shown in which the inner random access point (IRAP) samples of the components are aligned.

[0014] Figure 5 An example of using the least common multiple of the IRAP periods to indicate the V-PCC IRAP is shown.

[0015] Figure 6 An exemplary media container structure that may be used to enable spatial access to specific regions within a 3D space is shown. DETAILED DESCRIPTION

[0016] A more detailed understanding may be obtained from the following description given by way of example in conjunction with the accompanying drawings.

[0017] Figure 1A1 is a diagram illustrating an exemplary communication system 100 in which one or more disclosed embodiments may be implemented. The communication system 100 may be a multiple access system that provides content such as voice, data, video, messaging, broadcast, etc. to multiple wireless users. The communication system 100 may enable multiple wireless users to access such content through sharing of system resources (including wireless bandwidth). For example, the communication system 100 may employ one or more channel access methods such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single carrier FDMA (SC-FDMA), zero tail unique word DFT spread OFDM (ZT UW DTS-s OFDM), unique word OFDM (UW-OFDM), resource block filtered OFDM, filter bank multi-carrier (FBMC), etc.

[0018] like Figure 1A As shown, the communication system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, RAN 104 / 113, CN 106 / 115, public switched telephone network (PSTN) 108, Internet 110 and other networks 112, but it should be understood that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Each of the WTRUs 102a, 102b, 102c, 102d may be any type of device configured to operate and / or communicate in a wireless environment. By way of example, the WTRUs 102a, 102b, 102c, 102d (any of which may be referred to as a “station” and / or “STA”) may be configured to transmit and / or receive wireless signals and may include user equipment (UE), a mobile station, a fixed or mobile subscriber unit, a subscription-based unit, a pager, a cellular phone, a personal digital assistant (PDA), a smart phone, a laptop, a netbook, a personal computer, a wireless sensor, a hotspot or Mi-Fi device, an Internet of Things (IoT) device, a watch or other wearable device, a head-mounted display (HMD), a vehicle, a drone, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in an industrial and / or automated process chain environment), a consumer electronic device, a device operating on a commercial and / or industrial wireless network, etc. Any of the WTRUs 102a, 102b, 102c, and 102d may be interchangeably referred to as a UE.

[0019] The communication system 100 may also include a base station 114a and / or a base station 114b. Each of the base stations 114a, 114b may be any type of device configured to wirelessly interface with at least one of the WTRUs 102a, 102b, 102c, 102d to facilitate access to one or more communication networks, such as the CN 106 / 115, the Internet 110, and / or other networks 112. By way of example, the base stations 114a, 114b may be a base transceiver station (BTS), a Node B, an eNode B, a Home Node B, a Home eNode B, a gNB, an NR Node B, a site controller, an access point (AP), a wireless router, etc. While the base stations 114a, 114b are each depicted as a single element, it should be appreciated that the base stations 114a, 114b may include any number of interconnected base stations and / or network elements.

[0020] The base station 114a may be part of the RAN 104 / 113, which may also include other base stations and / or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), a relay node, etc. The base station 114a and / or the base station 114b may be configured to transmit and / or receive wireless signals on one or more carrier frequencies (which may be referred to as cells (not shown)). These frequencies may be in a licensed spectrum, an unlicensed spectrum, or a combination of licensed and unlicensed spectrums. A cell may provide coverage of wireless services to a specific geographic area, which may be relatively fixed or may change over time. The cell may be further divided into cell sectors. For example, a cell associated with the base station 114a may be divided into three sectors. Therefore, in one embodiment, the base station 114a may include three transceivers, i.e., one transceiver for each sector of the cell. In one embodiment, the base station 114a may employ multiple-input multiple-output (MIMO) technology and may utilize multiple transceivers for each sector of the cell. For example, beamforming may be used to transmit and / or receive signals in a desired spatial direction.

[0021] The base stations 114a, 114b may communicate with one or more of the WTRUs 102a, 102b, 102c, 102d over an air interface 116, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 116 may be established using any suitable radio access technology (RAT).

[0022] More specifically, as noted above, the communication system 100 may be a multiple access system and may employ one or more channel access schemes such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, the base station 114a in the RAN 104 / 113 and the WTRUs 102a, 102b, 102c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may use Wideband CDMA (WCDMA) to establish the air interface 115 / 116 / 117. WCDMA may include communication protocols such as High Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA may include High Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High Speed ​​UL Packet Access (HSUPA).

[0023] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface 116 using Long Term Evolution (LTE) and / or Advanced LTE (LTE-A) and / or Advanced LTE Pro (LTE-A Pro).

[0024] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as NR radio access, which may establish the air interface 116 using New Radio (NR).

[0025] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement multiple radio access technologies. For example, the base station 114a and the WTRUs 102a, 102b, 102c may implement LTE radio access and NR radio access together, for example using dual connectivity (DC) principles. Thus, the air interface utilized by the WTRUs 102a, 102b, 102c may be characterized by multiple types of radio access technologies and / or transmissions to / from multiple types of base stations (e.g., eNBs and gNBs).

[0026] In other embodiments, the base station 114a and the WTRUs 102a, 102b, 102c may implement radio technologies such as IEEE 802.11 (i.e., Wireless Fidelity (WiFi), IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile Communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), GSM EDGE (GERAN), etc.

[0027] Figure 1A The base station 114b in the may be, for example, a wireless router, a Home NodeB, a Home eNodeB, or an access point, and may utilize any suitable RAT to facilitate wireless connectivity in a local area, such as a business location, a home, a vehicle, a campus, an industrial facility, an air corridor (e.g., for use by drones), a road, and the like. In one embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as 802.11 to establish a wireless local area network (WLAN). In one embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, the base station 114b and the WTRUs 102c, 102d may utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.) to establish a picocell or a femtocell. As Figure 1A As shown, base station 114b may have a direct connection to Internet 110. Therefore, base station 114b may not need to access Internet 110 via CN 106 / 115.

[0028] The RAN 104 / 113 may be in communication with the CN 106 / 115, which may be any type of network configured to provide voice, data, applications, and / or Voice over Internet Protocol (VoIP) to one or more of the WTRUs 102a, 102b, 102c, 102d. The data may have different quality of service (QoS) requirements, such as different throughput requirements, delay requirements, error tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. The CN 106 / 115 may provide call control, billing services, mobile location-based services, pre-paid calling, Internet connectivity, video distribution, etc., and / or perform high-level security functions, such as user authentication. Although not described in detail in the accompanying drawings, the CN 106 / 115 may be used to provide a plurality of WTRUs 102a, 102b, 102c, 102d, and / or a plurality of WTRUs 102a, 102b, 102c, 102d. Figure 1AAlthough not shown in the figure, it will be appreciated that the RAN 104 / 113 and / or the CN 106 / 115 may be in direct or indirect communication with other RANs that employ the same RAT as the RAN 104 / 113 or a different RAT. For example, in addition to being connected to the RAN 104 / 113, which may utilize NR radio technology, the CN 106 / 115 may also be in communication with another RAN (not shown) that employs GSM, UMTS, CDMA 2000, WiMAX, E-UTRA, or WiFi radio technology.

[0029] The CN 106 / 115 may also serve as a gateway for the WTRUs 102a, 102b, 102c, 102d to access the PSTN 108, the Internet 110, and / or other networks 112. The PSTN 108 may include a circuit-switched telephone network that provides plain old telephone service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices that use common communication protocols, such as the Transmission Control Protocol (TCP), the User Datagram Protocol (UDP), and / or the Internet Protocol (IP) in the TCP / IP Internet protocol suite. The networks 112 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, the networks 112 may include another CN connected to one or more RANs, which may employ the same RAT as the RAN 104 / 113 or a different RAT.

[0030] Some or all of the WTRUs 102a, 102b, 102c, 102d in the communication system 100 may include multi-mode capabilities (e.g., the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers for communicating with different wireless networks via different wireless links). Figure 1A The illustrated WTRU 102c may be configured to communicate with the base station 114a, which may employ a cellular-based radio technology, and with the base station 114b, which may employ an IEEE 802 radio technology.

[0031] Figure 1B is a system diagram illustrating an exemplary WTRU 102. Figure 1B As shown, the WTRU 102 may include, among other things, a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a keypad 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power source 134, a global positioning system (GPS) chipset 136, and / or other peripherals 138. It will be appreciated that the WTRU 102 may include any subcombination of the foregoing elements while remaining consistent with an embodiment.

[0032] The processor 118 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. The processor 118 may perform signal coding, data processing, power control, input / output processing, and / or any other functionality that enables the WTRU 102 to operate in a wireless environment. The processor 118 may be coupled to the transceiver 120, which may be coupled to the transmit / receive element 122. Although Figure 1B The processor 118 and the transceiver 120 are depicted as separate components, but it is understood that the processor 118 and the transceiver 120 may be integrated together in an electronic package or chip.

[0033] The transmit / receive element 122 may be configured to transmit signals to or receive signals from a base station (e.g., base station 114a) via an air interface 116. For example, in one embodiment, the transmit / receive element 122 may be an antenna configured to transmit and / or receive RF signals. For example, in one embodiment, the transmit / receive element 122 may be an emitter / detector configured to transmit and / or receive IR, UV, or visible light signals. In yet another embodiment, the transmit / receive element 122 may be configured to transmit and / or receive RF and light signals. It should be understood that the transmit / receive element 122 may be configured to transmit and / or receive any combination of wireless signals.

[0034] Although the transmit / receive element 122 is Figure 1B 1 as a single element, the WTRU 102 may include any number of transmit / receive elements 122. More specifically, the WTRU 102 may employ MIMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit / receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 116.

[0035] The transceiver 120 may be configured to modulate signals to be transmitted by the transmit / receive element 122 and to demodulate signals received by the transmit / receive element 122. As noted above, the WTRU 102 may have multi-mode capabilities. Thus, the transceiver 120 may include multiple transceivers to enable the WTRU 102 to communicate via multiple RATs, such as NR and IEEE 802.11, for example.

[0036] The processor 118 of the WTRU 102 may be coupled to a speaker / microphone 124, a keypad 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) display unit or an organic light emitting diode (OLED) display unit) and may receive user input data therefrom. The processor 118 may also output user data to the speaker / microphone 124, the keypad 126, and / or the display / touchpad 128. In addition, the processor 118 may access information from and store data in any type of suitable memory, such as a non-removable memory 130 and / or a removable memory 132. The non-removable memory 130 may include a random access memory (RAM), a read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. In other embodiments, the processor 118 may access information from and store data in a memory that is not physically located on the WTRU 102, such as on a server or a home computer (not shown).

[0037] The processor 118 may receive power from the power source 134, and may be configured to distribute and / or control power to other components in the WTRU 102. The power source 134 may be any suitable device for powering the WTRU 102. For example, the power source 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel-metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0038] The processor 118 may also be coupled to the GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 102. In addition to or in lieu of the information from the GPS chipset 136, the WTRU 102 may receive location information from a base station (e.g., base stations 114a, 114b) over the air interface 116 and / or determine its location based on the timing of signals received from two or more nearby base stations. It will be appreciated that the WTRU 102 may acquire location information by any suitable location-determination method while remaining consistent with an embodiment.

[0039] The processor 118 may also be coupled to other peripherals 138, which may include one or more software modules and / or hardware modules that provide additional features, functionality, and / or wired or wireless connectivity. For example, the peripherals 138 may include an accelerometer, an electronic compass, a satellite transceiver, a digital camera (for photos and / or video), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands-free headset, module, a frequency modulation (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, a virtual reality and / or augmented reality (VR / AR) device, an activity tracker, etc. Peripheral device 138 may include one or more sensors, which may be one or more of the following: a gyroscope, an accelerometer, a Hall effect sensor, a magnetometer, an orientation sensor, a proximity sensor, a temperature sensor, a time sensor; a geolocation sensor; an altimeter, a light sensor, a touch sensor, a magnetometer, a barometer, a gesture sensor, a biometric sensor, and / or a humidity sensor.

[0040] The WTRU 102 may include a full-duplex radio for which transmission and reception of some or all signals (e.g., associated with specific subframes for UL (e.g., for transmission) and downlink (e.g., for reception)) may be concurrent and / or simultaneous. The full-duplex radio may include an interference management unit for reducing and / or substantially eliminating self-interference via hardware (e.g., choke) or via signal processing performed by a processor (e.g., a separate processor (not shown) or via the processor 118). In one embodiment, the WTRU 102 may include a half-duplex radio for which transmission and reception of some or all signals (e.g., associated with specific subframes for UL (e.g., for transmission) or downlink (e.g., for reception)) may be concurrent and / or simultaneous.

[0041] Figure 1C 1 is a system diagram illustrating the RAN 104 and the CN 106 according to one embodiment. As noted above, the RAN 104 may employ an E-UTRA radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 116. The RAN 104 may also be in communication with the CN 106.

[0042] The RAN 104 may include evolved Node-Bs 160a, 160b, 160c, though it will be appreciated that the RAN 104 may include any number of evolved Node-Bs while remaining consistent with an embodiment. The evolved Node-Bs 160a, 160b, 160c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116. In one embodiment, the evolved Node-Bs 160a, 160b, 160c may implement MIMO technology. Thus, the evolved Node-B 160a, for example, may use multiple antennas to transmit wireless signals to and / or receive wireless signals from the WTRU 102a.

[0043] Each of the evolved Node Bs 160a, 160b, 160c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the UL and / or DL, etc. Figure 1C As shown, the eNode-Bs 160a, 160b, 160c may communicate with one another via an X2 interface.

[0044] Figure 1C The illustrated CN 106 may include a mobility management entity (MME) 162, a serving gateway (SGW) 164, and a packet data network (PDN) gateway (or PGW) 166. While each of the foregoing elements is depicted as part of the CN 106, it should be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0045] The MME 162 may be connected to each of the evolved Node-Bs 162a, 162b, 162c in the RAN 104 via an S1 interface and may serve as a control node. For example, the MME 162 may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, bearer activation / deactivation, selecting a particular serving gateway during an initial attach of the WTRUs 102a, 102b, 102c, and the like. The MME 162 may provide a control plane function for switching between the RAN 104 and other RANs (not shown) that employ other radio technologies, such as GSM and / or WCDMA.

[0046] The SGW 164 may be connected to each of the evolved Node-Bs 160a, 160b, 160c in the RAN 104 via an S1 interface. The SGW 164 may generally route and forward user data packets to / from the WTRUs 102a, 102b, 102c. The SGW 164 may perform other functions, such as anchoring the user plane during inter-evolved Node-B handovers, triggering paging when DL data is available for the WTRUs 102a, 102b, 102c, managing and storing the context of the WTRUs 102a, 102b, 102c, and the like.

[0047] The SGW 164 may be connected to the PGW 166, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices.

[0048] The CN 106 may facilitate communications with other networks. For example, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices. For example, the CN 106 may include, or may be in communication with, an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that serves as an interface between the CN 106 and the PSTN 108. In addition, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0049] Although the WTRU Figures 1A to 1D Although described as a wireless terminal, it is contemplated that in certain representative embodiments, such a terminal may (eg, temporarily or permanently) use a wired communications interface with a communications network.

[0050] In a representative embodiment, the other network 112 may be a WLAN.

[0051] A WLAN in infrastructure basic service set (BSS) mode may have an access point (AP) for a BSS and one or more stations (STAs) associated with the AP. The AP may have access or an interface to a distribution system (DS) or another type of wired / wireless network that carries traffic to and / or carries traffic away from the BSS. Traffic originating from outside the BSS and leading to the STA may be reached by the AP and may be delivered to the STA. Traffic originating from the STA and leading to a destination outside the BSS may be sent to the AP to be delivered to the corresponding destination. Traffic between STAs within the BSS may be sent by the AP, for example, wherein the source STA may send traffic to the AP, and the AP may deliver the traffic to the destination STA. Traffic between STAs within the BSS may be considered and / or referred to as point-to-point traffic. Point-to-point traffic may be sent between the source and destination STAs (e.g., directly between them) using a direct link setup (DLS). In certain representative embodiments, the DLS may use 802.11e DLS or 802.11z tunnel DLS (TDLS). A WLAN using an independent BSS (IBSS) mode may not have an AP, and STAs (eg, all STAs) within or using the IBSS may communicate directly with each other. The IBSS communication mode may sometimes be referred to herein as an "ad-hoc" communication mode.

[0052] When using the 802.11ac infrastructure operating mode or a similar operating mode, the AP may transmit a beacon on a fixed channel (such as a primary channel). The primary channel may be a fixed width (e.g., a 20 MHz wide bandwidth) or a width dynamically set via signaling. The primary channel may be an operating channel of the BSS and may be used by the STA to establish a connection with the AP. In certain representative embodiments, carrier sense multiple access / collision avoidance (CSMA / CA) may be implemented, for example, in an 802.11 system. For CSMA / CA, a STA (e.g., each STA) (including the AP) may sense the primary channel. If the primary channel is sensed / detected and / or determined to be busy by a particular STA, the particular STA may back off. One STA (e.g., only one station) may transmit at any given time in a given BSS.

[0053] A high throughput (HT) STA may communicate using a 40 MHz wide channel, for example, via a combination of a 20 MHz primary channel and adjacent or non-adjacent 20 MHz channels to form a 40 MHz wide channel.

[0054] Very high throughput (VHT) STA can support 20MHz, 40MHz, 80MHz and / or 160MHz wide channels. 40MHz and / or 80MHz channels can be formed by combining continuous 20MHz channels. A 160MHz channel can be formed by combining 8 continuous 20MHz channels, or by combining two non-continuous 80MHz channels (this can be called 80+80 configuration). For the 80+80 configuration, after channel coding, the data can be passed through a segment parser that can divide the data into two streams. Each stream can be processed by inverse fast Fourier transform (IFFT) and time domain processing separately. These streams can be mapped to two 80MHz channels, and data can be transmitted by transmitting STA. At the receiver of the receiving STA, the above-mentioned operation for the 80+80 configuration can be reversed, and the combined data can be sent to the medium access control (MAC).

[0055] Sub 1GHz operating mode is supported by 802.11af and 802.11ah. Channel operating bandwidth and carrier are reduced in 802.11af and 802.11ah relative to those used in 802.11n and 802.11ac. 802.11af supports 5MHz, 10MHz and 20MHz bandwidths in the TV White Space (TVWS) spectrum, while 802.11ah uses non-TVWS spectrum to support 1MHz, 2MHz, 4MHz, 8MHz and 16MHz bandwidths. According to a representative embodiment, 802.11ah may support meter type control / machine type communication, such as MTC devices in macro coverage areas. MTC devices may have certain capabilities, such as limited capabilities, including support for (e.g., only support for) certain bandwidths and / or limited bandwidths. MTC devices may include batteries with battery life above a threshold (e.g., to maintain very long battery life).

[0056] WLAN systems that can support multiple channels and channel bandwidths (such as 802.11n, 802.11ac, 802.11af, and 802.11ah) include channels that can be designated as primary channels. The primary channel may have a bandwidth equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel may be set and / or limited by a STA (which supports the minimum bandwidth operating mode) from all STAs operating in the BSS. In the example of 802.11ah, the primary channel may be 1 MHz wide for STAs (e.g., MTC type devices) that support (e.g., only support) a 1 MHz mode, even if the AP and other STAs in the BSS support 2 MHz, 4 MHz, 8 MHz, 16 MHz, and / or other channel bandwidth operating modes. Carrier sensing and / or network allocation vector (NAV) settings may depend on the state of the primary channel. If the primary channel is busy, for example, because a STA (supporting only a 1 MHz operating mode) is transmitting to the AP, the entire available band may be considered busy even if most of the band remains idle and may be available.

[0057] In the United States, the available frequency band that can be used by 802.11ah is from 902MHz to 928MHz. In South Korea, the available frequency band is from 917.5MHz to 923.5MHz. In Japan, the available frequency band is from 916.5MHz to 927.5MHz. Depending on the country code, the total bandwidth available for 802.11ah is 6MHz to 26MHz.

[0058] Figure 1D1 is a system diagram showing the RAN 113 and the CN 115 according to one embodiment. As noted above, the RAN 113 may employ NR radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 116. The RAN 113 may also be in communication with the CN 115.

[0059] The RAN 113 may include gNBs 180a, 180b, 180c, though it will be appreciated that the RAN 113 may include any number of gNBs while remaining consistent with an embodiment. The gNBs 180a, 180b, 180c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116. In one embodiment, the gNBs 180a, 180b, 180c may implement MIMO technology. For example, the gNBs 180a, 180b may utilize beamforming to transmit signals to and / or receive signals from the gNBs 180a, 180b, 180c. Thus, the gNB 180a may, for example, use multiple antennas to transmit wireless signals to and / or receive wireless signals from the WTRU 102a. In one embodiment, the gNBs 180a, 180b, 180c may implement carrier aggregation technology. For example, gNB 180a may transmit multiple component carriers to WTRU 102a (not shown). A subset of these component carriers may be on unlicensed spectrum, while the remaining component carriers may be on licensed spectrum. In one embodiment, gNBs 180a, 180b, 180c may implement coordinated multi-point (CoMP) technology. For example, WTRU 102a may receive coordinated transmissions from gNB 180a and gNB 180b (and / or gNB 180c).

[0060] The WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using transmissions associated with scalable parameter sets. For example, OFDM symbol spacing and / or OFDM subcarrier spacing may vary for different transmissions, different cells, and / or different portions of the wireless transmission spectrum. The WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using subframes or transmission time intervals (TTIs) of varying or scalable lengths (e.g., containing varying numbers of OFDM symbols and / or durations of varying absolute time).

[0061] The gNBs 180a, 180b, 180c may be configured to communicate with the WTRUs 102a, 102b, 102c in a standalone configuration and / or a non-standalone configuration. In a standalone configuration, the WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c without also accessing other RANs (e.g., such as the eNodeBs 160a, 160b, 160c). In a standalone configuration, the WTRUs 102a, 102b, 102c may utilize one or more of the gNBs 180a, 180b, 180c as mobility anchors. In a standalone configuration, the WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using signals in an unlicensed band. In a non-standalone configuration, the WTRU 102a, 102b, 102c may communicate / be connected to a gNB 180a, 180b, 180c while also communicating / being connected to another RAN, such as an evolved Node-B 160a, 160b, 160c. For example, the WTRU 102a, 102b, 102c may implement the DC principle to communicate with one or more gNBs 180a, 180b, 180c and one or more evolved Node-Bs 160a, 160b, 160c substantially simultaneously. In a non-standalone configuration, the evolved Node-B 160a, 160b, 160c may serve as a mobility anchor for the WTRU 102a, 102b, 102c and the gNB 180a, 180b, 180c may provide additional coverage and / or throughput to serve the WTRU 102a, 102b, 102c.

[0062] Each of the gNBs 180a, 180b, 180c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in UL and / or DL, support of network slicing, dual connectivity, interworking between NR and E-UTRA, routing of user plane data towards a user plane function (UPF) 184a, 184b, routing of control plane information towards an access and mobility management function (AMF) 182a, 182b, etc. As shown in FIG. Figure 1D As shown, gNBs 180a, 180b, and 180c may communicate with each other via an Xn interface.

[0063] Figure 1DThe illustrated CN 115 may include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one session management function (SMF) 183a, 183b, and possible data networks (DNs) 185a, 185b. Although each of the foregoing elements is depicted as part of the CN 115, it should be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0064] The AMF 182a, 182b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 113 via the N2 interface and may serve as a control node. For example, the AMF 182a, 182b may be responsible for authenticating users of the WTRU 102a, 102b, 102c, support for network slicing (e.g., handling of different PDU sessions with different requirements), selecting a specific SMF 183a, 183b, management of registration areas, termination of NAS signaling, mobility management, etc. Network slicing may be used by the AMF 182a, 182b to customize CN support for the WTRU 102a, 102b, 102c based on the type of service that is utilizing the WTRU 102a, 102b, 102c. For example, different network slices may be established for different use cases, such as services relying on ultra-reliable low latency (URLLC) access, services relying on enhanced mobile broadband (eMBB) access, services for machine type communication (MTC) access, etc. The AMF 162 may provide a control plane function for switching between the RAN 113 and other RANs (not shown) that employ other radio technologies, such as LTE, LTE-A, LTE-APro, and / or non-3GPP access technologies, such as WiFi.

[0065] The SMF 183a, 183b may be connected to the AMF 182a, 182b via the N11 interface in the CN 115. The SMF 183a, 183b may also be connected to the UPF 184a, 184b via the N4 interface in the CN 115. The SMF 183a, 183b may select and control the UPF 184a, 184b and configure the routing of traffic through the UPF 184a, 184b. The SMF 183a, 183b may perform other functions such as managing and allocating UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, providing downlink data notification, etc. The PDU session type may be IP-based, non-IP-based, Ethernet-based, etc.

[0066] The UPF 184a, 184b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 113 via an N3 interface, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices. The UPF 184, 184b may perform other functions, such as routing and forwarding packets, enforcing user plane policies, supporting multi-homed PDU sessions, handling user plane QoS, buffering downlink packets, providing mobility anchoring, etc.

[0067] The CN 115 may facilitate communications with other networks. For example, the CN 115 may include or may communicate with an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that serves as an interface between the CN 115 and the PSTN 108. In addition, the CN 115 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers. In one embodiment, the WTRUs 102a, 102b, 102c may be connected to a local data network (DN) 185a, 185b via an N3 interface to the UPF 184a, 184b through the UPF 184a, 184b and via an N6 interface between the UPF 184a, 184b and the DN 185a, 185b.

[0068] Given that Figure 1A-Figure 1D and Figure 1A-Figure 1D 1, one or more or all of the functions described with respect to one or more of the following items may be performed by one or more simulation devices (not shown): WTRU 102a-102d, base station 114a-114b, evolved Node B 160a-160c, MME 162, SGW 164, PGW 166, gNB 180a-180c, AMF 182a-182b, UPF 184a-184b, SMF 183a-183b, DN 185a-185b and / or any other device described herein. The simulation device may be one or more devices configured to emulate one or more or all of the functions described herein. For example, the simulation device may be used to test other devices and / or simulate network and / or WTRU functions.

[0069] The simulation device may be designed to implement one or more tests of other devices in a laboratory environment and / or an operator network environment. For example, the one or more simulation devices may perform one or more or all functions while being fully or partially implemented and / or deployed as part of a wired and / or wireless communication network in order to test other devices within the communication network. The one or more simulation devices may perform one or more functions or all functions while being temporarily implemented / deployed as part of a wired and / or wireless communication network. The simulation device may be directly coupled to another device for testing purposes and / or may use over-the-air wireless communications to perform testing.

[0070] The one or more simulation devices may perform one or more (including all) functions without being implemented / deployed as part of a wired and / or wireless communication network. For example, the simulation device may be used in a test scenario in a test lab and / or a non-deployed (e.g., testing) wired and / or wireless communication network to implement testing of one or more components. The one or more simulation devices may be test equipment. Direct RF coupling and / or wireless communication via RF circuits (e.g., which may include one or more antennas) may be used by the simulation device to transmit and / or receive data.

[0071] 3D point clouds (e.g., high-quality 3D point clouds) can be used to represent immersive media. A point cloud may include one or more points (e.g., a set of points) that can be represented in 3D space using coordinates indicating the location of each point and / or one or more attributes. For example, the attributes may include one or more of the following: color, transparency, acquisition time, laser reflectivity, or material properties associated with each point, etc. Point clouds can be captured in a variety of ways. For example, multiple cameras and depth sensors can be used to capture point clouds. Light detection and ranging (LiDAR) laser scanners can be used to capture point clouds. The number of points included in a point cloud used to realistically reconstruct objects and / or scenes in 3D space can be on the order of millions or billions. Efficient representation and compression can facilitate storage and / or transmission of point cloud data.

[0072] Figure 2An exemplary structure of a bitstream 200 for video-based point cloud compression (V-PCC) is shown, which can be transmitted (e.g., signaled) by an encoding device and parsed and decoded by a decoding device. The V-PCC bitstream 200 may include a set of one or more V-PCC units 202, and Table 1 includes an exemplary syntax for signaling V-PCC units. Each V-PCC unit 202 may include a V-PCC unit header 204 and a V-PCC unit payload 206, and the V-PCC unit payload 206 may in turn include one or more of a sequence parameter set 208, occupied video data 210, various types of patch data groups 212, geometric video data 214, or attribute video data 216. The V-PCC unit header 204 may define the V-PCC unit type of the V-PCC unit (e.g., as indicated by the vpcc_unit_type field in Table 2), which may be one of a plurality of values, including, for example, VPCC_OVD, VPCC_GVD and VPCC_AVD, VPCC_PDG, VPCC_SPS, which may correspond to occupancy, geometry, attributes, patch data group, and sequence parameter set data units, respectively. Some or all of these unit types of V-PCC units may be used to reconstruct a point cloud. The V-PCC attribute unit header may specify an attribute type and its index. The V-PCC attribute unit header may allow for support of multiple instances of the same attribute type. As shown, vpcc_unit_type may indicate the type of the V-PCC unit, vpcc_sequence_parameter_set_id may indicate an identifier of a V-PCC sequence parameter set, vpcc_attribute_index may indicate an index of a V-PCC attribute, vpcc_attribute_dimension_index may indicate an index of a dimension partition of the V-PCC attribute, sps_multiple_layer_streams_present_flag may indicate whether a sequence parameter set (SPS) is associated with multiple layers or views, vpcc_layer_index may indicate an index of one of the multiple layers, pcm_separate_video_data may indicate pulse code modulation (PCM) video data (e.g., in a separate video stream) and / or parameters associated with encoding PCM data, and vpcc_reserved_zero_23bits or vpcc_reserved_zero_27bits may indicate the number of reserved zero bits.

[0073] The payload of an occupancy, geometry, and / or attribute V-PCC unit may correspond to a video data unit (e.g., a HEVC Network Abstraction Layer (NAL) unit) that may be decoded by a video decoding device (e.g., as specified in a corresponding occupancy, geometry, and attribute parameter set V-PCC unit). Table 3 shows an exemplary V-PCC unit payload syntax.

[0074] Table 1 — V-PCC unit syntax

[0075]

[0076] Table 2 — V-PCC unit header syntax

[0077]

[0078]

[0079] Table 3 — V-PCC unit payload syntax

[0080]

[0081] In an example (e.g., when lossless encoding is used in V-PCC), the encoder may generate a patch of missing points, which includes information about points that may be missing after reconstruction from a compressed V-PCC bitstream. The missing points may be referred to as missing pulse code modulation (PCM) points. The PCM points may be encoded directly, for example, without using a patch projection process. The patches of missing points may enable a decoder to reconstruct (e.g., fully reconstruct) the original point cloud that may be provided as input to the V-PCC encoder. The patches including information related to the missing points may be packaged in the same video (e.g., in the same video stream carrying other points) or in a separate video (e.g., in a video stream separate from the video stream carrying other points).

[0082] The patch data group (PDG) may be replaced by a patch NAL (PNAL) unit (e.g., or an atlas NAL unit). The PNAL unit may be equivalent to a network abstraction layer (NAL) unit for a video stream. Each PNAL unit may include a header that includes a unit type and / or additional information (e.g., such as a layer identification). The PNAL unit may be defined in one or more formats. The one or more formats may include a simple PNAL unit stream format and / or a sample stream format. In the sample stream format, an additional header may precede the PNAL unit. The additional header may indicate the size of the PNAL unit (e.g., the exact size).

[0083] The International Organization for Standardization (ISO) Base Media File Format (ISOBMFF) may define a structural, media-independent file format. ISOBMFF (e.g., an ISOBMFF container file) may include, for example, structure and / or media data information for timed presentation of media content (such as audio, video, etc.). An ISOBMFF container file may include support for non-timed data (such as metadata at different levels within the file structure). The logical structure of the file may have (e.g., may simulate) a movie that may include a collection of time-parallel tracks. The temporal structure of the file is a track that may include a sequence of time samples. The sequence may be mapped into the timeline of the entire movie. ISOBMFF may be based on the concept of a box-structured file. A box-structured file may include a series of boxes (e.g., atoms), which may have corresponding sizes and / or types (e.g., each box may be associated with a size and a type). A type may be a 32-bit value and may be represented by four printable characters, also referred to as a four-character code (4CC). Non-timed data may be included in a metadata box (e.g., at the file level) and / or may be attached to a movie box or one of the timed data streams (e.g., tracks) within the movie.

[0084] An ISOBMFF container (e.g., an ISOBMFF container file) may include a MovieBox ("moov"). The MovieBox may include metadata for media streams (e.g., continuous media streams) present in the file. The metadata may be signaled within the hierarchy of boxes in the Movie box, for example, within a TrackBox ("trak"). A track may represent a media stream (e.g., continuous media stream) present in the file. A media stream may include a sequence of samples (e.g., sample entries), such as audio or video access units of an elementary media stream, and may be encapsulated within a MediaDataBox ("mdat") (e.g., which may be present at the top level of a container). The metadata for each track may include a list of sample description entries, each sample description entry providing an encoding or encapsulation format used in the track and initialization data for processing the format. Each sample may be associated with one of the sample description entries for the track. Tools may be used to define an explicit timeline graph for each track. For example, an edit list may define an explicit timeline graph for each track. An edit list may be signaled using an EditListBox (or similar entity) via the exemplary syntax shown in Table 4, where each entry defines a portion of a track timeline, such as by mapping a portion of a composition timeline, or by indicating a "null" time or "null" edit (e.g., a portion of a presentation timeline that is not mapped to media).

[0085] Table 4—Example EditListBox Syntax

[0086]

[0087] ISOBMFF can be used to handle situations where the file author (e.g., an encoding device) can indicate certain actions to be performed on a player or renderer. In the case of a video stream, the file author can indicate such actions by using a restricted video scheme track. When the video track is a restricted video scheme track (e.g., as defined in subclause 8.15 of the ISO / IEC 14496-12 standard), the decoder post-requirements can be signaled on the track. The track can be converted into a restricted video scheme track by setting its sample entry code to the four-character code (4CC) 'resv' and adding a RestrictedSchemeInfoBox (or similar entity) to its sample description. One or more (e.g., all other) boxes can remain unmodified. The original sample entry type based on the video codec used to encode the stream can be stored in the OriginalFormatBox (or similar entity) within the RestrictedSchemeInfoBox. The RestrictedSchemeInfoBox may include three boxes: OriginalFormatBox, SchemeTypeBox, and SchemeInformationBox. The OriginalFormatBox may store the original sample entry type, which is based on the video codec used to encode the component stream. The properties (eg, attributes) of the restrictions may be defined in SchemeTypeBox.

[0088] Figure 3An exemplary structure of an ISOBMFF V-PCC container 300 is shown. Based on this exemplary structure, the V-PCC ISOBMFF container may include one or more of the following. The V-PCC ISOBMFF container may include a V-PCC track 302. The V-PCC track 302 may include one or more sequence parameter sets and / or samples that carry a payload of one or more non-video coding information V-PCC units (e.g., V-PCC unit types of VPCC_SPS and / or VPCC_PDG). The V-PCC track 302 may provide track references to other tracks containing samples that carry a payload of one or more video compression V-PCC units (e.g., V-PCC unit types of VPCC_GVD, VPCC_AVD, and / or VPCC_OVD). The V-PCC ISOBMFF container may include one or more restricted video scheme tracks 304, whose samples may include NAL units of a video coding elementary stream of geometry data (e.g., payload of a V-PCC unit of type VPCC_GVD). The V-PCC ISOBMFF container may include one or more constrained video scheme tracks 306, whose samples may include NAL units of a video coding elementary stream of attribute data (e.g., payload of a V-PCC unit of type VPCC_AVD). The V-PCC ISOBMFF container may include one or more constrained video scheme tracks 308, whose samples may include NAL units of a video coding elementary stream of occupancy map data (e.g., payload of a V-PCC unit of type VPCC_OVD).

[0089] A container format for point cloud data may be provided. The carriage of missing PCM point information may be supported by the container format for point cloud data. Signaling for V-PCC block groups and / or spatial access may be provided. The sample format of the V-PCC track may support PNAL units. Many file format structures may be provided to support PCM information and / or provide signaling that enables flexible access to different components, layers and / or spatial regions within a V-PCC bitstream. Tracks (e.g., only a single track) may be used to store information of layers (e.g., all layers) of a V-PCC component, for example, when the layers of the V-PCC component constitute a single video stream. A sample grouping mechanism may be used to group samples belonging to each layer.

[0090] If the layers are stored in separate tracks, track grouping tools may be used to signal that the separate tracks are for layers belonging to the same V-PCC component. For example, a track group type (e.g., VPCCComponentGroupBox or similar entity) may be defined, for example, by extending TrackGroupTypeBox. TrackGroupTypeBox may include a track_group_id field as an identifier of the group, and a track_group_type field that stores a four-character code identifying the group type. A pair of track_group_id and track_group_type may identify a track group within a container file. VPCCComponentGroupBox may be defined as follows. VPCCComponentGroupBox may be a box type of "vplg" and may be located within a TrackGroupBox container. In an example, VPCCComponentGroupBox may be optional (e.g., non-mandatory). In an example, multiple VPCCComponentGroupBox boxes may exist in a TrackGroupBox.

[0091] Table 5 shows an exemplary VPCCComponentGroupBox syntax.

[0092] Table 5—Example VPCCComponentGroupBox Syntax

[0093]

[0094] Tracks (e.g., all tracks) belonging to the layer of the same component may have a VPCCComponentGroupBox within a TrackGroupBox. In each VPCCComponentGroupBox, the value of track_group_id may be the same. A V-PCC media player may identify tracks belonging to the same V-PCC component by parsing each track in the container and / or identifying those tracks that have VPCCComponentGroupBoxes with the same track_group_id value.

[0095] To collectively reference tracks (e.g., all tracks) belonging to the same component, a track reference corresponding to a V-PCC component in a main V-PCC track may use the track_group_id of the track group of the component (e.g., to identify one or more track groups associated with the component). For example, a TrackReferenceTypeBox corresponding to a component may have an entry in its track_ID array that uses the track_group_id to identify the track group of the component.

[0096] In an example, TrackGroupTypeBox may include a tag field and a bit (e.g., bit 0 of the field, where bit 0 is the least significant bit) that may be used to indicate the uniqueness of the track_group_id. Tracks carrying geometry and / or attribute information of the same layer may be grouped. Grouping tracks carrying geometry and / or attribute information may enable a media player to perform scalable access to V-PCC content. A VPCCLayerGroupBox or similar entity may be defined and may be given a box type of "vplg". The VPCCLayerGroupBox may be located within a TrackGroupBox container. In an example, the VPCCLayerGroupBox may be optional (e.g., non-mandatory). In an example, multiple VPCCLayerGroupBox boxes may exist within a TrackGroupBox.

[0097] Table 6 shows an exemplary VPCCLayerGroupBox syntax.

[0098] Table 6—Example VPCCLayerGroupBox Syntax

[0099]

[0100]

[0101] As shown in the exemplary syntax, the VPCCLayerGroupBox field may include one or more of the following fields. The layer_index field may indicate the index of the layer to which one or more tracks of the group belong. The absolute_coding_flag field may indicate whether the geometry track in the track group depends on the geometry track in another layer. If absolute_coding_flag is set to 1, the track may not depend on another layer. If absolute_coding_flag is set to 0, the track may depend on another layer. The predictor_layer_index field may indicate the index of the layer to which the geometry track in the group depends.

[0102] A V-PCC component track may be provided. In an example (e.g., when one or more V-PCC stream components (such as occupancy, geometry, and / or attribute components) are video encoded), one or more tracks carrying information related to the V-PCC stream components (e.g., any of the occupancy, geometry, and / or attribute components) may be signaled as a restricted video scheme track. The restricted video scheme track may not be intended for direct rendering. The scheme_type field in the SchemeTypeBox may be set to 4CC (e.g., 'pccv') for components of the V-PCC content. Data associated with the V-PCC scheme may be stored in a SchemeInformationBox. For example, data associated with the V-PCC scheme may be signaled in a VPCCComponentInfoBox (or similar entity), which may be carried in the SchemeInformationBox and defined as follows.

[0103] Table 7 shows an exemplary VPCCComponentInfoBox syntax.

[0104] Table 7—Example VPCCComponentInfoBox Syntax

[0105]

[0106] As shown in the exemplary semantics, the VPCCComponentInfoBox may include one or more of the following fields. The component_type field may indicate the type of the component. For example, a value of 0 for component_type may be reserved. A value of 1 for component_type may indicate an occupancy map component. A value of 2 for component_type may indicate a geometry component. A value of 3 for component_type may indicate an attribute component. It should be noted that numbers are provided herein as examples, and other numbers may be used to indicate various component types. The is_pcm_flag field may indicate whether the information carried in the track is for a PCM point. When is_pcm_flag is set (e.g., set to a value of 1), the track may carry PCM information for the component indicated by component_type. The all_layers_present_flag field may indicate whether the track is carrying information for all layers of the component. For example, when all_layers_present_flag is set (e.g., set to a value of 1), coded data for layers (e.g., all layers) of the component may be present in the track. Otherwise (e.g., when all_layers_present_flag is not set or is set to a value of 0), the track may carry coded data for a single layer of the component. The layer_index field may indicate the index of a component layer to which data carried by the track belongs.

[0107] The SchemeInformationBox may include an additional VPCCAttributeInfoBox (or similar entity) that may provide additional description of the attribute component, for example, if the component track is carrying attribute information (eg, component_type is set to 3). The VPCCAttributeInfoBox may be defined as shown in Table 8.

[0108] Table 8—Example VPCCAttributeInfoBox Syntax

[0109]

[0110] As shown in the exemplary semantics of Table 8, the VPCCAttributeInfoBox may include one or more of the following fields. The attr_index field may indicate the index of the attribute in the attribute list. The attr_type field may indicate the attribute type of the attribute. The attr_dimensions field may indicate the number of dimensions of the attribute (e.g., the total number). The attr_first_dim_index field may indicate the index of the first attribute dimension carried by the track (e.g., a zero-based index).

[0111] VPCCAttributeInfoBox may include a partition index. Table 9 shows another exemplary VPCCAttributeInfoBox syntax.

[0112] Table 9—Example VPCCAttributeInfoBox Syntax

[0113]

[0114] As shown in the exemplary syntax of Table 9, the VPCCAttributeInfoBox may include one or more of the following fields. The attr_index field may indicate the index of the attribute in the attribute list. The attr_type field may indicate the type of the attribute. The attr_dimensions field may indicate the number of dimensions (e.g., the total number) of the attribute. The attr_dim_partition_index field may represent the index (e.g., zero-based) of the dimension partition carried by the track.

[0115] In an example, the VPCCComponentBox may carry (eg, directly carry) a vpcc_unit_header() HLS structure corresponding to the vpcc_unit_type of the component information carried by the track. If the vpcc_unit_type is VPCC_AVD, the presence of the VPCCAttributeInfoBox in the SchemeInformationBox may be optional.

[0116] Information about missing PCM points may include geometry data and / or attribute data. The PCM point information may be packaged in the video stream of the associated component and / or available as a separate video stream (e.g., one video stream for each component). The PCM point information may be carried in a separate track (e.g., one track of information related to a particular component), for example, when the PCM point information is available separately. The separate track may be signaled as a restricted video scheme track (e.g., as described herein), where the is_pcm_flag field in the VPCCComponentBox is set to 1. Each track carrying PCM point information may be included in the track group of the associated component. A track reference from the main track to the track_group_id of the V-PCC component may reference (e.g., co-reference) tracks of PCM and / or non-PCM points.

[0117] Geometry and attribute information associated with PCM may be grouped, for example, to enable easy identification and access to missing points. Track grouping may be defined using VPCCPCMTrackGroupBox (or similar entity) as shown in Table 10, for example, to identify tracks with PCM point information.

[0118] Table 10 - Example VPCCPCMTrackGroupBox Syntax

[0119]

[0120] A track carrying information related to PCM points of a particular V-PCC content may include a VPCCPCMTrackGroupBox (or similar entity) within a TrackGroupBox. In each VPCCPCMTrackGroupBox, the value of track_group_id may be the same.

[0121] A 4CC value (eg, 'pccp') may be defined for the reference_type field of the TrackReferenceTypeBox. The 4CC value may be used to signal a reference to a track carrying PCM point data (eg, including geometry and / or attribute data).

[0122] In an example, there may be no constraints on the prediction structure used to encode the various components of the V-PCC bitstream. Therefore, different components and / or different layers of the same component may be encoded (e.g., if the components are not in the same video stream) by an encoding configuration that will result in non-aligned intra-refresh periods across the various component substreams. Such non-aligned intra-refresh periods across the various component substreams may make random access challenging because intra-coded samples in a patch stream in a main V-PCC track at a given decoding time may not have corresponding intra-coded samples in other component tracks for the same decoding time. Without additional information indicating the location of synchronization samples in component tracks relative to the main track, a media player may resort to scanning the component track for the most recent synchronization sample.

[0123] The sync samples in one V-PCC component may not be aligned with the sync samples in other components. A sync sample in the master track may have a corresponding sync sample in other (e.g., all other) component tracks. For example, if the inner refresh period of the patch sequence stream is once every 30 frames, the inner refresh period of the geometry component may be once every 60 frames, and / or the inner refresh period of the texture attributes may be once every 30 frames. For example, an inner refresh frame may appear once every 30 seconds in the master track and in other (e.g., all other) components. The inner refresh frames may have the same decoding time.

[0124] Constraints on intra-coded random access point (IRAP) periods across components may be defined (e.g., to support random access) such that IRAP samples are aligned across tracks. For example, an encoder may be constrained to generate substreams with aligned synchronization samples at regular intervals. Using the constraints, a decoder and / or client may assume that an IRAP sample is available in one or more (e.g., all other) components at the same time that an IRAP sample is detected in one (e.g., any) component track. The IRAP for each component may represent the IRAP for the VPCC bitstream. Figure 4 An example of a constraint is shown where the IRAP samples of components are aligned.

[0125] The constraints described herein may eliminate the need for additional information to signal the correspondence of sync samples across tracks.When a sync sample is reached in a master track, a corresponding sync sample with the same decoding time may be found in other (eg, all other) component tracks.

[0126] In an example, the IRAP periods of different components and the main patch sequence track may be selected so that samples within the time alignment (e.g., synchronization) occur at regular intervals. When samples within the time alignment (e.g., synchronization) occur at regular intervals, a V-PCC media player accessing a synchronization sample in the main V-PCC track may find a corresponding synchronization sample of the same decoding time in other component tracks. Each component may have a different IRAP period. The IRAP period of the main V-PCC track may be the least common multiple of the IRAP periods of other (e.g., all) component tracks. The IRAP of the main V-PCC track may represent the IRAP of the V-PCC bitstream. Figure 5 An example of using the least common multiple of the IRAP periods to indicate the V-PCC IRAP is shown.

[0127] In an example, there may be no constraints on the IRAP period of the V-PCC component. The IRAP of the main V-PCC track may represent the IRAP of the V-PCC bitstream. For other components, the nearest IRAP may be located in view of the decoding and / or presentation time of the IRAP in the main V-PCC.

[0128] The V-PCC high-level syntax (HLS) may support block groups. In a video coding standard (e.g., HEVC), a 2D frame may be divided into a grid of blocks. One or more block groups may correspond to a rectangular area in a 2D frame comprising a plurality of blocks. A motion constrained block set (MCTS) may be decoded (e.g., independently decoded), and the MCTS may enable extraction of a specific area within a frame. In V-PCC, patches corresponding to points belonging to a specific area of ​​space (e.g., a 3D area or a cuboid) may be packaged in one or more MCTS. Block groups and MCTS may be used interchangeably herein.

[0129] Figure 6 An example of a V-PCC container structure 600 that can be used to implement spatial access to a specific region within a 3D space is shown. As shown, a 3D space 602 for a point cloud (e.g., a bounding box corresponding to the 3D space) can be divided into a 3D cube grid (e.g., cubes 602a, 602b, 602c, etc.) representing multiple regions and / or objects in the 3D space. Points belonging to each of the regions and / or objects in the 3D space can be clustered, and a bounding box can be used to represent the region or object. Points belonging to different parts of the same object can be grouped together and can be represented by the corresponding bounding box of the part.

[0130] The patches generated by the projection of points within each resulting bounding box may be packed together in one or more tile groups within a 2D frame of multiple V-PCC component streams or tracks (e.g., occupancy, geometry, and / or attribute streams or tracks). The patches may be encoded using a coding configuration that generates tile groups (e.g., independently decodable tile groups). These tile groups may be carried in separate tracks within an ISOBMFF container, and thus the term "tile group" may be used interchangeably with "track group" herein (e.g., a tile group may be an instance of a track group). Carrying tile groups in separate tracks may enable a decoding device (e.g., a media player) to access and / or download tracks that carry information related to specific regions or objects in 3D space. For example, when tile groups are carried in separate tracks of a V-PCC bitstream, a media player may only be able to access and / or download tracks related to specific regions of 3D space when decoding a specific region (e.g., when rendering a visual representation of the region).

[0131] Tracks with corresponding tile groups across V-PCC components (e.g., those carrying information representing points within a bounding box of a region or object) can be grouped together using track grouping tools. A TrackGroupBox ("trgr") or similar entity can be added to the TrackBox of each of these tracks, and the track grouping type for the V-PCC tile group can be defined by extending the TrackGroupTypeBox as shown below (e.g., using the track_group_id or tile_group_id field).

[0132] Table 11 shows an exemplary VPCCTileGroupBox syntax.

[0133] Table 11 - Example VPCCTileGroupBox Syntax

[0134]

[0135] As shown in the exemplary semantics of Table 11, the VPCCTileGroupBox may include a tile_group_id field (or similar field) that identifies the V-PCC tile group (e.g., as identified by it). In an example, the tile_group_id may correspond to (e.g., be the same as) a tile group address (e.g., a field such as ptgh_address, which may be included in a tile group header of the V-PCC bitstream). Tracks belonging to the same point cloud tile group may have the same value of track_group_id for track_group_type "vptg". The track_group_id of a track from one point cloud tile group may be different from the track_group_id of a track from any other point cloud tile group. For example, if Figure 6 As shown, the track group ID of the first tile group corresponding to the 3D region 602a may be 1, and the track group ID of the second tile group corresponding to the 3D region 602b may be 2. Therefore, the track_group_id within the TrackGroupTypeBox with track_group_type equal to 'vptg' (or similar 4CC value) may be used as an identifier of the point cloud tile group within the ISOBMFF container file.

[0136] For example, sample grouping may be used to signal which sample belongs to which V-PCC block group. For example, if for a V-PCC component, information related to more than one V-PCC block group is carried in a track (e.g., for a set of V-PCC block groups, a set of sample groups exists in the track, each of which is associated with a corresponding V-PCC block group), sample grouping may be used. Sample group entries may be defined (e.g., as shown in Table 12 below), where the semantics (e.g., definition) of tile_group_id may be the same as those of tile_group_id defined in VPCCTileGroupBox as provided herein. The group type may be 'vpge' or a similar 4CC value. The container may be a SampleGroupDescriptionBox ("sgpd") or a similar entity. VPCCTileGroupBox may not be required (e.g., may be optional), and each track may have multiple VPCCTileGroupBoxes (e.g., associated with them). Table 12 shows an exemplary syntax for VPCCTileGroupEntry.

[0137] Table 12 - Example VPCCTileGroupEntry Syntax

[0138]

[0139] In an example, a subtrack carrying one or more V-PCC block groups may be defined within a component track. One or more V-PCC block groups may be defined using a SubTrackSampleGroupBox (or similar entity) and a list of VPCCTileGroupEntry instances (or similar entities) corresponding to the V-PCC block groups carried in each subtrack in the corresponding SubTrackSampleGroupBox (e.g., by referencing their group_description_index). One or more V-PCC block groups may be defined by defining a V-PCC-specific VPCCTileGroupSubTrackBox (e.g., as shown in Table 13). The box type may be set to 'vpst' or a similar 4CC value. The container may be a SubTrackDefinitionBox ("strd") or similar entity. The VPCCTileGroupSubTrackBox may not be required (e.g., may be optional), and each track may have multiple VPCCTileGroupSubTrackBoxes.

[0140] Table 13 - Example VPCCTileGroupSubTrackBox Syntax

[0141]

[0142] The union (e.g., collection) of tile_group_ids in the VPCCTileGroupSubTrackBox may describe (e.g., collectively describe) the subtrack defined by the box. The semantics of the VPCCTileGroupSubTrackBox may include one or more of the following fields. The item_count field may represent a count of the number of tile groups listed in the VPCCTileGroupSubTrackBox. The tile_group_id field may represent an identifier of the V-PCC tile group included in this subtrack. The tile_group_id field in the VPCCTileGroupSubTrackBox may match (e.g., correspond to) the tile_group_id defined in the VPCCTileGroupEntry.

[0143] A mapping may be provided between a region or object in 3D space (e.g., each 3D bounding box in a 3D bounding box) and its corresponding tile group, for example, to enable a client (e.g., a media player or decoding device) to identify which tracks to access / download in order to render a particular region in 3D space (e.g., as represented by a bounding box). It should be noted that while the position of the tile group within the 2D frame may not change, the position and possibly size (e.g., dimensions) of the bounding box (e.g., region) within the 3D space may change over time, for example, due to movement of objects represented by points within the bounding box. A 3D region in a point cloud may be defined using the exemplary 3D region structure shown in Table 14.

[0144] Table 14—Example 3DRegionStruct Syntax

[0145]

[0146] As shown in the exemplary semantics of Table 14, the 3DRegionStuct may include one or more of the following fields. The region_id field may indicate a unique identifier of the 3D region. The region_x field may indicate the x-coordinate of a reference point associated with the 3D region (e.g., a bounding box associated with the region). The region_y field may indicate the y-coordinate of the reference point. The region_z field may indicate the z-coordinate of the reference point. The region_width field may indicate the length of the 3D region (e.g., a bounding box associated with the region) along the x-axis. The region_height field may indicate the length of the 3D region (e.g., a bounding box associated with the region) along the y-axis. The region_depth field may indicate the length of the 3D region (e.g., a bounding box associated with the region) along the z-axis. The dimensions_included_flag field may indicate whether the dimensions of the 3D region (e.g., a bounding box associated with the region) are signaled in the same instance of the structure. For example, if the value of dimensions_included_flag is 0, it may indicate that dimensions are not signaled, and dimensions may have been previously signaled for the same region (e.g., a previous instance of VPCC3DRegionStruct with the same region_id signaled dimensions). If the value of dimensions_included_flag is 1, it may indicate that dimensions are signaled.

[0147] A 3D region or object in a point cloud may be associated with one or more point cloud tile groups (eg, instances of a track group) using a VPCCRegionToTileGroupBox or similar entity. Table 15 shows an exemplary VPCCRegionToTileGroupBox syntax.

[0148] Table 15 - Example VPCCRegionToTileGroupBox Syntax

[0149]

[0150] As shown in the exemplary semantics of Table 15, VPCCRegionToTileGroupBox may indicate a mapping relationship between a region (or object) in a 3D space and one or more tile groups (e.g., track groups). VPCCRegionToTileGroupBox may include one or more of the following fields. The num_regions field may indicate the number of 3D regions in a point cloud associated with the 3D space. The region_id field may identify a 3D region (e.g., including its identifier). The num_tile_groups field may indicate the number of V-PCC tile groups associated with the 3D region. The tile_group_id field may identify a V-PCC tile group. Therefore, VPCCRegionToTileGroupBox may link one or more blocks to a 3D region via at least the tile_group_id field and the region_id field.

[0151] The VPCCRegionToTileGroupBox may be signaled in a sample entry of the main V-PCC track 604 or in a sample entry of a separate timed metadata track 606 associated with the main V-PCC track, such as Figure 6 As shown. A timed metadata track 606 (e.g., which may be separate from the main V-PCC track) may be included in the ISOBMFF container and may be used, for example, to update one or more characteristics (e.g., position and / or dimension) of a defined 3D region of a point cloud over time. The timed metadata track 606 may include a defined sample entry (e.g., VPCC3DRegionSampleEntry) with a 4CC of 'vp3r' (or similar 4CC value), and the defined sample entry may extend MetadataSampleEntry or a similar entity, as shown by the exemplary syntax in Table 16 (e.g., for a VPCC3DRegionInfoBox or a similar entity).

[0152] Table 16—Example VPCC3DRegionInfoBox Syntax

[0153]

[0154] As shown in the exemplary semantics of Table 16, the VPCC3DRegionInfoBox may include a num_regions field indicating the total number of 3D regions in the 3D space. The timed metadata track 606 may be linked to the main V-PCC track 604, for example, using 4CC of 'cdsc' (or similar 4CC value) as a track reference. The (e.g., each) sample in the timed metadata track may specify a 3D region, for example, using the exemplary syntax shown in Table 17 below. The derived track format may extend the VPCC3DRegionSample structure (e.g., or similar entity).

[0155] Table 17—Example VPCC3DRegionSample Syntax

[0156]

[0157] As shown in the exemplary semantics of Table 17, VPCC3DRegionSample may include a num_regions field that may indicate the number of 3D regions signaled in the sample. The number of 3D regions signaled in the sample may or may not be equal to the total number of available regions. For example, the number of 3D regions signaled in the sample may indicate the 3D regions whose characteristics (e.g., position and / or dimensions) are updated in the sample.

[0158] Patch information may be carried in a V-PCC track. A VPCCDecoderConfigurationRecord (or similar entity) and sample format syntax for a V-PCC track may be formatted as follows, for example, to support the carrying of a patch information substream structured as a sequence of patch network abstraction layer (PNAL) units. The VPCCDecoderConfigurationRecord may provide configuration information to a decoder (e.g., at the beginning of a decoding process). The VPCCDecoderConfigurationRecord may include one or more parameter sets and / or one or more supplemental enhancement information (SEI) messages. The VPCCDecoderConfigurationRecord may include a lengthSizeMinusOne field. An exemplary VPCCDecoderConfigurationRecord syntax may be shown in Table 18 below.

[0159] Table 18 - Example VPCCDecoderConfigurationRecord Syntax

[0160]

[0161] As shown in the exemplary semantics of Table 18, the VPCCDecoderConfigurationRecord may include a configurationVersion field indicating the current version of the configuration record. In an example, an incompatible change to the decoder configuration record may be indicated by a change in the configuration version number. The decoding device may be configured to not attempt to decode the configuration record or the stream to which it applies if the configuration version number is not recognized. The VPCCDecoderConfigurationRecord may include a lengthSizeMinusOne field, and the value of lengthSizeMinusOne plus 1 may indicate the length (e.g., in bytes) of the PNALUnitLength field in a V-PCC sample (e.g., in the stream to which the configuration record is applied). For example, a PNALUnitLength field length of one byte may be indicated by a lengthSizeMinusOne value of 0. The value of the lengthSizeMinusOne field may be 0, 1, or 3, which may correspond to a length (e.g., PNALUnitLength) encoded by 1, 2, or 4 bytes, respectively.

[0162] In an example, a decoder configuration record may include one or more setting unit arrays, such as a first setting unit array for a V-PCC parameter set (e.g., a Vsequence parameter set) and a second setting unit array for other setting units of a patch information substream. Table 19 below shows an example indicating one or more setting unit arrays.

[0163] Table 19 - Example VPCCDecoderConfigurationRecord Syntax

[0164]

[0165] As shown in the exemplary semantics of Table 19, the VPCCDecoderConfigurationRecord (or similar entity) may include one or more of the following fields. The configurationVersion field (or similarly named field) may indicate the current version of the configuration record. In an example, an incompatible change to the decoder configuration record may be indicated by a change in the configuration version number. The decoding device may be configured to not attempt to decode the configuration record or the stream to which it applies if the configuration version number is not recognized. The numOfSequenceParameterSets field (or similarly named field) may indicate the number of V-PCC parameter sets (e.g., arrays) signaled (e.g., defined) in the decoder configuration record (e.g., for the stream to which the decoder configuration record applies). The numOfSetupUnitArrays field may indicate the number of arrays of PNAL units of an indicated type (e.g., as indicated by PNAL_unit_type) signaled (e.g., defined) in the decoder configuration record (e.g., for the stream to which the decoder configuration record applies). The array_completeness field may indicate whether all PNAL units are included in the array. For example, when the array_completeness field is equal to 1, it may indicate that PNAL units of a given type (e.g., all PNAL units) are included in the following array (e.g., none are in the stream). When the array_completeness field is equal to 0, it may indicate that additional PNAL units of the indicated type may be in the stream. The default value and / or permitted values ​​of array_completeness may be limited by the sample entry name or sample entry type of the corresponding sample entry. For example, a VPCCDecoderConfigurationRecord may be used in different sample entries. The container of a VPCCDecoderConfigurationRecord may be a VPCCDecoderConfigurationBox (or similar entity), which may be a box contained within a VPCCSampleEntry. A VPCCSampleEntry may have different types, and the type of the sample entry may set constraints on the permitted values ​​and / or default values ​​of the array_completeness field in the encapsulated VPCCDecoderConfigurationRecord.

[0166] The VPCCDecoderConfigurationRecord (or similar entity) may include a PNAL_unit_type field indicating the type of the PNAL units in the following array (e.g., all PNAL units in the array may have the indicated type). The PNAL_unit_type field may have (e.g., be restricted to take) one of the following values ​​indicating a PUP_PSPS, PUP_PREFIX_SEI, or PUP_SUFFIX_SEIP NAL unit. The VPCCDecoderConfigurationRecord (or similar entity) may include a numPNALUnits field that indicates the number of PNAL units of the indicated type included in the configuration record (e.g., for the stream to which the configuration record applies). The supplemental enhancement information (SEI) array may include (e.g., only include) declarative SEI messages. Declarative SEI messages may include SEI messages that indicate information about the entire stream. For example, a user data SEI may be a declarative SEI message.

[0167] The VPCCDecoderConfigurationRecord (or similar entity) may include a pnalUnitLength field indicating the length of the PNAL unit (eg, in bytes). The VPCCDecoderConfigurationRecord (or similar entity) may include a pnalUnit field that may be used to hold a PUP_PSPS or a declarative SEIP NAL unit.

[0168] Based on the exemplary VPCCDecoderConfigurationRecord syntax shown herein, the sample format of a sample in a V-PCC track (eg, denoted as VPCCSample) may be shown in Table 20 below.

[0169] Table 20—Example VPCCSample Syntax

[0170]

[0171]

[0172] As shown in the exemplary semantics of Table 20, the VPCCDecoderConfigurationRecord field may indicate a decoder configuration record in the corresponding V-PCC sample entry. The PNALUnitLength field may indicate the size of the PNAL unit (e.g., measured in bytes). In an example, the PNALUnitLength field may include the size of both the PNAL unit header and the PNAL unit payload. In an example, the PNALUnitLength field may not include the size of the PNALUnitLength field itself. In addition, the PNALUnit field may be included to represent a PNAL unit (e.g., a single atlas NAL unit).

[0173] In an example, patch information in samples of a V-PCC track may be formatted based on a patch information sample stream (eg, an atlas sample stream). VPCCDecoderConfigurationRecord (or similar entity) may include a lengthSizeMinusOne field. Table 21 shows another exemplary VPCCDecoderConfigurationRecord syntax.

[0174] Table 21 - Example VPCCDecoderConfigurationRecord Syntax

[0175]

[0176] The fields (e.g., variables) in the exemplary syntax of Table 21 may be defined similarly to those in Table 19. For example, the value of lengthSizeMinusOne plus 1 may indicate the length (e.g., in bytes) of the PNALUnitLength field (e.g., in V-PCC samples in the stream to which the configuration record is applied). Thus, the size of one byte of the PNALUnitLength field may be indicated by the lengthSizeMinusOne field having a value of 0. In the exemplary syntax of Table 21, the lengthSizeMinusOne field may be defined as an unsigned int(3), and thus the value of the lengthSizeMinusOne field may be in the range of 0 to 7.

[0177] A point cloud (e.g., a sequence of point clouds) may represent a scene with multiple objects or a single object with multiple parts. Multiple such objects or parts may be carried in a track (e.g., a V-PCC track). It may be desirable to access (e.g., stream and / or render) a subset of these objects or parts without having to decode other objects or parts of the scene (e.g., without decoding the entire point cloud). The signaling of the point cloud may be configured to support partial access to a 3D point cloud (e.g., access some objects and / or parts without accessing other objects and / or parts). For example, a V-PCC message such as a V-PCC volume tiling SEI message may be used to signal and / or render the 2D position of a patch belonging to an object in a point cloud and / or the corresponding 3D bounding box of the object in 3D space. In this way, a V-PCC decoder (e.g., a streaming client) may not need to decode the entire atlas frame in order to render the target object. The V-PCC decoder may render the target object by downloading and / or decoding only those parts of the atlas frame associated with the target object.

[0178] Objects and / or portions of the 3D point cloud may be independently decodable (e.g., decodable without decoding other objects or portions of the 3D point cloud). In an example, patches belonging to each of these independently decodable objects or portions of the 3D point cloud may be assigned to corresponding tile groups in a corresponding atlas frame. A streaming client or player may download the tile groups (e.g., only the tile groups) associated with a target object or portion of the 3D point cloud in order to render the target object or portion of the 3D point cloud.

[0179] V-PCC block group sample groups may be provided to support the functionality described herein. One or more block groups associated with an atlas substream may be carried in the same V-PCC track (e.g., for non-streaming applications). ISOBMFF sample group entries may be used to group samples (e.g., all samples) of NAL units carrying block groups (e.g., to enable random access to block groups). As an example, a block group sample group entry (e.g., defined as VPCCTileGroupSampleGroupEntry) may have the following attributes. The group type of VPCCTileGroupSampleGroupEntry may be set to a four-character code of 'vtgi' or another suitable value. The container for VPCCTileGroupSampleGroupEntry may be a sample group description box with a four-character code of 'sgpd' or another suitable value. VPCCTileGroupSampleGroupEntry may not be mandatory, and there may be zero or more instances of VPCCTileGroupSampleGroupEntry.

[0180] VPCCTileGroupSampleGroupEntry may describe a block group. A group type may not be defined for this group (eg, there may be no grouping_type_parameter element for VPCCTileGroupSampleGroupEntry or for SampleToGroupBox with a grouping type of 'vtgi'). The syntax of VPCCTileGroupSampleGroupEntry may be defined as shown in Table 22.

[0181] Table 22 - Example VPCCTileGroupSampleGroupEntry Syntax

[0182]

[0183] As shown in Table 22, VPCCTileGroupSampleGroupEntry may include one or more of the following fields. The groupID field may be a unique identifier for the block group area described by the sample group entry. The value of groupID may be greater than 0 (e.g., the value 0 may be reserved for special purposes). The full_frame_flag field may be set or not set. When set (e.g., when having a non-zero value), the field may indicate that the block group associated with this block group sample group entry represents a complete atlas frame. Otherwise (e.g., when the value of the full_frame_flag field is zero), the field may indicate that the block group associated with this block group sample group entry is not a complete atlas frame. When the block group represents a complete atlas frame, two other fields (e.g., tile_group_width and tile_group_height) may be set to the width and height of the complete atlas frame, respectively. For example, the tile_group_width and tile_group_height fields may indicate (e.g., respectively) the width and height of a rectangular area covered by tiles in each tile group associated with this tile group sample group entry. When the tile group does not represent a complete atlas frame, two fields (e.g., horizontal_offset and vertical_offset) may indicate (e.g., respectively) the horizontal offset and vertical offset of the upper left pixel of the rectangular area covered by tiles in each tile group associated with this tile group sample group entry. The horizontal offset and vertical offset may be indicated, for example, relative to the upper left of the atlas frame.

[0184] The VPCCTileGroupSampleGroupEntry may include a has_dependency_list field. The has_dependency_list field, when set to 1, may indicate the presence of another field indicating the number of tile groups that each tile group associated with this tile group sample group entry may depend on, e.g., dependency_tile_group_count. For example, when the value of dependency_tile_group_count is greater than 0, it may indicate that a list of dependencyVPCCTileGroupSampleGroupIDs may be included in the tile group sample group entry. Each dependencyVPCCTileGroupSampleGroupID may indicate an identifier (e.g., as defined by the VPCCTileGroupSampleGroupEntry record) of a tile group sample group that the present tile group region (e.g., as defined by the groupID field) may depend on.

[0185] When the has_dependency_list field is set to 0, the dependency_tile_group_count field may not be included in the tile group sample group entry, and the list of dependencyVPCCTileGroupSampleGroupIDs may not exist in the tile group sample group entry.

[0186] A V-PCC block group track may be provided, for example to carry NAL units of one or more V-PCC block groups. Thus, the V-PCC block group track may enable direct access to one or more V-PCC block groups. The V-PCC block group track may be identified by a VPCCTileGroupSampleEntry sample description. A track reference such as 'tbas' may be used to link the V-PCC block group track to the main V-PCC track. For example, the main V-PCC track may include a sample entry type of 'vpc1' or 'vpcg'. The sample entry type of the V-PCC block group track may be 'vpt1' or another suitable value.

[0187] In an example (e.g., when an atlas frame is carried by samples in a V-PCC track and many V-PCC chunk group tracks), NAL units (e.g., all NAL units) applicable to the entire frame (e.g., of type VPCC_VPS) may be carried in the V-PCC track. The SampleDescriptionBox of the V-PCC chunk group track may not carry NAL units applicable to the complete atlas frame. NAL units that are not applicable to a chunk group may not be carried in the chunk group track that includes the chunk group.

[0188] The above VPCCTileGroupSampleEntry may be defined as follows. The sample entry type of the VPCCTileGroupSampleEntry may be set to a four-character code of 'vpt1' or another suitable value. The container for the VPCCTileGroupSampleEntry may be a sample description box with a four-character code of 'stsd' or another suitable value. The VPCCTileGroupSampleEntry may not be mandatory, and there may be zero or more instances of the VPCCTileGroupSampleEntry. Table 24 shows an exemplary VPCCTileGroupSampleEntry syntax.

[0189] Table 24 - Example VPCCTileGroupSampleEntry syntax.

[0190]

[0191] VPCCTileGroupSampleEntry may describe a media sample of a V-PCC chunk group track. VPCCTileGroupSampleEntry may not include a VPCCConfigurationBox (eg, a VPCCConfigurationBox may be included in a sample description of a main V-PCC track). VPCCTileGroupSampleEntry may include one or more other boxes (eg, optional boxes).

[0192] VPCCTileGroupSampleEntry may include one or more of the following fields. VPCCTileGroupSampleEntry may include a compressor_name field in the base class VolumetricSampleEntry, and this field may indicate the name of the compressor used. An exemplary value for the compressor_name field may be "\013VPCC Encoding". The first byte of the value (e.g., "\013") may indicate a count of the remaining bytes. In an example, an octal value of "13" may correspond to a decimal value of "11" and may indicate that there are 11 bytes in the remainder of the string.

[0193] When NAL units are not included in a sample in a main V-PCC track (e.g., all NAL units are in a V-PCC chunk group track), the size of the sample may be 0 and the sample may demonstrate (e.g., indicate) the timing (e.g., presentation time) of the sample and / or other characteristics of the sample (e.g., based on characteristics of the sample group with which the sample is associated). The V-PCC chunk group track may be in the same track group as a V-PCC component track belonging to an object or 3D region associated with the V-PCC chunk group. The track group may be indicated by, for example, a SpatialRegionGroupBox track group record.

[0194] For example, a V-PCC broadcast track group can be provided in ISOBMFF to indicate that V-PCC component tracks belonging to the same broadcast track group can be played together. For example, V-PCC components can be encoded into multiple versions to adapt to various receiving devices and / or network conditions. Multiple coded versions can correspond to the corresponding occupancy, geometry, attributes and / or other characteristics (e.g., such as resolution and / or bit rate) of the V-PCC component, and therefore multiple coded versions can be used to meet the needs of devices or networks with different conditions or capabilities (e.g., different processing capabilities, different display capabilities, etc.). Different coded versions of the V-PCC component can be stored in (e.g., carried by) the corresponding V-PCC component tracks that can be grouped together (e.g., in an alternative track group). For example, a V-PCC component track belonging to the same alternative group (e.g., having the same alternate_group value) can indicate that the V-PCC component tracks in the group represent different coded versions of the same V-PCC component. At a given time, one track (e.g., only one track) in the alternative track group can be played or streamed. This one track can be, for example, an occupied graph track, a geometry track, or a track associated with a specific attribute.

[0195] For example, an encoded version of a first V-PCC component may be considered compatible with (e.g., having a dependency on) an encoded version of a second V-PCC component because the encoded versions of the first V-PCC component and the second V-PCC component share the same geometry, occupancy, and / or properties (e.g., the encoded version of the first V-PCC component may be encoded in a substantially similar manner to the encoded version of the second V-PCC component). Thus, these compatible versions of the V-PCC components may be played together (e.g., undesirable artifacts may result if incompatible versions of one or more V-PCC components are played or combined together). V-PCC tracks that may be combined or played together may be indicated, for example, via a playout track group. A playout track group may be defined using a track group type of PlayoutTrackGroupBox that extends TrackGroupTypeBox, as shown in Table 25 (e.g., where track_group_id inherits from TrackGroupTypeBox).

[0196] Table 25—Example PlayoutTrackGroupBox Syntax

[0197]

[0198] As an example of indicating compatibility or grouping of V-PCC component tracks that are part of a playout group, a track group box, such as TrackGroupBox, having a four-character code of 'trgr' may be added to the track box (e.g., TrackBox) of each V-PCC component track in the V-PCC component track. And for each playout group of which the V-PCC component track is a member, a corresponding instance of PlayoutTrackGroupBox having a unique track_group_id identifying the PlayoutTrackGroupBox may be included in the track group box (e.g., TrackGroupBox) of the V-PCC component track. Thus, a V-PCC component track may be part of more than one playout group (e.g., as indicated by a V-PCC component track being associated with multiple instances of PlayoutTrackGroupBox having corresponding track_group_id), and multiple V-PCC component tracks may belong to the same playout group (e.g., as indicated by multiple V-PCC component tracks sharing the track_group_id of the same playout group).

[0199] Although features and elements are described above in specific combinations, it will be understood by those of ordinary skill in the art that each feature or element may be used alone or in any combination with other features and elements. In addition, the methods described herein may be implemented in a computer program, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted via a wired or wireless connection) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media (such as built-in hard disks and removable disks), magneto-optical media, and optical media (such as CD-ROM disks and digital versatile disks (DVDs)). A processor associated with software may be used to implement a radio frequency transceiver for a WTRU, UE, terminal, base station, RNC, or any host computer.

Claims

1. A video decoding device, comprising: A processor, the processor being configured to: Receiving a media container file, wherein the media container file indicates an atlas track associated with an atlas frame, the atlas track comprising information about one or more tiles of the atlas frame; Based on the media container file, obtain information of the one or more blocks of the atlas frame included in the atlas track; as well as At least a portion of the atlas frame is decoded based on the obtained information.

2. A method for video decoding, the method comprising: Receiving a media container file, wherein the media container file indicates an atlas track associated with an atlas frame, the atlas track comprising information about one or more tiles of the atlas frame; Based on the media container file, obtain information of the one or more blocks of the atlas frame included in the atlas track; as well as At least a portion of the atlas frame is decoded based on the obtained information.

Images