Dynamic adaptation of volumetric content component qbitstreams in streaming services

Abstract

A media content processing device can decode visual volumetric content based on one or more messages that can indicate which of one or more attribute sub-bitstreams indicated in a parameter set is active. The parameter set can comprise a parameter set based on visual volumetric video. The messages indicating one or more active attribute sub-bitstreams can be received by a decoder. The decoder can perform decoding based on the one or more messages, such as determining which attribute sub-bitstream to use to decode visual media content. The one or more messages can be generated and sent to a decoder, for example, to indicate deactivation of the one or more attribute sub-bitstreams. The decoder can determine, based on the one or more messages, an inactive attribute sub-bitstream and skip that inactive attribute sub-bitstream to decode the visual media content.

Description

[0001] This application is a divisional application of patent application No. 202080049240.4, filed on July 2, 2020, entitled "Dynamic Adaptation of Volume Content Component Quantum Bit Streams in Streaming Services".

[0002] Cross-reference of related applications This application claims priority to U.S. Provisional Patent Application 62 / 869,705, filed July 2, 2019, entitled "Dynamic Adaptation of Point Cloud Component Substreams in Point Cloud Streaming Services," the entire contents of which are incorporated herein by reference as if fully set forth herein. Background Technology

[0003] Video decoding systems can be used to compress digital video signals, for example, to reduce the storage and / or transmission bandwidth required for such signals. Video decoding systems can include, for example, wavelet-based systems, object-based systems, and / or block-based systems, such as block-based hybrid video decoding systems. Summary of the Invention

[0004] Systems, methods, and means for dynamically adapting volumetric content, such as point cloud component sub-bitstreams in a point cloud streaming service, are disclosed. Dynamic adaptation of volumetric content can be based on one or more messages and / or parameter sets that indicate one or more changes to the volumetric content. Changes may include, for example, changes to active attributes and / or codecs. For example, a bitrate adaptation process in a streaming session may (e.g., based on operating environment factors such as resource availability, bandwidth attributes, client decoding capabilities, and / or client rendering capabilities) add / drop one or more attributes of video components and / or change the representation of video components to a representation encoded using a different codec. One or more messages and / or parameter sets may be generated and sent to a decoder, for example, to indicate volumetric content processing information, which may include change indications. The decoder may perform decoding based on one or more messages and / or parameter sets, for example, determining which attribute sub-bitstream and / or codec to use for decoding.

[0005] The one or more messages and / or parameter sets may include, for example, Component Codec Change (CCC) messages, Active Attribute (AA) messages, Component Change Parameter Set (CCPS) and / or Parameter Set Activation (PSA) messages. For example, Visual Volumetric Content Bitstream Supplement Enhancement Information (SEI) messages and / or parameter sets may support adapted streaming of visual volumetric content. Messages (e.g., CCC SEI messages) may inform a decoder (e.g., a visual volumetric content decoder) of codec changes to one or more visual volumetric content components, enabling the decoder to determine which video codecs are used for the referenced component. Messages (e.g., AA SEI messages) may inform a decoder (e.g., a visual volumetric content decoder) of attribute changes to one or more visual volumetric content components, enabling the decoder to determine, for the referenced component, which active attributes are used and which inactive attributes are ignored. Parameter sets (e.g., CCPS) may include information about changes (e.g., attributes or codecs) made to one or more visual volumetric content components relative to a parameter set (e.g., a Sequence Parameter Set (SPS)), enabling the decoder to determine which video codecs and active attributes are used for the referenced component. Messages (e.g., PSA SEI messages) can indicate which parameter sets are active for the visual volumetric content component, enabling the decoder to determine which video codecs and active attributes are used for the referenced component. The decoder can, for example, determine, based on the messages and / or parameter sets, which of the multiple attribute sub-bitstreams indicated in the parameter sets associated with the visual volumetric content is active, and use the active attribute sub-bitstream to decode the visual volumetric content.

[0006] In the example, a message may indicate a set of attribute sub-bitstreams that are active (e.g., after the message is received) for use in the decoding process of the visual volumetric content bitstream. One or more messages and / or parameter sets may signal the activation of a subset of attribute sub-bitstreams in the visual volumetric content bitstream. One or more messages and / or parameter sets may signal changes to the codec of the visual volumetric content sub-bitstreams. The visual volumetric content decoder may be configured to perform, for example, one or more of the following: obtaining an indication of whether at least one attribute signaled in a referenced parameter set is inactive; for example, if the indication indicates that at least one attribute in the referenced parameter set is inactive, obtaining an active attribute indication (e.g., the number of active attributes and their corresponding attribute indices); for example, identifying inactive attributes based on the active attribute indication; or skipping the inactive attributes in the referenced parameter set during decoding.

[0007] In the example, a method can be implemented to perform dynamic adaptation of point cloud component sub-bitstreams in a point cloud streaming service. The method may include determining to deactivate an attribute sub-bitstream among a plurality of attribute sub-bitstreams indicated by a parameter set associated with visual volumetric content, and generating a message as described herein, for example, to indicate the deactivation of the attribute sub-bitstream. This method may be implemented, for example, by means of a visual media content processing or decoding apparatus. The visual media content processing or decoding apparatus may include a DASH client or a streaming client, such as a video-based point cloud compression (VPCC) client.

[0008] A method for decoding visual media content may include: for example, determining which attribute sub-bitstream to use to decode the visual volumetric content based on a message indicating which attribute sub-bitstream among a plurality of attribute sub-bitstreams indicated in a parameter set associated with the visual volumetric content is active; and using the active attribute sub-bitstream to decode the visual volumetric content based on the message. The attributes indicated in the parameter set may characterize the visual media content.

[0009] A method for processing visual media content may include, for example, determining to deactivate one of a plurality of attribute sub-bitstreams indicated in a parameter set associated with visual volumetric content; and generating a message indicating the deactivation of the attribute sub-bitstream.

[0010] A method for decoding visual media content may include, for example, obtaining a set of parameters associated with visual volumetric content; receiving a message indicating which of a plurality of attribute sub-bitstreams indicated in the parameter set is active; determining, based on the message, active and inactive attribute sub-bitstreams; and decoding the visual volumetric content using the active attribute sub-bitstream and skipping the inactive attribute sub-bitstreams.

[0011] The message may be signaled within the bitstream. The message may include a Supplemental Enhancement Information (SEI) message. This message may have a duration that extends until the end of the bitstream. The message may also have a duration that continues until another message different from this message is received. The message may include an indicator indicating the number of active attribute sub-bitstreams among the plurality of attribute sub-bitstreams indicated in the parameter set associated with the visual volumetric content.

[0012] The parameter set may include a visual volume parameter set, which includes attribute information. The message may refer to one of the attribute information in the active attribute sub-bitstream. The parameter set may indicate multiple attributes. The message may include an indicator indicating that the multiple attribute sub-bitstreams indicated in the parameter set are active.

[0013] The parameter set may indicate mapping information associated with corresponding attribute sub-bitstreams of the plurality of attributes. The message may indicate which mapping information is active, for example, by indicating which of the plurality of attribute sub-bitstreams indicated in the parameter set is active. The visual volumetric content can be decoded using active mapping information (e.g., mapping information associated with the active attribute sub-bitstream).

[0014] The plurality of attribute sub-bitstreams may indicate, for example, texture information, material information, transparency information, and / or reflectivity information associated with the visual volumetric content. Inactive attribute sub-bitstreams can be identified based on the message. These inactive attribute sub-bitstreams can be skipped during the decoding of the visual volumetric content. Attribute sub-bitstreams indicated in the parameter set but not in the message can be identified as inactive attribute sub-bitstreams. These inactive attribute sub-bitstreams can be skipped during the decoding of the visual volumetric content.

[0015] Deactivation of an attribute sub-bitstream can be indicated by a message, for example, by not referencing an indicator associated with the attribute sub-bitstream indicated in the parameter set. Deactivation of an attribute sub-bitstream can be determined, for example, based on bitrate adaptation.

[0016] One or more methods described herein can be implemented by an apparatus including one or more processors configured to execute computer-executable instructions, which may be stored on a computer-readable medium or a computer program product, and which, when executed by the one or more processors, perform the one or more methods. Therefore, the apparatus may include one or more processors configured to perform one or more methods. The computer-readable medium or the computer program product may include instructions that cause one or more processors to perform one or more methods by executing the instructions. The computer-readable medium may contain data content generated according to the one or more methods. Signals may include messages according to the one or more methods. Devices may include apparatuses such as visual media content processing or decoding apparatuses. Devices may include televisions, cellular phones, tablet computers, or set-top boxes. A device may include at least one of: (i) an antenna configured to receive a signal including data representing an image, (ii) a band limiter configured to limit the received signal to a band including the data representing the image, or (iii) a display configured to display the image.

[0017] Every feature disclosed anywhere in this document is described and may be implemented separately / individually and in any combination with any other feature disclosed herein and / or with any feature disclosed elsewhere that may be implicitly or explicitly referenced herein or may otherwise fall within the scope of the subject matter disclosed herein. Attached Figure Description

[0018] Figure 1A This is a system schematic diagram illustrating an exemplary communication system that can implement one or more of the disclosed embodiments.

[0019] Figure 1B It is shown that, according to the embodiment, it is possible to Figure 1A The diagram shows an exemplary wireless transmit / receive unit (WTRU) used within the communication system.

[0020] Figure 1C It is shown that, according to the embodiment, it is possible to Figure 1A The diagram shows an exemplary radio access network (RAN) and an exemplary core network (CN) used within the communication system.

[0021] Figure 1D It is shown that, according to the embodiment, it is possible to Figure 1A The diagram shows another exemplary RAN and another exemplary CN used within the communication system.

[0022] Figure 2 This is a schematic diagram showing an example video encoder.

[0023] Figure 3 This is a schematic diagram illustrating an example of a video decoder.

[0024] Figure 4 This is a schematic diagram illustrating an example of a system in which various aspects and examples can be implemented.

[0025] Figure 5 An example of a bitstream structure for video-based point cloud compression (V-PCC) is shown.

[0026] Figure 6 An example Media Presentation Description (MPD) hierarchical data model is shown.

[0027] Figure 7 An example of a method for processing visual volumetric content based on message transmission (e.g., the messages shown in Table 5) is illustrated.

[0028] Figure 8 An example of a method for processing visual volumetric content based on message transmission (e.g., the messages shown in Table 5) is illustrated. Detailed Implementation

[0029] A detailed description of illustrative embodiments will now be described with reference to the accompanying drawings. Although detailed examples of possible implementations are provided in this specification, it should be noted that these details are intended to be exemplary and are in no way intended to limit the scope of this application.

[0030] Figure 1A This is a schematic diagram illustrating an exemplary communication system 100 that can implement one or more of the disclosed embodiments. The communication system 100 can be a multiple access system providing content such as voice, data, video, messaging, and broadcasting to multiple wireless users. The communication system 100 enables multiple wireless users to access such content by sharing system resources, including wireless bandwidth. For example, the communication system 100 can use one or more channel access methods, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Frequency Division Multiple Access (FDMA), Orthogonal FDMA (OFDMA), Single Carrier FDMA (SC-FDMA), Zero-Tail Unique Word DFT-Extended OFDM (ZT UW DTS-s OFDM), Unique Word OFDM (UW-OFDM), Resource Block Filtering OFDM, and Filter Bank Multicarrier (FBMC), etc.

[0031] like Figure 1AAs shown, the communication system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, RAN 104 / 113, CN 106 / 115, public switched telephone network (PSTN) 108, Internet 110, and other networks 112. However, it should be understood that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network components. Each of WTRUs 102a, 102b, 102c, and 102d can be any type of device configured to operate and / or communicate in a wireless environment. For example, any one of WTRU102a, 102b, 102c, and 102d can be referred to as a "station" and / or "STA," and can be configured to transmit and / or receive wireless signals. It can include user equipment (UE), mobile stations, fixed or mobile subscriber units, subscription-based units, pagers, cellular phones, personal digital assistants (PDAs), smartphones, laptops, netbooks, personal computers, wireless sensors, hotspots or Mi-Fi devices, Internet of Things (IoT) devices, watches or other wearable devices, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in industrial and / or automated processing chain environments), consumer electronics devices, and devices operating on commercial and / or industrial wireless networks, etc. Any one of WTRU102a, 102b, 102c, and 102d can be interchangeably referred to as a UE.

[0032] The communication system 100 may further include base station 114a and / or base station 114b. Each of base stations 114a and 114b may be any type of device configured to enable access to one or more communication networks (e.g., CN106 / 115, Internet 110, and / or other network 112) by wirelessly interfacing with at least one of WTRUs 102a, 102b, 102c, and 102d. For example, base stations 114a and 114b may be base transceiver stations (BTS), node B, e-node B, home node B, home e-node B, gNB, new radio (NR) node B, site controller, access point (AP), and wireless router, etc. Although each of base stations 114a and 114b is described as a single component, it should be understood that base stations 114a and 114b may include any number of interconnected base stations and / or network components.

[0033] Base station 114a may be part of RAN 104 / 113, and the RAN may also include other base stations and / or network components (not shown), such as base station controllers (BSCs), radio network controllers (RNCs), relay nodes, etc. Base station 114a and / or base station 114b may be configured to transmit and / or receive radio signals on one or more carrier frequencies called cells (not shown). These frequencies may be in licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide radio service coverage for a specific geographic area that is relatively fixed or may change over time. A cell may be further divided into cell sectors. For example, a cell associated with base station 114a may be divided into three sectors. Thus, in one embodiment, base station 114a may include three transceivers, i.e., each transceiver corresponds to one sector of the cell. In embodiments, base station 114a may use multiple-input multiple-output (MIMO) technology and may use multiple transceivers for each sector of the cell. For example, by using beamforming, signals can be transmitted and / or received in a desired spatial direction.

[0034] Base stations 114a and 114b can communicate with one or more of WTRUs 102a, 102b, 102c, and 102d via air interface 116, wherein the air interface can be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, millimeter wave, infrared (IR), ultraviolet (UV), visible light, etc.). Air interface 116 can be established using any suitable radio access technology (RAT).

[0035] More specifically, as described above, the communication system 100 can be a multiple access system and can use one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, and SC-FDMA, etc. For example, base stations 114a and WTRUs 102a, 102b, and 102c in RAN104 / 113 can implement a certain radio technology, such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), wherein the technology can use Wideband CDMA (WCDMA) to establish air interfaces 115 / 116 / 117. WCDMA may include communication protocols such as High-Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA may include High-Speed ​​Downlink (DL) Packet Access (HSDPA) and / or High-Speed ​​UL Packet Access (HSUPA).

[0036] In an embodiment, base station 114a and WTRUs 102a, 102b, 102c may implement a certain radio technology, such as Evolved UMTS Terrestrial Radio Access (E-UTRA), wherein the technology may use Long Term Evolution (LTE) and / or Advanced LTE (LTE-A) and / or Advanced LTE Pro (LTE-A Pro) to establish air interface 116.

[0037] In the embodiments, base station 114a and WTRUs 102a, 102b, 102c may implement a radio technology that can establish an air interface 116 using a new radio (NR), such as NR radio access.

[0038] In the embodiments, base station 114a and WTRUs 102a, 102b, and 102c can implement various radio access technologies. For example, base station 114a and WTRUs 102a, 102b, and 102c can jointly implement LTE radio access and NR radio access (e.g., using the dual connectivity (DC) principle). Therefore, the air interface used by WTRUs 102a, 102b, and 102c can be characterized by various types of radio access technologies and / or transmissions sent to / from various types of base stations (e.g., eNBs and gNBs).

[0039] In other embodiments, base stations 114a and WTRUs 102a, 102b, 102c may implement the following radio technologies, such as IEEE 802.11 (i.e., WiFi), IEEE 802.16 (i.e., WiMAX), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Provisional Standard 2000 (IS-2000), Provisional Standard 95 (IS-95), Provisional Standard 856 (IS-856), Global System for Mobile Communications (GSM), Enhanced Data Rate for GSM Evolution (EDGE), and GSM EDGE (GERAN), etc.

[0040] Figure 1ABase station 114b can be, for example, a wireless router, home node B, home e node B, or access point, and can use any suitable RAT to facilitate wireless connectivity in a local area, such as a business premises, residence, vehicle, campus, industrial facility, air corridor (e.g., for use by drones), and road, etc. In one embodiment, base station 114b and WTRU102c, 102d can establish a wireless local area network (WLAN) by implementing a radio technology such as IEEE 802.11. In another embodiment, base station 114b and WTRU102c, 102d can establish a wireless personal area network (WPAN) by implementing a radio technology such as IEEE 802.15. In yet another embodiment, base station 114b and WTRU102c, 102d can establish a picocell or femtocell by using a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.). Figure 1A As shown, base station 114b can be directly connected to the Internet 110. Therefore, base station 114b does not need to access the Internet 110 via CN106 / 115.

[0041] RAN104 / 113 can communicate with CN106 / 115, which can be any type of network configured to provide voice, data, application, and / or Voice over Internet Protocol (VoIP) services to one or more of WTRU102a, 102b, 102c, and 102d. The data can have different Quality of Service (QoS) requirements, such as different throughput requirements, latency requirements, fault tolerance requirements, reliability requirements, data throughput requirements, and mobility requirements, etc. CN106 / 115 can provide call control, billing services, location-based services, prepaid calling, Internet connectivity, video distribution, etc., and / or can perform advanced security functions such as user authentication. Although in Figure 1A While not shown, it should be understood that RAN104 / 113 and / or CN106 / 115 can communicate directly or indirectly with other RANs that use the same RAT or a different RAT as RAN104 / 113. For example, in addition to connecting to RAN104 / 113 which uses NR radio technology, CN106 / 115 can also communicate with other RANs (not shown) that use GSM, UMTS, CDMA 2000, WiMAX, E-UTRA, or WiFi radio technologies.

[0042] CN106 / 115 can also act as a gateway for WTRU102a, 102b, 102c, 102d to access PSTN108, the Internet 110, and / or other networks 112. PSTN108 may include a circuit-switched telephone network providing Simple Old-Style Telephone Service (POTS). The Internet 110 may include a global interconnected computer network equipment system using common communication protocols (e.g., TCP, UDP, and / or IP from the Transmission Control Protocol / Internet Protocol (TCP / IP) suite). Network 112 may include wired or wireless communication networks owned and / or operated by other service providers. For example, network 112 may include another CN connected to one or more RANs, wherein the one or more RANs may use the same RAT or a different RAT as RAN104 / 113.

[0043] Some or all of the WTRUs 102a, 102b, 102c, and 102d in the communication system 100 may include multi-mode capability (e.g., WTRUs 102a, 102b, 102c, and 102d may include multiple transceivers communicating with different wireless networks on different wireless links). For example, Figure 1A The WTRU102c shown can be configured to communicate with base station 114a using cellular-based radio technology, and with base station 114b using IEEE 802 radio technology.

[0044] Figure 1B This is a system schematic diagram illustrating an exemplary WTRU102. (See attached diagram.) Figure 1B As shown, WTRU102 may include a processor 118, a transceiver 120, a transmitter / receiver unit 122, a speaker / microphone 124, a numeric keypad 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power supply 134, a Global Positioning System (GPS) chipset 136, and / or peripheral devices 138. It should be understood that, while remaining consistent with the embodiments, WTRU102 may also include any sub-combination of the foregoing components.

[0045] Processor 118 can be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), and a state machine, etc. Processor 118 can perform signal encoding, data processing, power control, input / output processing, and / or any other function that enables WTRU 102 to operate in a wireless environment. Processor 118 can be coupled to transceiver 120, and transceiver 120 can be coupled to transmitting / receiving unit 122. Although Figure 1B While the processor 118 and transceiver 120 are described as separate components, it should be understood that the processor 118 and transceiver 120 may also be integrated together in a single electronic component or chip.

[0046] Transmit / receive component 122 may be configured to transmit or receive signals to or from a base station (e.g., base station 114a) via air interface 116. For example, in one embodiment, transmit / receive component 122 may be an antenna configured to transmit and / or receive RF signals. As an example, in another embodiment, transmit / receive component 122 may be an emitter / detector configured to transmit and / or receive IR, UV, or visible light signals. In yet another embodiment, transmit / receive component 122 may be configured to transmit and / or receive RF and optical signals. It should be understood that transmit / receive component 122 may be configured to transmit and / or receive any combination of wireless signals.

[0047] Although Figure 1B While the transmit / receive component 122 is described as a single component, the WTRU 102 may include any number of transmit / receive components 122. More specifically, the WTRU 102 may use MIMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit / receive components 122 (e.g., multiple antennas) that transmit and receive wireless signals via the air interface 116.

[0048] Transceiver 120 can be configured to modulate signals to be transmitted by transmitter / receiver 122 and demodulate signals received by transmitter / receiver 122. As described above, WTRU 102 can have multimode capability. Therefore, transceiver 120 can include multiple transceivers that allow WTRU 102 to communicate using various RATs (e.g., NR and IEEE 802.11).

[0049] The processor 118 of WTRU102 can be coupled to a speaker / microphone 124, a numeric keypad 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) unit or an organic light-emitting diode (OLED) display unit), and can receive user input data from these components. The processor 118 can also output user data to the speaker / microphone 124, keypad 126, and / or display / touchpad 128. Furthermore, the processor 118 can access and store information from any suitable memory, such as non-removable memory 130 and / or removable memory 132. Non-removable memory 130 can include random access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. Removable memory 132 can include a subscriber identity module (SIM) card, memory stick, secure digital card (SD) memory card, etc. In other embodiments, the processor 118 can access and store information from memory that is not actually located in WTRU102; for example, such memory could be located in a server or home computer (not shown).

[0050] The processor 118 can receive power from the power supply 134 and can be configured to distribute and / or control power for other components in the WTRU 102. The power supply 134 can be any suitable device that powers the WTRU 102. For example, the power supply 134 may include one or more dry cell battery packs (such as nickel-cadmium (Ni-Cd), nickel-zinc (Ni-Zn), nickel-metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, and fuel cells, etc.

[0051] The processor 118 may also be coupled to a GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) related to the current location of the WTRU 102. As a supplement or replacement to the information from the GPS chipset 136, the WTRU 102 may receive location information from base stations (e.g., base stations 114a, 114b) via the air interface 116, and / or determine its location based on signal timing received from two or more nearby base stations. It should be understood that, while remaining consistent with the embodiments, the WTRU 102 may acquire location information using any suitable positioning method.

[0052] The processor 118 may also be coupled to other peripheral devices 138, which may include one or more software and / or hardware modules providing additional features, functions, and / or wired or wireless connectivity. For example, the peripheral device 138 may include an accelerometer, electronic compass, satellite transceiver, digital camera (for photos and / or video), Universal Serial Bus (USB) port, vibration device, television transceiver, hands-free headset, Bluetooth® module, FM radio unit, digital music player, media player, video game console module, internet browser, virtual reality and / or augmented reality (VR / AR) device, and activity tracker, etc. The peripheral device 138 may include one or more sensors, which may be one or more of the following: gyroscope, accelerometer, Hall effect sensor, magnetometer, orientation sensor, proximity sensor, temperature sensor, time sensor, geolocation sensor, altimeter, light sensor, touch sensor, magnetometer, barometer, gesture sensor, biometric sensor, and / or humidity sensor, etc.

[0053] WTRU102 may include a full-duplex wireless device, wherein the reception or transmission of some or all signals (e.g., associated with specific subframes for UL (e.g., for transmission) and downlink (e.g., for reception)) may be concurrent and / or simultaneous for the wireless device. The full-duplex wireless device may include an interference management unit 139 that reduces and / or substantially eliminates self-interference by means of hardware (e.g., choke coils) or by means of a processor (e.g., a separate processor (not shown) or by means of processor 118) for signal processing. In embodiments, WTRU102 may include a half-duplex wireless device that transmits and receives some or all signals (e.g., associated with specific subframes for UL (e.g., for transmission) or downlink (e.g., for reception).

[0054] Figure 1C This is a schematic diagram of a system according to an embodiment, showing RAN104 and CN106. As described above, RAN104 can communicate with WTRU102a, 102b, and 102c via air interface 116 using E-UTRA radio technology. RAN104 can also communicate with CN106.

[0055] RAN104 may include eNodeBs 160a, 160b, and 160c; however, it should be understood that RAN104 may include any number of eNodeBs while remaining consistent with the embodiments. Each of eNodeBs 160a, 160b, and 160c may include one or more transceivers communicating with WTRUs 102a, 102b, and 102c via air interface 116. In one embodiment, eNodeBs 160a, 160b, and 160c may implement MIMO technology. Thus, for example, eNodeB 160a may use multiple antennas to transmit radio signals to and / or receive radio signals from WTRU 102a.

[0056] Each of the e-nodes B160a, 160b, and 160c can be associated with a specific cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, user scheduling in UL and / or DL, etc. For example... Figure 1C As shown, nodes B160a, 160b, and 160c can communicate with each other via the X2 interface.

[0057] Figure 1C The CN106 shown may include a Mobility Management Entity (MME) 162, a Serving Gateway (SGW) 164, and a Packet Data Network (PDN) Gateway (or PGW) 166. While each of the foregoing components is described as part of the CN106, it should be understood that any of these components may be owned and / or operated by an entity other than a CN operator.

[0058] The MME162 can connect to each of the eNodeBs 162a, 162b, and 162c in RAN104 via the S1 interface and can act as a control node. For example, the MME162 can be responsible for authenticating users of WTRUs 102a, 102b, and 102c, performing bearer activation / deactivation processes, and selecting a specific serving gateway during the initial attach process of WTRUs 102a, 102b, and 102c, etc. The MME162 can provide control plane functions for handover between RAN104 and other RANs (not shown) using other radio technologies (such as GSM and / or WCDMA).

[0059] The SGW164 can connect to each of the eNodeBs B160a, 160b, and 160c in RAN104 via the S1 interface. The SGW164 typically routes and forwards user data packets to / from WTRU102a, 102b, and 102c. Furthermore, the SGW164 can perform other functions, such as anchoring the user plane during handover between eNBs, triggering paging processes when DL data is available to WTRU102a, 102b, and 102c, and managing and storing the context of WTRU102a, 102b, and 102c, etc.

[0060] SGW164 can be connected to PGW 146, which can provide packet-switched network (e.g., Internet 110) access for WTRU102a, 102b, 102c to facilitate communication between WTRU102a, 102b, 102c and IP-enabled devices.

[0061] CN106 can facilitate communication with other networks. For example, CN106 can provide WTRU102a, 102b, and 102c with access to a circuit-switched network (e.g., PSTN108) to facilitate communication between WTRU102a, 102b, and 102c and conventional landline communication equipment. For example, CN106 may include or communicate with an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server), and the IP gateway may act as an interface between CN106 and PSTN108. Furthermore, CN106 can provide WTRU102a, 102b, and 102c with access to the other network 112, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0062] Although Figure 1A-1D The WTRU is described as a wireless terminal; however, it should be understood that in some representative embodiments, such a terminal may use a wired communication interface (e.g., temporary or permanent) with the communication network.

[0063] In a representative embodiment, the other network 112 may be a WLAN.

[0064] A WLAN employing an Infrastructure Basic Services Set (BSS) model may have an Access Point (AP) for the BSS and one or more Stations (STAs) associated with the AP. The AP may access or interface with a Distributed System (DS) or other types of wired / wireless networks that send traffic into and / or out of the BSS. Traffic originating outside the BSS and destined for a STA can be delivered to the STA via the AP. Traffic originating from a STA and destined for a destination outside the BSS can be sent to the AP for delivery to the appropriate destination. Traffic between STAs within the BSS can be sent via the AP, for example, where a source STA can send traffic to the AP and the AP can deliver traffic to the destination STA. Traffic between STAs within the BSS may be considered and / or referred to as point-to-point traffic. Point-to-point traffic can be sent between the source and destination STAs (e.g., directly therebetween) using Direct Link Establishment (DLS). In some representative embodiments, the DLS may use 802.11e DLS or 802.11z Channelized DLS (TDLS). For example, a WLAN using the Standalone BSS (IBSS) mode does not have an access point (AP) and is located within the IBSS or the STAs using the IBSS (e.g., all STAs) can communicate directly with each other. Here, the IBSS communication mode is sometimes referred to as an "ad-hoc" communication mode.

[0065] When operating in 802.11ac infrastructure mode or a similar mode, the AP can transmit beacons on a fixed channel (e.g., the primary channel). The primary channel can have a fixed width (e.g., a 20 MHz bandwidth) or a width dynamically set via signaling. The primary channel can be the operating channel of the BSS and can be used by STAs to establish connections with the AP. In some representative embodiments, carrier-sense multiple access with collision avoidance (CSMA / CA) can be implemented (e.g., in an 802.11 system). For CSMA / CA, STAs, including the AP (e.g., each STA), can sense the primary channel. If a particular STA senses / detects and / or determines that the primary channel is busy, that particular STA can fall back. In a given BSS, at any given time, there is only one STA (e.g., only one station) transmitting.

[0066] High-throughput (HT) STAs can communicate using a 40MHz wide channel (e.g., by combining a 20MHz wide main channel with adjacent or non-adjacent 20MHz wide channels to form a 40MHz wide channel).

[0067] Very High Throughput (VHT) STAs can support channels with widths of 20MHz, 40MHz, 80MHz, and / or 160MHz. 40MHz and / or 80MHz channels can be formed by combining consecutive 20MHz channels. A 160MHz channel can be formed by combining eight consecutive 20MHz channels or by combining two non-consecutive 80MHz channels (this combination may be referred to as an 80+80 configuration). For the 80+80 configuration, after channel coding, data is transmitted and passed through a segmented parser that splits the data into two streams. Inverse Fast Fourier Transform (IFFT) processing and time-domain processing can be performed individually on each stream. The streams can be mapped onto two 80MHz channels, and the data can be transmitted by the STA performing the transmission. On the receiver of the STA performing the reception, the above operations for the 80+80 configuration can be reversed, and the combined data can be sent to the Media Access Control (MAC).

[0068] 802.11af and 802.11ah support sub-1 GHz operating modes. Compared to 802.11n and 802.11ac, 802.11af and 802.11ah utilize reduced channel bandwidth and carriers. 802.11af supports 5 MHz, 10 MHz, and 20 MHz bandwidths in the TV white space (TVWS) spectrum, while 802.11ah supports 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz bandwidths using non-TVWS spectrum. According to a representative embodiment, 802.11ah can support instrument-type control / machine-type communication (MTC) (e.g., MTC devices in macro coverage areas). MTC devices may have certain capabilities, such as limited capabilities including support (e.g., only support) certain and / or limited bandwidths. MTC devices may include a battery with a battery life exceeding a threshold (e.g., for maintaining a very long battery life).

[0069] For WLAN systems that can support multiple channels and channel bandwidths (e.g., 802.11n, 802.11ac, 802.11af, and 802.11ah), these systems include a channel that can be designated as the primary channel. The bandwidth of the primary channel can be equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel can be set and / or limited by a single STA, which is derived from all STAs operating in the BSS supporting the minimum bandwidth operating mode. In the example of 802.11ah, even if the AP and other STAs in the BSS support 2MHz, 4MHz, 8MHz, 16MHz, and / or other channel bandwidth operating modes, the width of the primary channel can be 1MHz for STAs that support (e.g., only support) the 1MHz mode (e.g., MTC type devices). Carrier sensing and / or Network Allocation Vector (NAV) settings can depend on the status of the primary channel. If the primary channel is busy (e.g., because an STA (which only supports the 1MHz operating mode) is transmitting to the AP), then the entire available band can be considered busy even if most of the available band remains idle and available.

[0070] In the United States, the available frequency band for 802.11ah is 902MHz to 928MHz. In South Korea, the available frequency band is 917.5MHz to 923.5MHz. In Japan, the available frequency band is 916.5MHz to 927.5MHz. Depending on the country code, the total bandwidth available for 802.11ah is 6MHz to 26MHz.

[0071] Figure 1D This is a system schematic diagram showing RAN113 and CN115 according to an embodiment. As described above, RAN113 can communicate with WTRU102a, 102b, and 102c via air interface 116 using NR radio technology. RAN113 can also communicate with CN115.

[0072] RAN113 may include gNBs 180a, 180b, and 180c; however, it should be understood that RAN113 may include any number of gNBs while maintaining conformity to the embodiments. Each of gNBs 180a, 180b, and 180c may include one or more transceivers for communicating with WTRUs 102a, 102b, and 102c via air interface 116. In one embodiment, gNBs 180a, 180b, and 180c may implement MIMO technology. For example, gNBs 180a and 180b may use beamforming to transmit and / or receive signals to and / or from gNBs 180a, 180b, and 180c. Thus, for example, gNB 180a may use multiple antennas to transmit radio signals to and receive radio signals from WTRU 102a. In embodiments, gNBs 180a, 180b, and 180c may implement carrier aggregation technology. For example, gNB180a can transmit multiple component carriers to WTRU102a (not shown). A subset of these component carriers may be on unlicensed spectrum, while the remaining component carriers may be on licensed spectrum. In embodiments, gNB180a, 180b, and 180c may implement Cooperative Multipoint (CoMP) technology. For example, WTRU102a may receive cooperative transmissions from gNB180a and gNB180b (and / or gNB180c).

[0073] WTRU102a, 102b, and 102c can communicate with gNB180a, 180b, and 180c using transmissions associated with scalable digital configurations. For example, the OFDM symbol spacing and / or OFDM subcarrier spacing can be different for different transmissions, different cells, and / or different portions of the radio transmission spectrum. WTRU102a, 102b, and 102c can communicate with gNB180a, 180b, and 180c using subframes or transmission time intervals (TTIs) of different or scalable lengths (e.g., containing different numbers of OFDM symbols and / or varying absolute durations).

[0074] gNB180a, 180b, and 180c can be configured to communicate with WTRU102a, 102b, and 102c in standalone and / or non-standalone configurations. In standalone configuration, WTRU102a, 102b, and 102c can communicate with gNB180a, 180b, and 180c without accessing other RANs (e.g., eNodeB160a, 160b, and 160c). In standalone configuration, WTRU102a, 102b, and 102c can use one or more of gNB180a, 180b, and 180c as their mobile anchor point. In standalone configuration, WTRU102a, 102b, and 102c can use signals in unlicensed frequency bands to communicate with gNB180a, 180b, and 180c. In a non-standalone configuration, WTRU102a, 102b, and 102c communicate / connect with gNB180a, 180b, and 180c simultaneously with other RANs (e.g., eNodeBs B160a, 160b, and 160c). For example, WTRU102a, 102b, and 102c can communicate substantially simultaneously with one or more gNBs B180a, 180b, and 180c, as well as one or more eNodeBs B160a, 160b, and 160c, by implementing DC principles. In a non-standalone configuration, eNodeBs B160a, 160b, and 160c can act as mobile anchor points for WTRU102a, 102b, and 102c, and gNBs B180a, 180b, and 180c can provide additional coverage and / or throughput to service WTRU102a, 102b, and 102c.

[0075] Each of gNB180a, 180b, and 180c can be associated with a specific cell (not shown) and can be configured to handle radio resource management decisions, handover decisions, user scheduling in UL and / or DL, support network slicing, dual connectivity, implement interoperability processing between NR and E-UTRA, route user plane data to User Plane Functions (UPF) 184a and 184b, and route control plane information to Access and Mobility Management Functions (AMF) 182a and 182b, etc. Figure 1D As shown, gNB180a, 180b, and 180c can communicate with each other via the Xn interface.

[0076] Figure 1DThe CN115 shown may include at least one AMF182a, 182b, at least one UPF184a, 184b, at least one Session Management Function (SMF)183a, 183b, and may include Data Network (DN) 185a, 185b. While each of the foregoing components is described as part of CN115, it should be understood that any of these components may be owned and / or operated by an entity other than a CN operator.

[0077] AMF182a and 182b can connect to one or more of gNB180a, 180b, and 180c in RAN113 via the N2 interface and can act as control nodes. For example, AMF182a and 182b can be responsible for authenticating users of WTRU102a, 102b, and 102c, supporting network slicing (e.g., handling different PDU sessions with different needs), selecting specific SMF183a and 183b, managing registration areas, terminating NAS signaling, and mobility management, etc. AMF182a and 182b can use network slicing to customize the CN support provided to WTRU102a, 102b, and 102c based on the service type used by WTRU102a, 102b, and 102c. As an example, different network slices can be established for different use cases, such as services relying on Ultra Reliable Low Latency (URLLC) access, services relying on Enhanced Massive Mobile Broadband (eMBB) access, and / or services for Machine Class Communication (MTC) access, etc. AMF 182 can provide control plane functions for switching between RAN113 and other RANs (not shown) that use other radio technologies (e.g., LTE, LTE-A, LTE-A Pro and / or non-3GPP access technologies such as WiFi).

[0078] SMF183a and 183b can connect to AMF182a and 182b in CN115 via the N11 interface. SMF183a and 183b can also connect to UPF184a and 184b in CN115 via the N4 interface. SMF183a and 183b can select and control UPF184a and 184b, and can configure traffic routing through UPF184a and 184b. SMF183a and 183b can perform other functions, such as managing and allocating WTRU or UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, and providing downlink data notifications, etc. PDU session types can be IP-based, non-IP-based, Ethernet-based, etc.

[0079] UPF184a and 184b can connect to one or more of gNB180a, 180b, and 180c in RAN113 via the N3 interface, thus providing WTRU102a, 102b, and 102c with access to a packet-switched network (e.g., Internet 110) to facilitate communication between WTRU102a, 102b, and 102c and IP-enabled devices. UPF184 and 184b can perform other functions, such as routing and forwarding packets, enforcing user plane policies, supporting multihomed PDU sessions, handling user plane QoS, buffering downlink packets, and providing mobility anchoring, etc.

[0080] CN115 can facilitate communication with other networks. For example, CN115 may include or can communicate with an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that acts as an interface between CN115 and PSTN108. Furthermore, CN115 can provide WTRU102a, 102b, and 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers. In one embodiment, WTRU102a, 102b, and 102c can be connected to DN185a and DN185b via the N3 interface connected to UPF184a and 184b and the N6 interface between UPF184a and 184b and local data networks (DNs) 185a and 185b.

[0081] In view of Figure 1A-1D And about Figure 1A-1D The corresponding descriptions herein refer to one or more of the following functions, which can be performed by one or more emulation devices (not shown): WTRU102a-d, base station 114a-b, eNodeB160a-c, MME162, SGW164, PGW166, gNB180a-c, AMF182a-b, UPF184a-b, SMF183a-b, DN185a-b, and / or one or more other devices described herein. These emulation devices can be one or more devices configured to simulate one or more of the functions described herein. For example, these emulation devices can be used to test other devices and / or simulate network and / or WTRU functions.

[0082] The simulation device may be designed to perform one or more tests on other devices in a laboratory environment and / or a carrier network environment. For example, the one or more simulation devices may perform one or more functions while being implemented and / or deployed, wholly or partially, as part of a wired and / or wireless communication network, to test other devices within the communication network. The one or more simulation devices may perform one or more functions while being implemented or deployed temporarily as part of a wired and / or wireless communication network. The simulation device may be directly coupled to other devices to perform tests, and / or may use over-the-air wireless communication to perform tests.

[0083] The one or more simulation devices can perform one or more functions, including all functionalities, without being implemented or deployed as part of a wired and / or wireless communication network. For example, the simulation device can be used in a test laboratory and / or a test scenario where a wired and / or wireless communication network is not deployed (e.g., under test) to perform tests on one or more components. The one or more simulation devices can be test equipment. The simulation device can transmit and / or receive data using direct RF coupling and / or wireless communication via RF circuitry (e.g., the circuitry may include one or more antennas).

[0084] This application describes various aspects, including tools, features, examples or embodiments, models, methods, etc. Many of these aspects are described as specific and are generally described in a manner that may sound restrictive, at least to illustrate individual characteristics. However, this is for the purpose of clarity and does not limit the scope of this application or those aspects. In fact, all the different aspects can be combined and interchanged to provide other aspects. Furthermore, these aspects can also be combined and interchanged with aspects described in earlier documents.

[0085] The aspects described and contemplated in this application can be implemented in many different forms. Figure 5-8 Some embodiments may be provided, but other embodiments may also be considered. Figure 5-8 The discussion does not limit the breadth of the implementations described. At least one of the aspects relates primarily to video encoding and decoding, and at least one other aspect relates primarily to transmitting the generated or encoded bitstream. These and other aspects may be implemented as methods, apparatus, computer-readable storage media having instructions stored thereon for encoding or decoding video data according to any of the described methods, and / or computer-readable storage media having a bitstream generated according to any of the described methods stored thereon.

[0086] In this application, the terms “reconstruction” and “decoding” are used interchangeably, the terms “pixel” and “sample” are used interchangeably, and the terms “image”, “picture” and “frame” are used interchangeably.

[0087] This document describes various methods, and each method includes one or more steps or actions for implementing the described method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions may be modified or combined. Furthermore, terms such as "first" and "second" may be used in various embodiments to modify elements, components, steps, operations, etc., such as, for example, "first decoding" and "second decoding." Unless specifically required, the use of these terms does not imply an ordering of the modified operations. Therefore, in this example, the first decoding does not need to be performed before the second decoding and may occur, for example, before, during, or in a time period overlapping with the second decoding.

[0088] The various methods and other aspects described in this application can be modified (e.g., for example) as follows: Figure 2 and Figure 3 The modules of the video encoder 200 and decoder 300 shown, such as intra-frame prediction and entropy decoding and / or decoding modules (260, 360, 245, 330), are further presented in the subject matter herein, which is not limited to VVC or HEVC and can be applied to, for example, any type, format, or version of video decoding, whether described in standards or recommendations, whether pre-existing or future-developed, and any such standard and recommendation extensions (e.g., including VVC and HEVC). Unless otherwise indicated or technically excluded, the aspects described in this application may be used individually or in combination.

[0089] Various numerical values ​​are used in the examples described in this application, such as bit value logic (e.g., logic values ​​of 0 or 1), numerical ranges (e.g., 0 to 255), payload type value of 12 for CCC SEI messages, payload type value of 13 for AA SEI messages, etc. These and other specific values ​​are for illustrative purposes, and the aspects described are not limited to these specific values.

[0090] Figure 2 This is a schematic diagram illustrating an example video encoder. Variations of the example encoder 200 can be envisioned, but for clarity, the encoder 200 is described below, without describing all anticipated variations.

[0091] Before being encoded, the video sequence may undergo pre-coding processing (201), such as applying a color transformation to the input color image (e.g., a conversion from RGB 4:4:4 to YCbCr 4:2:0), or performing a remapping of the input image components to obtain a signal distribution that is more resilient to compression (e.g., histogram equalization using one of the color components). Metadata may be associated with this pre-processing and appended to the bitstream.

[0092] In encoder 200, the image is encoded by encoder elements as described below. The image to be encoded is segmented (202) in units such as decoding units (CUs). Each unit is encoded using, for example, intra-frame or inter-frame modes. When a unit is encoded in intra-frame mode, intra-frame prediction (260) is performed. In inter-frame mode, motion estimation (275) and compensation (270) are performed. The encoder decides (205) to encode the unit using either intra-frame or inter-frame mode, and indicates the intra-frame / inter-frame decision by, for example, a prediction mode flag. For example, the prediction residual is calculated by subtracting (210) the prediction block from the original image block.

[0093] Then, the predicted residual is transformed (225) and quantized (230). The quantized transform coefficients, motion vectors, and other syntax elements are entropy decoded (245) to output a bitstream. The encoder may skip the transform and directly apply quantization to the untransformed residual signal. The encoder may bypass the transform and quantization, i.e., directly decode the residual without applying the transform or quantization.

[0094] The encoder decodes the coded block to provide a reference for further prediction. The quantization transform coefficients are dequantized (240) and inverse transformed (250) to decode the prediction residual. The image block is reconstructed by combining the decoded prediction residual with the prediction block (255). An in-loop filter (265) is applied to the reconstructed image to perform, for example, deblocking / SAO (sample adaptation offset) filtering, thereby reducing coding artifacts. The filtered image is stored in a reference image buffer (280).

[0095] Figure 3 This is a schematic diagram illustrating an example video decoder. In the example decoder 300, the bitstream is decoded by decoder elements as described below. The video decoder 300 typically performs the same operations as... Figure 2The encoder 200 is a decoding process that is the reverse of the encoding process described herein. It can also typically perform video decoding as part of the encoded video data. For example, the encoder 200 can perform one or more of the video decoding steps presented herein. The encoder can reconstruct the decoded image, for example, to maintain synchronization with the decoder relative to one or more of the following: a reference image, an entropy decoding context, and other decoder-related state variables.

[0096] Specifically, the input to the decoder comprises a video bitstream, which can be generated by the video encoder 200. This bitstream is first entropy-decoded (330) to obtain transform coefficients, motion vectors, and other decoded information. Image partitioning information indicates how the image is partitioned. The decoder can therefore partition (335) the image based on the decoded image partitioning information. The transform coefficients are dequantized (340) and inverse transformed (350) to decode the prediction residuals. Image blocks are reconstructed by combining the decoded prediction residuals with prediction blocks (355). The prediction blocks can be obtained from intra-frame prediction (360) or motion-compensated prediction (i.e., inter-frame prediction) (375) (370). An in-loop filter (365) is applied to the reconstructed image. The filtered image is stored in a reference image buffer (380).

[0097] The decoded image can be further processed by post-decoding (385), such as inverse color transformation (e.g., conversion from YCbCr 4:2:0 to RGB 4:4:4) or inverse remapping of the remapping process performed in the pre-encoding process (201). The post-decoding process may use metadata derived in the pre-encoding process and signaled in the bitstream.

[0098] Figure 4This is a schematic diagram illustrating examples of systems in which the various aspects and embodiments described herein can be implemented. System 400 can be implemented as a device including the various components described below and configured to perform one or more aspects described herein. Examples of such devices include, but are not limited to, various electronic devices such as personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, and servers. Elements of system 400 can be implemented individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one example, the processing and encoder / decoder elements of system 400 are distributed across multiple ICs and / or discrete components. In various embodiments, system 400 is communicatively coupled to one or more other systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports. In various embodiments, system 400 is configured to implement one or more aspects described herein.

[0099] The system 400 includes at least one processor 410 configured to execute instructions loaded therein for implementing various aspects described herein, such as those described herein. The processor 410 may include embedded memory, input / output interfaces, and various other circuitry known in the art. The system 400 includes at least one memory 420 (e.g., a volatile memory device and / or a non-volatile memory device). The system 400 includes a storage device 440, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, the storage device 440 may include internal storage devices, attached storage devices (including removable and non-removable storage devices), and / or network-accessible storage devices.

[0100] System 400 includes an encoder / decoder module 430 configured to, for example, process data to provide encoded or decoded video, and the encoder / decoder module 430 may include its own processor and memory. The encoder / decoder module 430 represents one or more modules that may be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of the encoding and decoding modules. Alternatively, the encoder / decoder module 430 may be implemented as a separate element of system 400, or it may be incorporated into processor 410 as a combination of hardware and software known to those skilled in the art.

[0101] Program code to be loaded onto processor 410 or encoder / decoder 430 to execute the various aspects described herein may be stored in storage device 440 and subsequently loaded onto memory 420 for execution by processor 410. According to various embodiments, one or more of processor 410, memory 420, storage device 440, and encoder / decoder module 430 may store one or more of various items during the execution of the processes described herein. These stored items may include, but are not limited to, input video, decoded video or portions of the decoded video, bitstreams, matrices, variables, and intermediate or final results from the processing of equations, formulas, operations, and operational logic.

[0102] In some embodiments, the memory within the processor 410 and / or encoder / decoder module 430 is used to store instructions and provides working memory for processing required during encoding or decoding. However, in other embodiments, external memory (e.g., the processing device may be the processor 410 or the encoder / decoder module 430) is used for one or more of these functions. The external memory may be the memory 420 and / or the storage device 440, for example, volatile memory and / or non-volatile flash memory. In several embodiments, external non-volatile flash memory is used to store, for example, the operating system of a television. In at least one embodiment, a fast external dynamic volatile memory such as RAM is used as the working memory for video decoding and decoding operations, such as MPEG-2 (MPEG stands for Moving Picture Experts Group; MPEG-2 is also known as ISO / IEC 13818, 13818-1 is also known as H.222, and 13818-2 is also known as H.262), HEVC (HEVC stands for High Efficiency Video Decoding; it is also known as H.265 and MPEG-H Part 2) or VVC (Various Video Decoding, which is a new standard developed by JVET (Joint Video Experts Group)).

[0103] As shown in box 445, input to the components of system 400 can be provided through various input devices. Such input devices include, but are not limited to, (i) a radio frequency (RF) section that receives, for example, RF signals transmitted over the air by a broadcasting company, (ii) a component input terminal (or a set of COMP input terminals), (iii) a universal serial bus (USB) input terminal, and / or (iv) a high-definition multimedia interface (HDMI) input terminal. Figure 4 Other examples not shown include composite videos.

[0104] In various embodiments, the input device of block 445 has associated corresponding input processing elements known in the art. For example, the RF section may be associated with elements suitable for: (i) selecting a desired frequency (also referred to as selecting a signal, or band-limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) band-limiting it again to a narrower frequency band to select, for example, a signal band, which in some embodiments may be referred to as a channel, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section of various embodiments includes one or more elements to perform these functions, such as frequency selectors, signal selectors, band limiters, channel selectors, filters, down-converters, demodulators, error correctors, and demultiplexers. The RF section may include tuners that perform various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or baseband. In one set-top box embodiment, the RF section and its associated input processing elements receive RF signals transmitted via a wired (e.g., cable) medium and perform frequency selection to a desired frequency band through filtering, down-conversion, and re-filtering. Various embodiments may rearrange the order of the above (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions. Adding elements may include inserting elements between existing elements, such as inserting amplifiers and analog-to-digital converters. In various embodiments, the RF section includes an antenna.

[0105] Additionally, the USB and / or HDMI terminals may include corresponding interface processors for connecting system 400 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Solomon error correction) may be implemented as needed, for example, within a separate input processing IC or within processor 410. Similarly, various aspects of USB or HDMI interface processing may be implemented as needed, either within a separate interface IC or within processor 410. The demodulated, error-corrected, and demultiplexed stream is provided to various processing elements, including, for example, processor 410 and encoder / decoder 430, which operate in conjunction with memory and storage elements to process the data stream as needed for presentation on an output device.

[0106] Various components of system 400 can be housed within an integrated housing. Within this integrated housing, the various components can be interconnected and data can be transmitted therebetween using suitable connection means 425, such as internal buses known in the art, including inter-IC (I2C) buses, wiring, and printed circuit boards.

[0107] The system 400 includes a communication interface 450 that enables communication with other devices via a communication channel 460. The communication interface 450 may include, but is not limited to, a transceiver configured to send and receive data via the communication channel 460. The communication interface 450 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 460 may be implemented, for example, in a wired and / or wireless medium.

[0108] In various embodiments, data is streamed or otherwise provided to system 400 using a wireless network, such as a Wi-Fi network, for example, IEEE 802.11 (IEEE refers to the Institute of Electrical and Electronics Engineers). In these examples, Wi-Fi signals are received via a communication channel 460 and a communication interface 450 suitable for Wi-Fi communication. The communication channel 460 in these embodiments is typically connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top communications. Other embodiments use a set-top box to provide streaming data to system 400, transmitting the data via an HDMI connection to input box 445. Still other embodiments use an RF connection to input box 445 to provide streaming data to system 400. As described above, various embodiments provide data in a non-streaming manner. Additionally, various embodiments use wireless networks other than Wi-Fi, such as cellular networks or Bluetooth networks.

[0109] The system 400 can provide output signals to various output devices, including a display 475, a speaker 485, and other peripheral devices 495. The display 475 in various embodiments includes one or more of the following: for example, a touchscreen display, an organic light-emitting diode (OLED) display, a flexible display, and / or a foldable display. The display 475 can be used in televisions, tablets, laptops, cellular phones (mobile phones), or other devices. The display 475 can also be integrated with other components (e.g., as in a smartphone) or separate (e.g., as an external monitor for a laptop computer). In various examples of embodiments, the other peripheral devices 495 include one or more of the following: a standalone digital video disc (or digital multifunction disc) (DVR, used for both terms), a disc player, a stereo system, and / or a lighting system. Various embodiments use one or more peripheral devices 495 that provide functionality based on the output of the system 400. For example, a disc player performs the function of playing the output of the system 400.

[0110] In various embodiments, control signals can be transmitted between system 400 and display 475, speaker 485, or other peripheral devices 495 using signaling via communication protocols such as AV Link, Consumer Electronics Control (CEC), or other communication protocols that enable device-to-device control with or without user intervention. The output devices can be communicatively coupled to system 400 via dedicated connections through corresponding interfaces 470, 480, and 490. Alternatively, the output devices can be connected to system 400 via communication interface 450 using communication channel 460. The display 475 and speaker 485 can be integrated with other components of system 400 into a single unit within an electronic device (e.g., a television set). In various embodiments, the display interface 470 includes a display driver, such as a timing controller (TCon) chip.

[0111] For example, if the RF portion of input 445 is part of a separate set-top box, then display 475 and speaker 485 may alternatively be separated from one or more other components. In various embodiments where display 475 and speaker 485 are external components, the output signal may be provided via a dedicated output connection, which may include, for example, an HDMI port, a USB port, or a COMP output.

[0112] The embodiments described herein may be executed by computer software implemented by the processor 410, or by hardware, or by a combination of hardware and software. As a non-limiting example, the embodiments may be implemented by one or more integrated circuits. The memory 420 may be of any type suitable for the technical environment and may be implemented using any suitable data storage technology, such as, as a non-limiting example, optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory. The processor 410 may be of any type suitable for the technical environment and, as a non-limiting example, may include one or more of the following: microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures.

[0113] Various implementations involve decoding. As used in this application, "decoding" can include all or part of a process performed, for example, on a received encoded sequence, to produce a final output suitable for display. In various embodiments, such a process includes one or more processes typically performed by a decoder: for example, entropy decoding, inverse quantization, inverse transform, and differential decoding. In various embodiments, such a process may also optionally include a process performed by a decoder of various implementations described herein, such as determining which attribute sub-bitstream is active for decoding the visual volumetric content based on a message indicating which attribute sub-bitstream is active in a parameter set associated with the visual volumetric content; using the active attribute sub-bitstream to decode the visual volumetric content; obtaining a parameter set including attribute information characterizing the visual volumetric content; receiving a message indicating which attribute sub-bitstream among a plurality of attribute sub-bitstreams in the parameter set associated with the visual volumetric content is active; determining active and inactive attribute sub-bitstreams based on the message; receiving the active attribute sub-bitstream; using the active attribute sub-bitstream to decode the visual volumetric content and skipping the inactive attribute sub-bitstreams; determining inactive attribute sub-bitstreams based on the message; determining that an attribute sub-bitstream indicated in the parameter set but not in the message is an inactive attribute sub-bitstream; skipping the inactive attribute sub-bitstreams for the decoding of the visual volumetric content; and so on.

[0114] As a further embodiment, in one example, "decoding" refers only to entropy decoding; in another embodiment, "decoding" refers only to differential decoding; and in yet another embodiment, "decoding" refers to a combination of entropy decoding and differential decoding. Whether the phrase "decoding process" is intended to specifically refer to a subset of operations or generally to a broader decoding process can be clearly determined based on the specific context of the description and is believed to be well understood by those skilled in the art.

[0115] Various implementations involve encoding. In a manner similar to the above discussion of “decoding,” “encoding,” as used herein, can include all or part of a process performed, for example, on an input video sequence to produce an encoded bitstream. In various embodiments, such a process includes one or more processes typically performed by an encoder, such as partitioning, differential coding, transform, quantization, and entropy coding. In various embodiments, such a process also or alternatively includes processes performed by an encoder of the various implementations described herein, such as determining to deactivate one of a plurality of attribute sub-bitstreams indicated in a parameter set associated with visual volumetric content; determining to deactivate one of the plurality of attribute sub-bitstreams indicated in the parameter set associated with the visual volumetric content based on bitrate adaptation; generating and sending a message indicating the deactivation of the attribute sub-bitstream; and so on.

[0116] As a further example, in one embodiment, “encoding” refers only to entropy encoding; in another embodiment, “encoding” refers only to differential encoding; and in yet another embodiment, “encoding” refers to a combination of differential and entropy encoding. Whether the phrase “encoding process” is intended to specifically refer to a subset of operations or generally to a broader encoding process can be clearly determined based on the specific context of the description and is believed to be fully understood by those skilled in the art.

[0117] Note that the grammatical elements used herein (such as those indicated in Tables 1-7 and in the discussions or figures given here) are descriptive terms. Therefore, they do not preclude the use of other grammatical element names.

[0118] When the accompanying drawings are presented as flowcharts, it should be understood that they also provide block diagrams of the corresponding devices. Similarly, when the accompanying drawings are presented as block diagrams, it should be understood that they also provide flowcharts of the corresponding methods / processes.

[0119] During the encoding process, a balance or trade-off between rate and distortion is typically considered, usually with constraints on computational complexity. Rate-distortion optimization is generally formulated as minimizing a rate-distortion function, which is a weighted sum of rate and distortion. Different approaches exist to address the rate-distortion optimization problem. For example, these approaches can be based on extensive testing of all encoding options, including all considered modes or encoding parameter values, to fully evaluate their encoding costs and the associated distortion of the reconstructed signal after encoding and decoding. Faster methods can also be used to save encoding complexity, specifically by calculating approximate distortion based on predicting or predicting the residual signal instead of the reconstructed signal. A hybrid of these two approaches can also be used, for example by using approximate distortion only for some possible encoding options and full distortion for others. Other methods evaluate only a subset of possible encoding options. More generally, many methods can employ any of a variety of techniques to perform the optimization, but the optimization is not necessarily a full evaluation of both decoding costs and associated distortion.

[0120] The implementations and aspects described herein can be implemented, for example, in methods or processes, apparatuses, software programs, data streams, or signals. Even if discussed only in the context of a single form of implementation (e.g., discussed only as a method), the implementation of the features in question can also be implemented in other forms (e.g., apparatuses or programs). For example, an apparatus can be implemented with appropriate hardware, software, and firmware. The method can be implemented, for example, in a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices, such as computers, cellular phones, portable / personal digital assistants (“PDAs”), and other devices that facilitate information communication between end users.

[0121] References to “an embodiment,” “an example,” “an implementation,” or “implementation,” and other variations, mean that a particular feature, structure, characteristic, etc., described in connection with that embodiment is included in at least one embodiment. Therefore, the phrases “in an embodiment,” “in an example,” “in an implementation,” or “in an implementation,” and any other variations appearing in various places throughout this application, do not necessarily all refer to the same embodiment or example.

[0122] Additionally, this application may relate to "determining" various types of information. Determining such information may include one or more of the following: estimating the information, calculating the information, predicting the information, or retrieving the information from memory. Obtaining may include receiving, retrieving, constructing, generating, and / or determining.

[0123] Furthermore, this application may relate to "accessing" various types of information. Accessing such information may include one or more of the following: for example, receiving the information, retrieving the information (e.g., from memory), storing the information, moving the information, copying the information, calculating the information, determining the information, predicting the information, or estimating the information.

[0124] Additionally, this application can refer to "receiving" various types of information. Like "accessing," receiving is intended to be a broad term. Receiving the information may include one or more of the following: for example, accessing the information or (e.g., retrieving the information from memory). Furthermore, "receiving" is generally referred to in one or more ways during operations such as storing the information, processing the information, sending the information, moving the information, copying the information, erasing the information, calculating the information, determining the information, predicting the information, or estimating the information.

[0125] It should be understood that, for example, in the cases of “A / B,” “A and / or B,” and “at least one of A and B,” the use of any of the following “ / ,” “and / or,” and “at least one of” is intended to cover the selection of only the first listed option (A), or only the selection of only the second listed option (B), or the selection of both options (A and B). As a further example, in the cases of “A, B, and / or C” and “at least one of A, B, and C,” such wording is intended to include selecting only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or selecting all three options (A, B, and C). As will be apparent to those skilled in the art and related fields, this can be extended to multiple listed items.

[0126] Furthermore, as used herein, the term "signal" specifically refers to indicating something to the corresponding decoder. For example, in some embodiments, the encoder (e.g., to the decoder) signals a change in the video component bitstream used for decoding, a change in the video component codec used for decoding or an active / inactive codec, a change in the video component attribute used for decoding or an active / inactive attribute, a change in the set of video component attributes used for decoding or an active / inactive parameter set, etc. Thus, in one embodiment, the same parameters are used on both the encoder and decoder sides. Therefore, for example, the encoder can send (explicit signaling) specific parameters to the decoder so that the decoder can use the same specific parameters. Conversely, if the decoder already has specific parameters as well as other parameters, signaling can be used without sending (implicit signaling) to simply allow the decoder to know and select specific parameters. Bit savings are achieved in various embodiments by avoiding the transmission of any actual functionality. It should be understood that signaling can be implemented in various ways. For example, in various embodiments, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder. Although the foregoing refers to the verb form of the term "signal," the term "signal" may also be used as a noun herein.

[0127] As will be apparent to those skilled in the art, implementations can generate various signals formatted to carry information, such as information that can be stored or transmitted. This information may include, for example, instructions for performing a method, or data generated by one of the described implementations. For example, the signal may be formatted to carry a bitstream of the described embodiments. Such a signal may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of a spectrum) or baseband signals. The formatting may include, for example, encoding a data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal can be transmitted via various wired or wireless links. The signal may be stored on a processor-readable medium.

[0128] The examples described herein can be applied to 3D content. The examples described herein can be applied to visual volumetric content. This visual volumetric content can be captured in a point cloud. Visual volumetric content can be captured in immersive video content. The examples described herein can be applied to video-based point cloud compression (V-PCC) or video-based decoding based on visual volumetricity (V3C). For example, although some embodiments may be described according to V-PCC, these embodiments are equally applicable to V3C. Therefore, in a sense, these terms are used interchangeably, and the examples described according to V-PCC are equally applicable to V3C.

[0129] 3D point clouds can be used to represent 3D content (e.g., immersive media). A point cloud can include a set of points represented in three-dimensional (3D) space. (Each) point can be associated with coordinates indicating the location of that point and / or one or more attributes (e.g., point color, transparency, acquisition time, laser reflectivity, or material properties, etc.). Point clouds can be captured or deployed, for example, using one or more cameras, depth sensors, and / or light detection and ranging (LiDAR) laser scanners. A point cloud can include multiple points. (Each) point can be represented by a set of coordinates mapped in three-dimensional (3D) space (e.g., x, y, z coordinates). Points can be generated based on sampling of objects. In the example, the number of points within a point cloud can be in the millions or billions. Point clouds can be used to reconstruct one or more objects and / or scenes.

[0130] V3C, including decoding and encoding, can be used to process the 3D content. Volumetric content can be represented and / or compressed, for example, for efficient storage and transmission. Volumetric content can include visual volumetric content. Visual volumetric content can be processed based on: visual volumetric video encoding (V3C) and / or video-based point cloud compression (V-PCC). Visual volumetric content can include V3C content or V-PCC content.

[0131] V3C can be based on point cloud compression. Point cloud compression can support lossy and / or lossless decoding (e.g., encoding or decoding) of the geometric coordinates and / or attributes of point clouds. Point clouds can be deployed to support a variety of applications (e.g., telepresence, virtual reality (VR), and large-scale dynamic 3D maps).

[0132] Figure 5 An example bitstream structure for video-based point cloud compression (V-PCC) is shown. Figure 5 As shown, one or more video bitstreams can be generated. For example, a V-PCC bitstream can be generated by multiplexing the generated video bitstreams (one or more) with the corresponding metadata.

[0133] A V-PCC bitstream may include a set of V-PCC units, such as... Figure 5 As shown in Table 1, an example of the syntax for a V-PCC cell can be included, which may include a V-PCC cell header and a V-PCC cell payload. Table 2 shows an example of the syntax for a V-PCC cell header. Table 3 shows an example of the syntax for a V-PCC cell payload. The V-PCC cell header may include an indication of the V-PCC cell type (e.g., as shown in Table 2). In the examples, a V-PCC cell with cell types VPCC_OVD, VPCC_GVD, or VPCC_AVD may include, for example, occupancy data cells, geometric data cells, and / or attribute data cells. The three components (i.e., occupancy, geometric, and / or attribute data cells) can be used to reconstruct a visual volumetric content (e.g., represented by a point cloud). The V-PCC attribute cell header may indicate the attribute type and an index that can be associated with that attribute type. Multiple instances of the attribute type may exist.

[0134] The payload, including occupancy V-PCC units, geometric V-PCC units, and attribute V-PCC units, may correspond to video data units (e.g., network abstraction layer (NAL) units) that can be decoded by a video decoder (e.g., a decoder), and may be specified in the corresponding set of occupancy, geometry, and attribute parameters of the V-PCC units.

[0135] Table 1 - Syntax Examples of V-PCC Units

[0136] Table 2 - Syntax Examples of V-PCC Unit Headers

[0137] Table 3 - Syntax Examples of V-PCC Element Payload

[0138] One or more messages (e.g., supplemental enhancement messages) can be used to assist one or more processes related to the decoding (e.g., encoding or decoding), reconstruction, display, etc. of media content.

[0139] Changes in network transmission conditions can be dynamically adapted. For example, MPEG Dynamic Adaptive Streaming over HTTP (MPEG-DASH) is a transmission format that can dynamically adapt to changes in network transmission conditions, for example, to provide a better end-user experience.

[0140] Dynamic HTTP streaming can deliver multimedia content at one or more bitrates available at the server. Multimedia content can include multiple media components (e.g., audio, video, text). Different media components can have different characteristics. The characteristics of a media component can be described, for example, through a Media Presentation Description (MPD).

[0141] Figure 6 An example MPD hierarchical data model is shown. For example... Figure 6 As shown, the MPD can describe a sequence of time periods (e.g., a time interval). For example, the set of encoded versions of media content components may remain unchanged during a time period. A time period can be associated with a start time and a duration. A time period may include one or more adaptation sets (e.g., adaptation sets, such as...). Figure 6 The adapter set shown is I). The DASH streaming client can be WTRU, for example, as referenced here. Figure 1A-1D As described.

[0142] An adapter set can represent a collection of encoded versions of one or more media content components that share one or more of the same attributes, such as language, media type, image aspect ratio, character, accessibility, viewpoint, and / or rating attributes. In the example, an adapter set may include different bitrates for the visual components of the multimedia content. An adapter set may include different bitrates for the audio components of the same multimedia content (e.g., lower-quality stereo and / or higher-quality surround sound). An adapter set may include multiple representations.

[0143] A representation can describe a deliverable encoded version of one or more (e.g., several) media components. A representation can differ from other representations, for example, by bitrate, resolution, number of channels, and / or other characteristics. A representation can include one or more segments. Attributes of the representation element (e.g., @id, @bandwidth, @qualityRanking, and @dependencyId) can (e.g., be used to) specify one or more attributes of the representation.

[0144] Fragments can be retrieved using requests (e.g., HTTP requests). A fragment (e.g., each fragment) may have a URL (e.g., an addressable address on a server). Fragments can be downloaded, for example, using an HTTP GET or an HTTP GET with a one-byte range.

[0145] A DASH client can parse an MPD XML document. The DASH client can select an adaptation set (e.g., an adaptation set suitable for its environment) based on elements of the adaptation set. The client can select (e.g., one) a representation for an adaptation set (e.g., within each adaptation set). The client can select the representation based on values ​​such as the @bandwidth attribute, client decoding capabilities, and / or client rendering capabilities. The client can download an initial fragment of the selected representation. The client can access the content (e.g., by requesting the entire fragment or a byte range of the fragment). For example, the client can continue consuming media content after the start of rendering or during rendering. The client can request (e.g., continuously) media fragments and / or portions of media fragments during rendering. The client can play content according to the media rendering timeline. The client can switch from a first representation to a second representation, for example, based on updated information from the client's environment. The client can play the content continuously over two or more time periods, for example. Media rendering (e.g., consumed by the client in fragments) can be terminated, time periods can be started, and / or the MPD can be re-acquired, for example, when the media announced in the representation is about to end.

[0146] Changes in the volumetric content substream (e.g., changes in activity attributes and / or codecs) can be signaled to the decoder. For example, in a streaming session, bitrate adaptation processes may cause changes.

[0147] Multimedia applications such as virtual reality (VR) and immersive 3D graphics can be implemented using or represented by 3D point clouds, enabling updated forms of interaction and / or communication with the virtual world. Dynamic volumetric content (e.g., represented by point clouds) can generate large amounts of information. Efficient decoding algorithms can be used to compress volumetric content, for example, reducing the storage and / or transmission resource utilization of the volumetric content. For instance, a compressed bitstream of dynamic volumetric content can utilize fewer transmission resources than a bitstream of uncompressed information.

[0148] DASH can be used to carry (e.g., stream) volumetric content information / data. DASH signaling can support the streaming of DASH volumetric content data. The volumetric content bitstream can include multiple component substreams, which can increase the dimension for bitrate adaptation. Computing devices (e.g., DASH clients, such as smartphones) can have multiple video codec instances per codec. DASH clients (e.g., WTRUs) can, for example, select (e.g., dynamically) the decoded media version of the volumetric content components, based on or managing media processing capabilities or multiple capabilities. The DASH client can be a WTRU.

[0149] Dynamically selecting the decoded media version of volumetric content components can be based on, for example, the number of instances, resolution, and / or the codec types supported by the client. For instance, support for dynamic selection can be implemented using high-level syntax. For example, a V-PCC track can include a V-PCC Sequence Parameter Set (SPS), such as VPCC_SPS, which can include codec identifiers (IDs) for the component streams of the video decoding. For example, switching between different codecs can be achieved by using multiple VPCC_SPS units with different combinations of codec IDs for the component streams. Content creators can generate or not generate different V-PCC tracks that include different combinations of codecs and resolutions.

[0150] Some attribute components may not be used for volumetric content reconstruction. Bitrate adaptation (e.g., for use with streaming clients) may include, or can be achieved by, for example, discarding (e.g., not downloading or deactivating) attribute sub-bitstreams that may not be used for volumetric content reconstruction (e.g., point clouds). The number of attributes used for volumetric content reconstruction and their corresponding indices may be stored in a VPCC_SPS unit. The volumetric content decoder may (e.g., be configured to) handle changes in the number of attributes, for example, based on Supplemental Enhancement Information (SEI) messages (such as Component Codec Change (CCC) SEI messages), without reinitializing with a new SPS.

[0151] Visual media content processing devices (e.g., encoders or decoders) may discard attributes, which may include, for example, not downloading attributes or information about or associated with attributes. Decoders may skip or ignore inactive attributes, which may include, for example, not decoding, not assigning decoders, not expecting or receiving updated data, not using inactive attributes or information about or associated with inactive attributes in the video bitstream (e.g., inactive attribute sub-bitstreams) and / or the like.

[0152] Substream, sub-stream, and sub-bitstream are used interchangeably to refer to a portion of a bitstream. A substream can be associated with an attribute, and an attribute sub-bitstream can be referenced.

[0153] Attributes (e.g., indicated in the parameter set) can characterize volumetric content. Attributes can include, for example, scalar or vector properties that can be associated with points in a volumetric frame (e.g., each), such as color, transparency, reflectivity, texture, surface normal, timestamp, material ID, etc.

[0154] Attribute information can include information about attributes, such as the number of attributes, attribute index, attribute type, attribute ID, codec ID, attribute persistence, number of attribute dimensions or channels, attribute partitions, etc.

[0155] A parameter set can include attribute information that characterizes the volumetric content. A parameter set can include one or more parameters, such as V-PCC components (e.g., V-PCC cells, such as occupancy, geometry, or attribute data cells), codecs (e.g., occupancy, geometry, or attributes), geometry (e.g., coordinates of points in a point cloud), attributes (e.g., having states such as changed, active, or inactive), attribute information, etc. Parameter sets can be of any type or format, such as Sequence Parameter Sets (SPS), Component Change Parameter Sets (CCPS), Video-Based Point Cloud Compression Parameter Sets (VPS), etc.

[0156] SEI messages can support adapted streaming of volumetric content. For example, a Component Codec Change (CCC) SEI message can notify the volumetric content decoder of codec changes for one or more visual volumetric content components. A CCC SEI message can reference a volumetric content sequence parameter set. The codec ID change signaled in the CCC SEI message can be associated with (e.g., related to) the corresponding component that can be signaled in the referenced SPS unit. The volumetric content decoder can instantiate a video encoder (e.g., a new video decoder) for the corresponding component and codec ID signaled in the received CCC SEI message. Table 4 shows an example of the syntax for a CCC SEI message.

[0157] Messages (e.g., CCC SEI messages) may include prefixed SEI messages and / or may be carried within Patch Data Group (PDG) units of type PDG_PREFIX_SEI. For example, the payloadType value of a CCC SEI message may be set to 12. The duration of a CCC SEI message may include the remainder of the bitstream. For example, codec changes for components signaled by a signal may persist until the end of the stream or until a different CCC SEI message is encountered.

[0158] Bitrate adaptation may include a decision to switch (e.g., change or update) the representation of a volumetric content component to another representation. A volumetric content streaming client may make a decision to switch the representation of a visual volumetric content component to another representation. Different codecs may be used to decode the representations available for switching. For example, AVC may be used to encode the first representation and HEVC may be used to encode the second representation. The volumetric content streaming client may (e.g., based on a switching indication to another available representation) insert a VPCC_PDG unit including a CCC SEI message that includes the codec ID of one or more codecs for the indicated, selected, or chosen representation(s). The VPCC_PDG unit may be inserted before, for example, the volumetric content bitstream that will be sent to a decoding device (e.g., the decoder). For example, the streaming client may decide to change or update the representation of a volumetric content component to a representation encoded using a codec different from the representation specified in the SPS (e.g., V-PCC SPS). As described herein, for example, a change or update to the representation of a volumetric content component may be performed in response to limited network bandwidth during a streaming session. For example, if the available bandwidth is low during a streaming session (e.g., below a threshold), the representation of the volumetric content component can be changed or updated to a representation with a lower bit rate, which can be encoded using a different codec.

[0159] Table 4 - Syntax Examples of CCC SEI Messages

[0160] The semantics of fields in the CCC SEI message syntax (such as the example CCC SEI message syntax shown in Table 4) may include one or more of the following, for example.

[0161] Variables such as sps_id can indicate the ID of the visual volumetric content sequence parameter set (e.g., V-PCC sequence parameter set) associated with the CCC SEI message.

[0162] Variables such as occupancy_codec_change_flag can indicate whether the codec used to occupy a component has changed. For example, an occupancy_codec_change_flag value of 1 can indicate that the codec has changed.

[0163] Variables such as `geometry_codec_change_flag` can indicate whether the codec for a geometric component has changed. For example, a `geometry_codec_change_flag` value of 1 can indicate that the codec has changed.

[0164] Variables such as `attributes_codecs_change_flag` can indicate whether the codec used for one or more attribute components has changed. For example, an `attributes_codecs_change_flag` value of 1 can indicate that the codec used for at least one of the attributes has changed, while an `attributes_codecs_change_flag` value of 0 can indicate that no codec change has occurred for any of the attributes.

[0165] Variables such as occupancy_codec_id can indicate an identifier for a new or updated codec used to occupy mapping information. For example, occupancy_codec_id can be set to a value in the range of 0 to 255, including end values.

[0166] Variables such as geometry codec_id can indicate the identifier of a new or updated codec used for geometric information. For example, the geometry codec ID can be set to a value in the range of 0 to 255, including end values.

[0167] Variables such as `pcm_geometry_codec_change_flag` can indicate whether the codec used for the geometry of PCM decoding points has changed. For example, a `geometry_codec_change_flag` value of 1 can indicate that the codec has changed. PCM can stand for Pulse Decoding Modulation. In some instances, PCM can represent a point cloud map.

[0168] For example, if a variable such as pcm_geometry_codec_id exists, that variable could indicate an identifier for a new codec used for the geometry data of the PCM decode points (e.g., if the PCM decode points are encoded in a separate stream). The value of pcm_geometry_codec_id could, for example, range from 0 to 255, including end values.

[0169] Variables such as attribute_count_minus1,plus1 can indicate the number of attributes that have been changed by the codec, which is notified by a signal in the CCC SEI message.

[0170] Variables such as attribute_idx[i] can indicate the attribute index in the volume content sequence parameter set (e.g., V-PCC sequence parameter set) for the i-th attribute of the relevant SEI message.

[0171] Variables such as attribute_codec_change_flag[i] can indicate whether the codec used for the i-th attribute in an SEI message has changed. For example, an attribute_codec_change_flag[i] value of 1 can indicate that the codec has changed.

[0172] Variables such as attribute_codec_id[i] can indicate the identifier of a new codec used for the attribute video data with index i in the SEI message. For example, attribute_codec_id[i] can be in the range of 0 to 255, including end values.

[0173] Variables such as pcm_attribute_codec_change_flag[i] can indicate whether the codec for the attribute video data at the PCM decoding point of attribute i in the SEI message has changed. For example, a value of 1 for pcm_attribute_codec_change_flag[i] can indicate that the codec has changed.

[0174] For example, if a variable such as pcm_attribute_codec_id[i] exists, it can indicate the identifier of a new codec for the attribute data of the PCM decode point for attribute i in the SEI message (e.g., if the PCM decode point is encoded in a separate stream). For example, the value of pcm_attribute_codec_id[i] can be in the range of 0 to 255, including end values.

[0175] The Active Attribute (AA) SEI message can notify (e.g., indicate) the visual volumetric content decoder that, for example, one or more attributes have changed (e.g., in the V-PCC of the codec) for one or more of the visual volumetric content components. The AA SEI message can reference a specific visual volumetric content sequence parameter set. For example, if (e.g., only) a subset of attributes should be active, the AA SEI message can include the attribute index of the active attribute. The sequence parameter set decoder can ignore (e.g., consider it inactive) other attributes in the referenced SPS (e.g., attributes not listed in the AA SEI message). Table 5 shows an example of the syntax used for the AA SEI message.

[0176] An AA SEI message can be a prefix SEI message. An AA SEI message can be carried within a patch data group cell of type PDG_PREFIX_SEI. The payload type value of the AA SEI message can be, for example, 13. The duration of the AA SEI message can include, for example, the remainder of the bit stream. For example, activity attributes notified by a signal can persist until the end of the stream or until a subsequent AA SEI message is received.

[0177] Bitrate adaptation may include (e.g., may be implemented by) a volumetric content streaming client deciding to discard one or more attributes of the volumetric content. For example, due to limited network bandwidth, the volumetric content streaming client may decide to discard one or more attributes of the volumetric content. For example, if the bitrate adaptation process in the V-PCC streaming client decides to discard one or more attributes of the V-PCC content, the volumetric content stream may insert a VPCC_PDG unit containing an AA SEI message with a list of active attributes. For example, the volumetric content stream may insert a VPCC_PDG unit containing an AA SEI message with a list of active attributes before sending the volumetric content bitstream to the decoder. The decoder (e.g., which receives the VPCC_PDG unit) may skip the inactive attributes and / or may not expect volumetric content units in the bitstream associated with the inactive attributes.

[0178] Messages can convey information, for example, to aid in the decoding, reconstruction, and / or display of volumetric content. This information may include one or more of the following: attributes, attribute information, an indication of which attribute sub-bitstream, as indicated by a parameter set associated with the volumetric content, is active or inactive, an indication of which attribute, as indicated by attribute information in the parameter set, is active, an indication of message persistence (e.g., until the end of the bitstream or until another message is received), an indication of the number of active attributes identified in the parameter set, an indication of multiple active attributes in the parameter set, and / or an indication that an attribute is deactivated if the message does not reference an attribute indicated in the parameter set. Messages can be of any type or format, such as SEI messages (e.g., Component Codec Change (CCC) messages, Active Attribute (AA) messages, Parameter Set Activation (PSA) messages, etc.). Messages may refer to, belong to, or be associated with a parameter set (e.g., a V3C sequence parameter set). Messages may be included in a bitstream.

[0179] Table 5 - Examples of syntax for AA SEI messages

[0180] The semantics of fields in the AA SEI message syntax (such as the example AA SEI message syntax shown in Table 5) may include one or more of the following, for example.

[0181] Indicators such as sps_id can indicate the ID of a set of sequence parameters for visual volumetric content (e.g., V-PCC) associated with an AA SEI message.

[0182] Indicators such as all_attributes_active_flag can indicate whether an attribute signaled in the referenced SPS is active. For example, an all_attributes_active_flag value of 1 can indicate that all attributes are active. An all_attributes_active_flag value of 0 can indicate that a subset of the attributes is active.

[0183] Variables such as attribute_count_minus1,plus1 can indicate the number of active attributes that are signaled in the AA SEI message.

[0184] Variables such as attribute_idx[i] can indicate the index in the associated AA SEI message used in the V-PCC SPS. i The attribute index of the activity attribute at that location.

[0185] The semantics of the fields in the AA SEI message syntax in Table 6 are merely examples. A message (e.g., the AA SEI message) may include less information. For example, the message may not include the indicated sps_id. A message (e.g., the AASEI message) may include more information (e.g., activity layer or mapping information indicated in the parameter set).

[0186] The component change parameter set (CCPS) can include information about changes made to the component substream, for example, relative to the volume sequence parameter set (e.g., V-PCCSPS). The CCPS can include information about codec changes and / or activity attributes. The CCPS can be carried, for example, in a V-PCC cell of a specific type. Table 6 shows examples of the CCPS syntax.

[0187] Table 6 - CCPS Syntax Examples

[0188]

[0189] The semantics of fields in CCPS syntax (e.g., examples of CCPS message syntax shown in Table 6) may include one or more of the following, for example.

[0190] Variables such as ccps_component_change_parameter_set_id can provide identifiers for CCPS so that they can be referenced by other syntax elements. For example, the value of ccps_component_change_parameter_set_id can be in the range of 0 to 255, including end values.

[0191] Variables such as ccps_sps_id can indicate the ID of a sequence parameter set of volumetric content (e.g., V-PCC) associated with CCPS.

[0192] Variables such as ccps_component_codec_change_flag can indicate whether a codec change has occurred for one or more of the volumetric content (e.g., V-PCC) components. For example, a ccps_component_codec_change_flag value of 1 can indicate that a codec change has occurred.

[0193] Variables such as ccps_active_attributes_change_flag can indicate whether the active attribute set has changed. For example, a value of 1 for ccps_active_attributes_change_flag can indicate that a change has occurred in that active attribute set.

[0194] Variables such as ccps_occupancy_codec_change_flag can indicate whether the codec used to occupy components has changed. For example, a value of 1 for ccps_occupancy_codec_change_flag indicates that the codec has changed.

[0195] Variables such as ccps_geometry_codec_change_flag can indicate whether the codec used for geometric components has changed. For example, a value of 1 for ccps_geometry_codec_change_flag can indicate that the codec has changed.

[0196] Variables such as `ccps_attributes_codecs_change_flag` can indicate whether the codecs used for one or more attribute components have changed. For example, a `ccps_attributes_codecs_change_flag` value of 1 can indicate that the codecs for at least one of the said attributes have changed. A `ccps_attributes_codecs_change_flag` value of 0 can indicate that no codec change has occurred for any of the said attributes.

[0197] Variables such as ccps_occupancy_codec_id can indicate the identifier of a new codec used to occupy mapping information. For example, the value of occupancy_codec_id can be in the range of 0 to 255, including end values.

[0198] Variables such as ccps_geometry_codec_id can indicate the identifier of a new codec used for geometric information. For example, the value of geometry_codec_id can be in the range of 0 to 255, including end values.

[0199] Variables such as ccps_pcm_geometry_codec_change_flag can indicate whether the codec used for the geometric video data at PCM decoding points has changed. For example, a value of 1 for ccps_pcm_geometry_codec_change_flag can indicate that the codec has changed.

[0200] For example, if a variable such as ccps_pcm_geometry_codec_id exists, it could indicate an identifier for the new codec used for the geometric video data of the PCM decoding points, which could happen, for example, if the PCM decoding points were encoded in a separate video stream. The value of pcm_geometry_codec_id could, for example, range from 0 to 255, including the end values.

[0201] For example, the variable ccps_codec_change_attribute_count_minus1,plus1 can indicate the number of attributes that the codec in CCPS has changed, which are signaled to it.

[0202] Variables such as ccps_codec_change_attribute_idx[i] can indicate the attribute index in the V-PCC SPS for the attribute at index i in the CCPS's codec_change_information() struct.

[0203] Variables such as ccps_attribute_codec_change_flag[i] can indicate whether the codec for the i-th attribute in the codec_change_information() struct used by CCPS has changed. For example, a value of 1 for ccps_attribute_codec_change_flag[i] can indicate that the codec used for the i-th attribute has changed.

[0204] Variables such as ccps_attribute_codec_id[i] can indicate an identifier for a new codec that has attribute data at index i of the codec_change_information() struct in CCPS. For example, the value of attribute_codec_id[i] can be in the range of 0 to 255, including end values.

[0205] Variables such as ccps_pcm_attribute_codec_change_flag[i] can indicate whether the codec for the attribute video data of the PCM decoding point at attribute i in the codec_change_information() struct used by CCPS has changed. For example, a value of 1 for ccps_pcm_attribute_codec_change_flag[i] can indicate that the codec has changed.

[0206] For example, if a variable `ccps_pcm_attribute_codec_id[i]` exists, then, for instance, if the PCM decode point is encoded in a separate stream, this variable can indicate the identifier of the new codec used for the attribute data of the PCM decode point of attribute `i` in the `codec_change_information()` struct of `CCPS`. For example, the value of `pcm_attribute_codec_id[i]` can be in the range of 0 to 255, including end values.

[0207] Variables such as `ccps_all_attributes_active_flag` can indicate whether an attribute signaled in the referenced SPS is active. For example, a `ccps_all_attributes_active_flag` value of 1 indicates that all attributes are active. A `ccps_all_attributes_active_flag` value of 0 indicates that a subset of the attributes is active.

[0208] Variables such as ccps_active_attribute_count_minus1, plus1 can indicate the number of active attributes that are signaled in the CCPS.

[0209] Variables such as ccps_active_attribute_idx[i] can indicate the attribute index in the volume contents (e.g., V-PCC) SPS, for the active attribute at index i in the active_attribute_information() struct of CCPS.

[0210] The Parameter Set Activation (PSA) SEI message can indicate which parameter sets are active for volume content (e.g., V-PCC) cells that may follow the PSA SEI message. Table 7 shows an example of the syntax for the PSA SEI message.

[0211] Table 7 - Examples of PSA SEI message syntax

[0212] The semantics of fields in the PSA SEI message syntax (such as the example PSA SEI message syntax shown in Table 7) may include one or more of the following, for example.

[0213] Variables such as active_sequence_parameter_set_id can indicate and / or be equal to the value of sps_sequence_parameter_set_id of the active volume content SPS (e.g., the V-PCC SPS that should be activated). For example, the value of active_sequence_parameter_set_id can be in the range of 0 to 15, including the end values.

[0214] Variables such as component_change_flag can indicate whether a component change parameter set should be activated. For example, a component_change_flag value of 1 indicates that a component change parameter set should be activated.

[0215] Variables such as active_component_change_parameter_set_id can indicate and / or be equal to the value of ccps_component_change_parameter_set_id of the CCPS to be activated. The value of active_component_change_parameter_set_id can be in the range of 0 to 255, including the end values.

[0216] PSA SEI messages (e.g., as shown in Table 7, which may be generated by example encoder 200 and received by example decoder 300) may indicate which sets of parameters associated with volumetric content (e.g., V-PCC) are active for V-PCC components. CCCSEI messages (e.g., generated by example encoder 200 and received by example decoder 300) may indicate which codec should be used for volumetric content (e.g., V-PCC). Figure 5 The volumetric content components (e.g., in the V-PCC bitstream shown) Figure 5 Decode the V-PCC units shown in Tables 1, 2 and 3.

[0217] Figure 7 An example of a method for processing visual volumetric content based on one or more messages is shown. Figure 7 The methods described herein can be executed by the example encoder or the example client device. The examples and other examples disclosed herein can be adapted from... Figure 7 The example method 700 shown herein operates as follows. Method 700 includes 702 and 704. In 702, a determination can be made to deactivate an attribute sub-bitstream in one or more attribute sub-bitstreams indicated in a parameter set associated with the visual volumetric content. In 704, a message indicating the deactivation of the attribute sub-bitstream can be generated and sent. Example method 700 can be implemented, for example, by an encoder such as example encoder 200. Example method 700 can be implemented, for example, by a streaming client such as a visual volumetric content streaming client. Example method 700 can be implemented, for example, according to the example syntax and semantics described herein for visual volumetric bitstream SEI messages and / or parameter sets, which include, for example, CCC messages, AA messages, CCPS and / or PSA messages.

[0218] Figure 8 An example of a method for processing visual volumetric content based on one or more messages is shown. Figure 8 The methods described herein can be applied to decoders. The examples and other examples disclosed herein can be used as a basis for further study. Figure 8The example method 800 shown herein operates as follows. Method 800 includes 802 and 804. In 802, it is possible to determine which attribute sub-bitstream to use to decode the visual volume content based on a message indicating which attribute sub-bitstream among one or more attribute sub-bitstreams indicated in a parameter set associated with the visual volume is active. In 804, it is possible to decode the visual volume content using the active attribute sub-bitstream (e.g., the active attribute sub-bitstream determined in 802) based on the message. Example method 800 can be implemented, for example, by a decoder such as example decoder 300. Example method 800 can be implemented, for example, by a media content decoder such as a visual volume content decoder. For example, example method 800 can be implemented according to the example syntax and semantics for visual volume bitstream SEI messages and / or parameter sets described herein, which include, for example, CCC messages, AA messages, CCPS and / or PSA messages.

[0219] This document describes numerous embodiments. Features of the embodiments may be provided individually or in any combination across various claim classes and types. Furthermore, embodiments may span various claim classes and types, individually or in any combination including one or more of the features, devices, or aspects described herein, such as any of the following.

[0220] The decoder can decode media content based on one or more messages, which may indicate which of one or more attribute sub-bitstreams indicated in a parameter set is active. The media content may include, for example, visual volumetric content. The parameter set may be associated with the visual volumetric content. For example, the parameter set may include a parameter set based on visual volumetric video. The decoder can perform decoding based on the one or more messages, such as determining which attribute sub-bitstream to use to decode the visual volumetric content. The media content decoder (e.g., based on...) Figure 8 The example decoder 300 shown in the example method operation can determine which attribute sub-bitstream to use to decode the visual volumetric content based on a message indicating which attribute sub-bitstream in a parameter set associated with the media content is active. The media content decoder (e.g., according to...) Figure 8 The example decoder 300 shown in the example method operation can decode the visual volume content based on the message using the active attribute sub-bitstream.

[0221] The attribute sub-bitstreams indicated in the parameter set(s) may indicate information about the visual volumetric content. The attribute sub-bitstreams may include sub-bitstreams associated with attributes characterizing the visual volumetric content. The attribute sub-bitstreams may include attributes characterizing the media content, such as color, transparency, reflectivity, texture, etc.

[0222] The message may include, for example, a visual volumetric SEI message, such as an AA SEI message (e.g., as shown in Table 5), which may reference a visual volumetric SPS and may indicate one or more attribute indices of the active attribute(s). A message indicating a list of active attribute sub-bitstreams may be received by the decoder. The message may be generated and sent to the decoder, for example, to indicate the deactivation of the one or more attribute sub-bitstreams. The decoder may determine inactive attribute sub-bitstreams based on the message. The decoder may use the attribute sub-bitstreams indicated as active in the message (e.g., the AA SEI message) to decode the visual volumetric content. The decoder may skip attribute sub-bitstreams indicated as inactive in the message to decode the visual volumetric content.

[0223] Decoding tools and techniques, including one or more of entropy decoding, inverse quantization, inverse transform, and differential decoding, can be used to implement, for example, in the decoder. Figure 8 The methods described herein. These decoding tools and techniques can be used to achieve one or more of the following: according to Figure 8 The methods described herein or other methods described herein receive media content such as visual volumetric content, according to Figure 8 Decoding media content using the methods described herein or other methods described herein; according to, for example Figure 8 The method described herein or otherwise described herein determines which attribute sub-bitstream to use for decoding media content, such as visual volumetric content; based on Figure 8 The method described herein or otherwise described herein receives and parses messages, such as SEI messages, including AA SEI messages, CCC SEI messages, and PSA SEI messages; according to... Figure 8 The method described herein or otherwise includes receiving and parsing a set of parameters, such as SPS and CCPS, that indicate mapping information associated with attribute sub-bitstreams, wherein the active mapping information is indicated by the indicated active attribute sub-bitstream(s); according to Figure 8 The methods described herein, or other methods described herein, decode visual volumetric content using an active attribute sub-bitstream determined based on messages and / or parameter sets, or active mapping information associated with the active attribute sub-bitstream; according to Figure 8The methods described herein or other methods described herein determine the persistence of a message, such as until the end of the bitstream or until another message arrives; according to, for example Figure 8 The method described in or otherwise herein determines the number of active attribute sub-bitstreams in one or more attribute sub-bitstreams indicated by a parameter set associated with visual volumetric content, said parameter set including, for example, a parameter set of a VPS having attribute information referenced by messages to indicate active attribute sub-bitstreams; according to, as Figure 8 The method described herein, or otherwise described herein, determines, based on indicators in the message, that one or more attribute sub-bitstreams in a parameter set associated with the visual volumetric content are active; according to Figure 8 The methods described herein or other methods described herein determine that an attribute sub-bitstream is inactive (e.g., based on the attribute sub-bitstream being indicated in a parameter set rather than in the message, or, for example, based on the message not referencing the attribute sub-bitstream or an indicator of the attribute sub-bitstream, or, for example, based on bitrate adaptation), and skip the inactive attribute sub-bitstream to decode the visual volumetric content; and other decoder behaviors related to any of the foregoing.

[0224] An encoder (e.g., an encoder as described herein) can encode media content, for example, to generate visual volumetric content that can be sent as a bitstream in a streaming service. The encoder or a client device (e.g., an application) can generate and send one or more messages indicating which of one or more attribute sub-bitstreams indicated in a parameter set is active. The parameter set can be associated with the visual volumetric content. For example, the parameter set may include a visual volumetric video parameter set (VPS).

[0225] An encoder or client device (e.g., an application) may determine, for example, which of the one or more attribute sub-bitstreams indicated in a parameter set is active or inactive based on an assessment of the relevance of the attribute sub-bitstream, operating environment, resource availability, bandwidth attributes, client decoding capabilities, and / or client rendering capabilities. In one example, the encoder or client device may determine which of the one or more attribute sub-bitstreams indicated in a parameter set will be deactivated based on bitrate adaptation in a streaming session. The encoder or client device may indicate the deactivation of one or more attributes based on the determination of which attribute sub-bitstreams indicated in the parameter set will be deactivated. Figure 7The media content processor (e.g., an encoder, such as example encoder 200) operating the example method shown can determine to deactivate one attribute sub-bitstream in one or more attribute sub-bitstreams indicated in a parameter set associated with visual volumetric content. The media content processor may include a streaming client device as described herein.

[0226] according to Figure 7 The media content processor (e.g., an encoder, such as example encoder 200) operating the example method shown can generate a message indicating deactivation of one or more attribute sub-bitstreams and send it to, for example, a decoder. This message may include, for example, a visual volumetric SEI message, such as an AA SEI message (e.g., as shown in Table 5), which may reference a VPS and may indicate one or more attribute indices for (one or more) active attributes. Messages with one or more inactive and / or active attribute sub-bitstreams can be sent by the encoder to the decoder.

[0227] Encoding tools and techniques, including one or more of quantization, entropy coding, inverse quantization, inverse transform, and differential coding, can be used to implement, in encoders, such as Figure 7 The methods described herein. These coding tools and techniques can be used to implement one or more of the following: according to Figure 7 The methods described herein or other methods described herein are used to generate or transmit media content such as visual volumetric content; according to Figure 7 The methods described herein or other methods described herein are used to encode media content; according to Figure 7 The methods described herein or other methods described herein determine which attribute sub-bitstream to use for encoding media content, such as visual volumetric content; based on Figure 7 The methods described herein or other means described herein generate and send messages, such as SEI messages, including AA SEI messages, CCC SEI messages, and PSA SEI messages; according to, Figure 7 The method described herein, or otherwise described herein, generates and transmits a set of parameters, such as SPS and CCPS, that can indicate mapping information associated with attribute sub-bitstreams, wherein active mapping information is indicated by the indicated active attribute sub-bitstream(s); according to Figure 7 The methods described herein, or other methods described herein, use activity attribute sub-bitstreams or activity mapping information associated with activity attributes to encode visual volumetric content; according to Figure 7 The methods described herein, or other methods described herein, indicate the persistence of a message, such as until the end of the bitstream or until another message arrives; according to, for example... Figure 7The method described in or otherwise herein indicates the number of active attribute sub-bitstreams in one or more attribute sub-bitstreams indicated in a parameter set associated with visual volumetric content (e.g., a parameter set of a VPS having attribute information referenced by messages to indicate active attribute sub-bitstreams); according to... Figure 7 The method described herein, or otherwise described herein, indicates in a message that one or more attribute sub-bitstreams in a parameter set associated with the visual volumetric content are active; according to Figure 7 The methods described herein or other methods described herein indicate that an attribute sub-bitstream is inactive (e.g., based on the attribute sub-bitstream being indicated in a parameter set rather than in a message, or, for example, based on the message not referencing the attribute sub-bitstream or its indicator, or, for example, based on bitrate adaptation), and discard or skip the inactive attribute sub-bitstream to encode visual volumetric content; and other encoder behaviors associated with any of the foregoing.

[0228] One or more syntax elements (such as those shown in Tables 1-7) may be inserted into the signaling, for example, to enable the decoder to recognize instructions regarding active attribute sub-bitstreams and / or codecs to perform actions such as Figure 8 The decoding method described herein. For example, the syntax element may include indications of one or more of the following: attribute sub-bitstream, attribute sub-bitstream ID, parameter set, attribute sub-bitstream change indication, attribute codec ID, attribute activity indication, attribute sub-bitstream inactivity indication, active attribute count indication, active parameter set ID, and active parameter set change indication, for example, to indicate to the decoder whether one or more of them are active or inactive for decoding. As an example, the syntax element may include indications of one or more of the following: attribute index, attribute state (e.g., active, inactive, or change flag), and / or attribute count as described herein, and / or indications of parameters used by the decoder to perform one or more of the examples herein.

[0229] It is possible, for example, to select and / or apply based on (one or more) syntax elements to be applied at the decoder. Figure 8 The method described herein. For example, the decoder may receive an indication (e.g., in a message or parameter set) indicating a change in the active / inactive state of the attribute sub-bitstream and / or codec to be used to decode the visual volumetric content. Based on this indication, the decoder may select, as described above... Figure 8 The attribute sub-bitstreams and / or codecs described herein are used for decoding visual volumetric components.

[0230] A bitstream or signal may contain one or more of the following: a described syntax element or a variation thereof. For example, a bitstream or signal may contain one or more syntax elements that indicate information about performing, such as Figure 8 The active attribute sub-bitstream and / or codec indication of the decoding method described herein.

[0231] Bitstreams or signals may contain syntax that conveys information generated according to one or more examples in this document. For example, in the execution of... Figure 7 and Figure 8 The example shown (included in) Figure 7 and Figure 8 Information or data is generated when any example shown in this article is within the scope of the examples described herein.

[0232] Syntax elements can be inserted into the signal that enable the decoder to decode the visual volumetric components using one or more active attribute sub-bitstreams and one or more codecs in a manner corresponding to that used by the encoder. For example, one or more examples in this paper can be used to generate one or more messages and / or parameter sets that indicate one or more attribute sub-bitstreams and one or more codecs for decoding.

[0233] A method, process, apparatus, medium for storing instructions, medium for storing data, or signal for creating and / or sending and / or receiving and / or decoding a bitstream or signal comprising one or more described syntax elements or variations thereof.

[0234] A method, process, apparatus, medium for storing instructions, medium for storing data, or signal for creating and / or sending and / or receiving and / or decoding, according to any of the examples described.

[0235] A method, process, apparatus, medium for storing instructions, medium for storing data, or signal, based on, but not limited to, one or more of, in any number or combination of: processing, decoding, or decoding media content; performing dynamic adaptation of point cloud component sub-bitstreams in a point cloud streaming service; obtaining an indication indicating whether at least one attribute signaled in a referenced parameter set is inactive; determining to deactivate attribute sub-bitstreams in one or more attribute sub-bitstreams indicated in the parameter set associated with visual volumetric content; generating and sending a message indicating the deactivation of the attribute sub-bitstreams; obtaining an active attribute indication (e.g., the number of active attributes and their corresponding attribute indices), which may occur In cases where, for example, the indication indicates that at least one attribute in the referenced parameter set is inactive; the inactive attribute is identified, for example, based on the active attribute indication; during decoding, the inactive attribute in the referenced parameter set is skipped; a parameter set associated with the visual volumetric content is obtained; a message indicating which of one or more attribute sub-bitstreams indicated in the parameter set is active is received; based on the message, the active and inactive attribute sub-bitstreams are determined; based on a message indicating which of one or more attribute sub-bitstreams indicated in the parameter set associated with the visual volumetric content is active, the appropriate attribute sub-bitstream is determined to be used for the visual volumetric content. The visual volume content is decoded; based on the message, the visual volume content is decoded using an active attribute sub-bitstream; the visual volume content is decoded using the active attribute sub-bitstream, and the inactive attribute sub-bitstream is skipped; messages, such as SEI messages, are signaled or received in the bitstream; messages with a persistent range are signaled or received, the persistent range lasting until the end of the bitstream or until another message different from the message is received; messages with an indicator are signaled or received, the indicator indicating the number of active attribute sub-bitstreams in the plurality of attribute sub-bitstreams indicated in the parameter set associated with the visual volume content; messages including V are signaled or received. A parameter set for a VPS, the VPS including attribute information; signaling or receiving a message relating to an attribute information in the VPS of the active attribute sub-bitstream; signaling or receiving a message including an indicator indicating that the plurality of attribute sub-bitstreams indicated in the parameter set are active; signaling or receiving a parameter set indicating mapping information associated with the corresponding attribute sub-bitstreams of the plurality of attributes; signaling or receiving a message indicating which mapping information is active, which may be done, for example, by indicating which attribute sub-bitstream among the plurality of attribute sub-bitstreams indicated in the parameter set is active; using the active mapping information associated with the active attribute sub-bitstream to decode visual volumetric content;Notify and / or receive one or more attribute sub-bitstreams, said attribute sub-bitstreams indicating, for example, texture information, material information, transparency information, and / or reflectivity information associated with or characterizing the visual volumetric content; determine inactive attribute sub-bitstreams based on messages; determine attribute sub-bitstreams indicated in the parameter set but not indicated in the message as inactive sub-bitstreams; skip inactive attribute sub-bitstreams for decoding the visual volumetric content; indicate or receive indications for deactivation of attribute sub-bitstreams in messages; determine that an attribute sub-bitstream is deactivated if no attribute sub-bitstream or its indicator is referenced in the message; and / or determine that an attribute sub-bitstream is deactivated based on bitrate adaptation.

[0236] A TV, set-top box, cellular phone, tablet computer, or other electronic device that performs dynamic adaptation of visual volumetric content, such as visual volumetric component quantum bitstreams in a visual volumetric streaming service, according to any of the examples described herein.

[0237] A TV, set-top box, cellular phone, tablet computer, or other electronic device, according to any of the examples described herein, performs dynamic adaptation of visual volumetric content and displays (e.g., using a monitor, screen, or other type of display) the resulting visual representation, such as a visual volumetric quantum bitstream in a visual volumetric streaming service.

[0238] A TV, set-top box, cellular phone, tablet computer, or other electronic device, according to any example described herein, selects (e.g., using a tuner) a channel to receive signals including encoded volumetric frames and performs dynamic adaptation of visual volumetric content, such as visual volumetric component quantum bitstreams in a visual volumetric streaming service.

[0239] A TV, set-top box, cellular phone, tablet computer, or other electronic device, according to any example described herein, receives over the air (e.g., using an antenna) a signal comprising encoded volumetric frames and performs dynamic adaptation of visual volumetric content, such as visual volumetric component quantum bitstreams in a visual volumetric streaming service.

[0240] Although the features and elements described above are in specific combinations, those skilled in the art will understand that each feature or element can be used alone or in any combination with other features and elements. Furthermore, the methods described herein can be implemented in computer programs, software, or firmware embedded in a computer-readable medium and executed by a computer or processor. Examples of computer-readable media include electronic signals (transmitted via wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, buffer memory, semiconductor storage devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM discs and digital multipurpose discs (DVDs). The processor associated with the software can be used to implement a radio frequency transceiver used in a WTRU, UE, terminal, base station, RNC, or any host computer.

Claims

1. An apparatus for processing visual volumetric content, comprising one or more processors, said one or more processors being configured to: Obtain signaling information associated with the visual volume content, wherein the signaling information identifies multiple attribute sub-bit streams; Receive a message indicating at least one attribute sub-bitstream from the plurality of attribute sub-bitstreams for processing the visual volumetric content; as well as The visual volumetric content is processed based on at least one indicated attribute sub-bit stream.

2. The apparatus of claim 1, wherein the message includes a Supplemental Enhancement Information (SEI) message.

3. The apparatus of claim 2, wherein the SEI message is signaled in the video data.

4. The apparatus of claim 2, wherein the SEI message has a duration that lasts until the end of the video data.

5. The apparatus of claim 2, wherein the SEI message has a duration that continues until another SEI message different from the SEI message is received.

6. The apparatus of claim 2, wherein the SEI message includes an indicator indicating the number of active attribute sub-bitstreams among the plurality of attribute sub-bitstreams.

7. The apparatus of claim 2, wherein the signaling information includes a parameter set, wherein the parameter set includes a visual volumetric video parameter set (VPS), the VPS including attribute information for the plurality of attribute sub-bitstreams, and wherein the SEI message references the attribute information associated with the plurality of attribute sub-bitstreams.

8. The apparatus according to claim 1, wherein the apparatus is an encoder.

9. The apparatus of claim 1, wherein the apparatus is a decoder.

10. A method for decoding media content, comprising: Obtain signaling information associated with visual volumetric content, wherein the signaling information identifies multiple attribute sub-bit streams; Receive a message indicating at least one attribute sub-bit stream from the plurality of attribute sub-bit streams for processing the visual volumetric content; as well as The visual volumetric content is processed based on at least one indicated attribute sub-bit stream.

11. The method of claim 10, wherein the message includes a Supplemental Enhancement Information (SEI) message.

12. The method of claim 11, wherein the SEI message has a duration that lasts until the end of the video data.

13. The method of claim 11, wherein the SEI message has a duration that lasts until another SEI message different from the SEI message is received.

14. The method of claim 11, wherein the SEI message includes an indicator indicating the number of active attribute sub-bitstreams among the plurality of attribute sub-bitstreams.

15. The method of claim 11, wherein the signaling information includes a parameter set, wherein the parameter set includes a visual volumetric video parameter set (VPS), the VPS including attribute information for the plurality of attribute sub-bitstreams, wherein the SEI message references the attribute information associated with the plurality of attribute sub-bitstreams.

16. A base station, comprising one or more processors and a wireless communication interface, the base station being configured to: Using radio access technology to provide wireless connectivity for one or more wireless transmit / receive units (WTRUs) within a local area; and Data is routed between the one or more WTRUs and the external data network via a direct connection to the external data network, without routing the data through the core network.

Images