Streaming of volumetric point cloud content based on Session Description Protocol and Real-time Protocol

Abstract

The present disclosure describes a process for real-time protocol (RTP) data streaming. A process includes generating a plurality of codec bitstreams, each codec bitstream configured to carry encoded information, the encoded information including one of occupancy map data, geometry data, or attribute data for a volumetric video. The process includes generating a dedicated RTP stream for each of the plurality of codec bitstreams, the dedicated RTP stream including an RTP payload format based on a corresponding codec bitstream for the RTP stream. The process includes multiplexing each dedicated RTP stream into a resulting RTP stream for the volumetric video.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 874,873, filed on July 16, 2019, entitled “SDP AND RTP-BASED METHODS FORSTREAMING OF VOLUMETRIC POINT CLOUD CONTENT,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates generally to wireless communications, including protocols for streaming data. Background Art

[0004] The use of wireless communication systems is growing rapidly. In addition, wireless communication technology has evolved from voice-only communications to also include the transmission of data (such as the Internet and multimedia content) to various devices. In order to accommodate the increasing number of devices that transmit both voice and data signals, many wireless communication systems share available communication channel resources between devices. Summary of the invention

[0005] This specification describes a Session Description Protocol (SDP)-based procedure and a Real-time Protocol (RTP)-based procedure to support streaming of volumetric video content. In general, volumetric content distribution can be configured to deliver a 6-degree-of-freedom (6DoF) immersive media experience. Real-time delivery protocols (such as RTP) and session management protocols (such as SDP) are updated to provide a method for sharing volumetric content in real time.

[0006] In a general aspect, a method for real-time protocol (RTP) data streaming includes generating a plurality of codec bitstreams, each codec bitstream configured to carry encoded information including one of occupancy map data, geometry data, or attribute data for volumetric video. The method includes generating a dedicated RTP stream for each of the plurality of codec bitstreams, the dedicated RTP stream including an RTP payload format based on a corresponding codec bitstream for the RTP stream. The method includes multiplexing each dedicated RTP stream into a resulting RTP stream for volumetric video.

[0007] In one embodiment, the method includes performing video-based point cloud coding (V-PCC) using the resulting RTP stream.In one embodiment, each codec bitstream includes a High Efficiency Video Coding (HEVC) bitstream.

[0008] In one embodiment, the method comprises generating an additional dedicated RTP stream comprising the compressed patch sequence substream. In one embodiment, the method comprises multiplexing the additional RTP stream into the resulting RTP stream for the V-PCC video.

[0009] In one embodiment, the method includes generating an RTP header extension of the resulting RTP stream, the RTP header extension configured to include the compressed patch sequence sub-stream.

[0010] In one embodiment, the method includes generating one or more Session Description Protocol (SDP) attributes to indicate the ability to carry point cloud video as part of the resulting RTP stream.

[0011] In one embodiment, the one or more SDP attributes of the resulting RTP stream indicate at least one of: the ability to carry encoded point cloud information about occupancy map data, geometry data, attribute data, and patch sequence data in a dedicated RTP stream; the ability to carry a dedicated RTP stream about point cloud information such as occupancy map data, geometry data, attribute data, and patch sequence data in a single multiplexed RTP stream; the ability to carry sequence parameter set (SPS) unit type information in an RTP header extension; the ability to carry point cloud metadata information as part of the RTP payload format; and the ability to support codec capabilities to meet minimum quality requirements for point cloud video.

[0012] In one embodiment, the RTP payload format includes at least one Supplementary Information Enhancement (SEI) message, wherein the data carried in the point cloud video specific SEI message is used at the receiver for the rendering stage.

[0013] In one embodiment, the one or more SDP attributes indicate at least one of the following: the ability to use RTCP feedback messages to signal the desired viewport; and the ability to use RTP header extension messages to signal the actual transmitted viewport. In one embodiment, each dedicated RTP stream includes a different media type.

[0014] In a general aspect, a user equipment (UE) is configured for real-time protocol (RTP) data streaming. The UE includes a processing circuit configured to: generate a plurality of codec bitstreams, each codec bitstream configured to carry encoded information, the encoded information including one of occupancy map data, geometry data, or attribute data for volumetric video. The processing circuit is configured to generate a dedicated RTP stream for each of the plurality of codec bitstreams, the dedicated RTP stream including an RTP payload format based on a corresponding codec bitstream for the RTP stream. The processing circuit is configured to multiplex each dedicated RTP stream into a result RTP stream for volumetric video. The UE includes a baseband circuit coupled to the processing circuit, the baseband circuit configured to generate a result RTP stream for transmission to a networked device.

[0015] In one embodiment, the processing circuit is further configured to perform video-based point cloud coding (V-PCC) using the resulting RTP stream.In one embodiment, each codec bitstream comprises a High Efficiency Video Coding (HEVC) bitstream.

[0016] In one embodiment, the processing circuit is further configured to generate an additional dedicated RTP stream, the additional dedicated RTP stream comprising a compressed patch sequence substream. In one embodiment, the processing circuit is further configured to multiplex the additional RTP stream into the result RTP stream for the V-PCC video.

[0017] In one embodiment, the processing circuit is further configured to generate an RTP header extension of the resulting RTP stream. The RTP header extension is configured to include the compressed patch sequence sub-stream.

[0018] In one embodiment, the processing circuit is further configured to generate one or more Session Description Protocol (SDP) attributes to indicate the ability to carry point cloud video as part of the resulting RTP stream.

[0019] In one embodiment, the one or more SDP attributes of the resulting RTP stream indicate at least one of: the ability to carry encoded point cloud information about occupancy map data, geometry data, attribute data, and patch sequence data in a dedicated RTP stream; the ability to carry a dedicated RTP stream about point cloud information such as occupancy map data, geometry data, attribute data, and patch sequence data in a single multiplexed RTP stream; the ability to carry sequence parameter set (SPS) unit type information in an RTP header extension; the ability to carry point cloud metadata information as part of the RTP payload format; and the ability to support codec capabilities to meet minimum quality requirements for point cloud video.

[0020] In one embodiment, the RTP payload format includes at least one Supplementary Information Enhancement (SEI) message. The data carried in the point cloud video specific SEI message is used at the receiver for the rendering stage.

[0021] In one embodiment, the one or more SDP attributes indicate at least one of the following: the ability to use RTCP feedback messages to signal the desired viewport; and the ability to use RTP header extension messages to signal the actual transmitted viewport. In one embodiment, each dedicated RTP stream includes a different media type.

[0022] In some embodiments, the process is performed by a network element, a UE, or a base station, such as a next generation node (gNB). In some implementations, one or more non-transitory computer-readable media store instructions that, when executed by at least one processing device, cause the at least one processing device (or another device in communication with the at least one processing device) to perform the process.

[0023] In a general aspect, a method for real-time protocol (RTP) data streaming includes: demultiplexing an RTP stream representing a volumetric video into a plurality of codec bitstreams. The method includes decoding texture data from the plurality of codec bitstreams, the texture data including occupancy map data, geometry data, and attribute data for the volumetric video. The method includes generating a point cloud reconstruction for the volumetric video from the texture data. The method includes rendering the volumetric video based on the point cloud reconstruction.

[0024] In an embodiment, the method includes decoding one or more supplemental information enhancement (SEI) messages from the plurality of codec bitstreams, the supplemental information enhancement messages including metadata for rendering the volumetric video.

[0025] In an embodiment, the method includes decoding patch sequence data from the plurality of encoded bitstreams, wherein generating the point cloud reconstruction is based on the patch sequence data.

[0026] In an embodiment, rendering the volumetric video comprises: receiving sensor data indicating a position of a display from a sensor associated with a device configured to display the volumetric video; and rendering the volumetric video based on the sensor data indicating the position. In an embodiment, each codec bitstream comprises a High Efficiency Video Coding (HEVC) bitstream.

[0027] In an embodiment, the method includes receiving an RTP header extension of an RTP stream, the RTP header extension being configured to include a compressed patch sequence substream. In an embodiment, the method includes receiving data including one or more Session Description Protocol (SDP) attributes to indicate a capability to carry a point cloud video as part of the RTP stream.

[0028] In an embodiment, one or more SDP attributes of the RTP stream indicate at least one of the following: the ability to carry encoded point cloud information about occupancy map data, geometry data, attribute data, and patch sequence data in a dedicated RTP stream; the ability to carry a dedicated RTP stream about point cloud information such as occupancy map data, geometry data, attribute data, and patch sequence data in a single multiplexed RTP stream; the ability to carry sequence parameter set (SPS) unit type information in an RTP header extension; the ability to carry point cloud metadata information as part of the RTP payload format; and the ability to support codec capabilities to meet minimum quality requirements for point cloud video.

[0029] In an embodiment, the one or more SDP attributes indicate at least one of: the ability to signal a desired viewport using an RTCP feedback message; and the ability to signal an actual transmitted viewport using an RTP header extension message.

[0030] In an embodiment, each codec bitstream includes a different media type.

[0031] In an embodiment, the method is performed by a user equipment (UE), a networking device, or an access node (AN).

[0032] The details of one or more specific implementations are set forth in the drawings and description below. Other features and advantages will be apparent from the detailed description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 An exemplary wireless communication system according to various embodiments herein is shown.

[0034] Figure 2 Examples of platforms or devices according to various embodiments herein are shown.

[0035] Figure 3 Exemplary protocol functions that may be implemented in a wireless communication system according to various embodiments herein are shown.

[0036] Figure 4 An example of a computer system or device according to various embodiments herein is shown.

[0037] Figure 5 An example of a video-based point cloud coding (V-PCC) architecture according to various embodiments herein is shown.

[0038] Figure 6 An example of an overview of possible receiver architectures according to various embodiments herein is shown.

[0039] Figure 7 and Figure 8 An exemplary process for streaming volumetric point cloud content based on the Session Description Protocol and the Real-Time Protocol according to various embodiments herein is shown.

[0040] Like reference symbols in the various drawings indicate like elements. DETAILED DESCRIPTION

[0041] This specification describes a Session Description Protocol (SDP)-based procedure and a Real-time Protocol (RTP)-based procedure to support streaming of volumetric video content. In general, volumetric content distribution can be configured to deliver a 6-degree-of-freedom (6DoF) immersive media experience. Real-time delivery protocols (such as RTP) and session management protocols (such as SDP) are updated to provide a method for sharing volumetric content in real time.

[0042] Volumetric video can be configured to deliver a real-time or near real-time 6DoF experience. Volumetric video includes spatial data and enables an observer to walk around and interact with people and objects. Volumetric video is more immersive than 360-degree based video footage because it captures motion in an environment in three dimensions (3D). For example, volumetric video can capture the motion of a person or object in 3D. An observer or user of volumetric video can observe these motions from any angle by using position tracking.

[0043] Point cloud is a volume representation for describing 3D objects or scenes. Point cloud can include a set of unordered data points in 3D space, each of which is specified by a corresponding spatial (x, y, z) position (or other coordinates). In some specific implementations, each point of point cloud can have associated attributes, such as red, green and blue (RGB) color data, surface normal data and reflectivity data. In other words, point cloud is roughly the 3D equivalent of the well-known pixel for representing two-dimensional (2D) video. The data points describe the 3D geometry and texture of the scene or object together. In some specific implementations, point cloud can be voxelized by quantizing the point positions on the integer grid within the bounding cube, which can achieve more efficient real-time processing. Generally speaking, if a voxel contains any point of the point cloud, then the voxel is occupied.

[0044] Such volumetric representations lend themselves to immersive forms of interaction and presentations that exploit 6DoF. Because point cloud representations require relatively large amounts of data compared to 3D video, it is desirable to develop efficient compression techniques that can reach consumers using typical broadband access systems. This specification describes efficient compression techniques that enable real-time streaming of volumetric data in typical broadband access systems.

[0045] Figure 1An exemplary wireless communication system 100 is shown. For convenience and not limitation, the exemplary system 100 is described in the context of LTE and 5G NR communication standards defined by the 3rd Generation Partnership Project (3GPP) technical specifications. More specifically, the wireless communication system 100 is described in the context of a non-standalone (NSA) network that combines both LTE and NR, such as an E-UTRA (Evolved Universal Terrestrial Radio Access)-NR dual connectivity (EN-DC) network and a NE-DC network. However, the wireless communication system 100 may also be an independent (SA) network that combines only NR. In addition, other types of communication standards are also possible, including future 3GPP systems (e.g., sixth generation (6G)) systems, IEEE 802.16 protocols (e.g., WMAN, WiMAX, etc.), and the like.

[0046] System 100 includes UE 101a and UE 101b (collectively referred to as "UE 101"). In this example, UE 101 is shown as a smartphone (eg, a handheld, touch-screen mobile computing device that can connect to one or more cellular networks). In other examples, any of the multiple UEs 101 may include other mobile computing devices or non-mobile computing devices, such as consumer electronic devices, cellular phones, smart phones, feature phones, tablet computers, wearable computer devices, personal digital assistants (PDAs), pagers, wireless handheld devices, desktop computers, laptop computers, in-vehicle infotainment (IVI), in-car entertainment (ICE) devices, instrument clusters (ICs), heads-up display (HUD) devices, on-board diagnostic (OBD) devices, on-board mobile equipment (DME), mobile data terminals (MDTs), electronic engine management systems (EEMS), electronic / engine control units (ECUs), electronic / engine control modules (ECMs), embedded systems, microcontrollers, control modules, engine management systems (EMS), networked or "smart" appliances, machine type communication (MTC) devices, machine-to-machine (M2M) devices, Internet of Things (IoT) devices, or combinations thereof, etc.

[0047] In some examples, any of the plurality of UEs 101 may be an IoT UE, which may include a network access layer designed for low-power IoT applications utilizing short-term UE connections. The IoT UE may utilize technologies such as M2M or MTC to exchange data with an MTC server or device utilizing, for example, a public land mobile network (PLMN), a short-range service (ProSe), a device-to-device (D2D) communication, a sensor network, an IoT network, or a combination thereof. The M2M or MTC data exchange may be a machine-initiated data exchange. The IoT network describes interconnected IoT UEs, which may include uniquely identifiable embedded computing devices (within the Internet infrastructure) with short-lived connections. The IoT UE may execute background applications (e.g., keep-alive messages or status updates) to facilitate connectivity to the IoT network.

[0048] UE 101 is configured to connect (e.g., be communicatively coupled) to an access network (AN) or radio access network (RAN) 110. In some examples, RAN 110 may be a next generation RAN (NG RAN), an evolved UMTS terrestrial radio access network (E-UTRAN), or a legacy RAN, such as a UMTS terrestrial radio access network (UTRAN) or a GSM EDGE radio access network (GERAN). As used herein, the term "NG RAN" may refer to a RAN 110 operating in a 5G NR system 100, while the term "E-UTRAN" may refer to a RAN 110 operating in an LTE or 4G system 100.

[0049] In order to connect to RAN 110, multiple UEs 101 utilize connections (or channels) 103 and 104, respectively, each of which may include a physical communication interface or layer, as described below. In this example, connections 103 and 104 are shown as air interfaces to achieve communication coupling, and may be consistent with cellular communication protocols, such as global system for mobile communications (GSM) protocols, code division multiple access (CDMA) network protocols, push-to-talk (PTT) protocols, cellular PTT (POC) protocols, universal mobile telecommunications system (UMTS) protocols, 3GPP LTE protocols, 5G NR protocols, or combinations thereof, as well as other communication protocols. In some examples, multiple UEs 101 may use interfaces 105 such as ProSe interfaces to directly exchange communication data. Interface 105 may alternatively be referred to as sidelink interface 105 and may include one or more logical channels, such as physical sidelink control channel (PSCCH), physical sidelink shared channel (PSSCH), physical sidelink downlink channel (PSDCH) or physical sidelink broadcast channel (PSBCH) or combinations thereof, etc.

[0050] UE 101b is shown configured to access access point (AP) 106 (also referred to as "WLAN node 106", "WLAN 106", "WLAN termination 106", "WT 106", etc.) using connection 107. Connection 107 may include a local wireless connection, such as a connection consistent with any IEEE 802.11 protocol, wherein AP 106 will include wireless fidelity Router. In this example, AP 106 is shown connected to the Internet without being connected to the core network of the wireless system, as described in further detail below. In various examples, UE 101b, RAN 110 and AP 106 may be configured to use LTE-WLAN aggregation (LWA) operation or LTW / WLAN radio level operation integrated with IPsec tunnel (LWIP). LWA operation may involve UE 101b in RRC_CONNECTED state being configured by RAN nodes 111a, 111b to utilize radio resources of LTE and WLAN. LWIP operation may involve UE 101b using IPsec protocol tunnel to use WLAN radio resources (e.g., connection 107) to authenticate and encrypt packets (e.g., IP packets) sent through connection 107. IPsec tunnel transmission may include encapsulating the entire original IP packet and adding a new packet header, thereby protecting the original header of the IP packet.

[0051] The RAN 110 includes one or more AN nodes or RAN nodes 111a and 111b (collectively referred to as "RAN nodes 111") that enable connections 103 and 104. As used herein, the terms "access node", "access point", etc. may describe equipment that provides radio baseband functions for data or voice connections or both between a network and one or more users. These access nodes may be referred to as base stations (BS), gNodeBs, gNBs, eNodeBs, eNBs, NodeBs, RAN nodes, road side units (RSUs), transmit receive points (TRxPs or TRPs), etc., and may include ground stations (e.g., terrestrial access points) or satellite stations that provide coverage within a geographic area (e.g., a cell), etc. As used herein, the terms "NG RAN nodes" and the like may refer to RAN nodes 111 (e.g., gNBs) operating in a 5G NR system 100, while the term "E-UTRAN node" may refer to RAN nodes 111 (e.g., eNBs) operating in an LTE or 4G system 100. In some examples, multiple RAN nodes 111 may be implemented as one or more of dedicated physical devices such as macrocell base stations or low power (LP) base stations for providing femtocells, picocells, or other similar cells with smaller coverage areas, smaller user capacity, or higher bandwidth than macrocells.

[0052] In some examples, some or all of the multiple RAN nodes 111 may be implemented as one or more software entities running on a server computer as part of a virtual network that may be referred to as a cloud RAN (CRAN) or a virtual baseband unit pool (vBBUP). The CRAN or vBBUP may implement RAN functional partitioning, such as a packet data convergence protocol (PDCP) partitioning, where the radio resource control (RRC) and PDCP layers are operated by the CRAN / vBBUP, and other layer 2 (e.g., data link layer) protocol entities are operated by individual RAN nodes 111; a medium access control (MAC) / physical layer (PHY) partitioning, where the RRC, PDCP, MAC, and radio link control (RLC) layers are operated by the CRAN / vBBUP, and the PHY layer is operated by individual RAN nodes 111; or a "lower PHY" partitioning, where the RRC, PDCP, RLC, and MAC layers and the upper portion of the PHY layer are operated by the CRAN / vBBUP, and the lower portion of the PHY layer is operated by individual RAN nodes 111. This virtualization framework allows idle processor cores of the RAN nodes 111 to execute, for example, other virtualized applications. In some examples, individual RAN nodes 111 may represent nodes using respective F1 interfaces ( Figure 1 In some examples, the gNB-DU may include one or more remote radio heads or RFEMs, and the gNB-CU may be operated by a server (not shown) located in the RAN 110 or by a server pool in a manner similar to the CRAN / vBBUP. In addition or alternatively, one or more of the RAN nodes 111 may be next-generation eNBs (ng-eNBs), including RAN nodes that provide E-UTRA user plane and control plane protocol terminals to the UE 101 and are connected to a 5G core network (e.g., the core network 120) using a next-generation interface.

[0053] In a vehicle-to-everything (V2X) scenario, one or more of the RAN nodes 111 may be or act as an RSU. The term "roadside unit" or "RSU" refers to any traffic infrastructure entity used for V2X communication. The RSU may be implemented in or by a suitable RAN node or a stationary (or relatively stationary) UE, wherein an RSU implemented in or by a UE may be referred to as a "UE-type RSU", an RSU implemented in or by an eNB may be referred to as an "eNB-type RSU", an RSU implemented in or by a gNB may be referred to as a "gNB-type RSU", and so on. In some examples, the RSU is a computing device coupled to a radio frequency circuit located on the road side that provides connectivity support to a passing vehicle UE 101 (vUE 101). The RSU may also include internal data storage circuitry for storing intersection map geometry, traffic statistics, media, and applications or other software for sensing and controlling ongoing vehicle and pedestrian traffic. The RSU may operate on the 5.9 GHz Direct Short Range Communications (DSRC) band to provide extremely low latency communications required for high-speed events, such as collision avoidance, traffic warnings, and the like. In addition or alternatively, the RSU may operate on the cellular V2X band to provide the aforementioned low latency communications as well as other cellular communications services. In addition or alternatively, the RSU may operate as a Wi-Fi hotspot (2.4 GHz band) or provide connectivity to one or more cellular networks to provide uplink and downlink communications, or both. Some or all of the computing device and the RSU's RF circuitry may be packaged in a weather-resistant enclosure suitable for outdoor installation, and may include a network interface controller to provide a wired connection (e.g., Ethernet) to a traffic signal controller or a backhaul network, or both.

[0054] Any of the RAN nodes 111 may serve as an endpoint for the air interface protocol and may be the first point of contact for the UE 101. In some examples, any of the multiple RAN nodes 111 may perform various logical functions of the RAN 110, including but not limited to functions of a radio network controller (RNC), such as radio bearer management, uplink and downlink dynamic radio resource management and data packet scheduling, and mobility management.

[0055] In some examples, multiple UEs 101 may be configured to communicate with each other or with any of multiple RAN nodes 111 over a multi-carrier communication channel using orthogonal frequency division multiplexing (OFDM) communication signals according to various communication techniques, such as, but not limited to, OFDMA communication techniques (e.g., for downlink communications) or SC-FDMA communication techniques (e.g., for uplink and ProSe or sidelink communications), although the scope of the techniques described herein is not limited in this respect. The OFDM signal may include multiple orthogonal subcarriers.

[0056] In some examples, a downlink resource grid may be used for downlink transmissions from any of a plurality of RAN nodes 111 to a plurality of UEs 101, while uplink transmissions may utilize similar techniques. The grid may be a time-frequency grid, referred to as a resource grid or a time-frequency resource grid, which is a physical resource in the downlink in each time slot. For OFDM systems, such a time-frequency plane representation is common practice, which makes wireless resource allocation intuitive. Each column and each row of the resource grid corresponds to an OFDM symbol and an OFDM subcarrier, respectively. The duration of the resource grid in the time domain corresponds to a time slot in a radio frame. The smallest time-frequency unit in the resource grid is represented as a resource element. Each resource grid includes a plurality of resource blocks, which describe the mapping of certain physical channels to resource elements. Each resource block includes a set of resource elements; in the frequency domain, this may represent the minimum amount of resources that can currently be allocated. Such resource blocks are used to transmit several different physical downlink channels.

[0057] In some examples, multiple UEs 101 and multiple RAN nodes 111 communicate (e.g., transmit and receive) data over a licensed medium (also referred to as a "licensed spectrum" or a "licensed band") and an unlicensed shared medium (also referred to as an "unlicensed spectrum" and / or an "unlicensed band"). The licensed spectrum may include channels operating in a frequency range of approximately 400 MHz to approximately 3.8 GHz, and the unlicensed spectrum may include a 5 GHz band. NR in an unlicensed spectrum may be referred to as NR-U, and LTE in an unlicensed spectrum may be referred to as LTE-U, License Assisted Access (LAA), or MulteFire.

[0058] To operate in an unlicensed spectrum, multiple UEs 101 and multiple RAN nodes 111 may operate using license assisted access (LAA), enhanced LAA (eLAA), or another enhanced LAA (feLAA) mechanism. In these specific implementations, multiple UEs 101 and multiple RAN nodes 111 may perform one or more known medium sensing operations or carrier sensing operations or both to determine whether one or more channels in the unlicensed spectrum are unavailable or otherwise occupied before transmitting in the unlicensed spectrum. The medium / carrier sensing operation may be performed according to a listen-before-talk (LBT) protocol. LBT is a mechanism for equipment (e.g., multiple UEs 101, multiple RAN nodes 111) to sense a medium (e.g., a channel or carrier frequency) and transmit when the medium is sensed to be idle (or when a specific channel in the medium is sensed to be unoccupied). The medium sensing operation may include a clear channel assessment (CCA), which uses energy detection to determine whether there are other signals on the channel in order to determine whether the channel is occupied or clear. The LBT mechanism allows cellular / LAA networks to coexist with existing systems in unlicensed spectrum and with other LAA networks. Energy detection may include sensing RF energy over an intended transmission band over a period of time and comparing the sensed RF energy to a predefined or configured threshold.

[0059] The existing system in the 5 GHz band may be a WLAN based on IEEE 802.11 technology. The WLAN adopts a contention-based channel access mechanism (e.g., CSMA with collision avoidance (CSMA / CA). In some examples, when a WLAN node (e.g., a mobile station (MS) such as UE 101, AP 106, etc.) intends to transmit, the WLAN node may first perform CCA before transmitting. In addition, in the case where more than one WLAN node senses the channel as idle and transmits at the same time, a backoff mechanism is used to avoid conflicts. The backoff mechanism can be a counter randomly drawn within the contention window size (CWS), which increases exponentially when a conflict occurs and is reset to a minimum value when the transmission is successful. In some examples, the LBT mechanism designed for LAA is similar to CSMA / CA for WLAN. In some examples, the LBT process for a DL or UL transmission burst (including PDSCH or PUSCH transmission) may have a LAA contention window of variable length between X and Y extended CAA (ECCA) time slots, where X and Y are the minimum and maximum values ​​of the CWS for LAA. In one example, the minimum CWS for LAA transmission may be 9 microseconds (μs); however, the size of the CWS and the maximum channel occupancy time (e.g., transmission burst) may be based on government regulatory requirements.

[0060] In some examples, the LAA mechanism is built on the carrier aggregation technology of the LTE-Advanced system. In CA, each aggregated carrier is called a component carrier. In some examples, the component carrier may have a bandwidth of 1.4, 3, 5, 10, 15, or 20 MHz, and up to five component carriers may be aggregated to provide a maximum aggregate bandwidth of 100 MHz. In a frequency division duplex (FDD) system, the number of aggregated carriers may be different for DL ​​and UL. For example, the number of UL component carriers may be equal to or less than the number of DL component carriers. In some cases, each component carrier may have a different bandwidth from other component carriers. In a time division duplex (TDD) system, the number of component carriers and the bandwidth of each component carrier are typically the same for DL ​​and UL.

[0061] Carrier aggregation may also include separate serving cells to provide separate component carriers. The coverage of the serving cells may be different, for example, because component carriers on different frequency bands may experience different path losses. The primary serving cell (PCell) may provide a primary component carrier for both UL and DL, and may handle RRC and non-access stratum (NAS) related activities. Other serving cells are referred to as secondary component carriers (SCells), and each SCell may provide a separate secondary component carrier for both UL and DL. Secondary component carriers may be added and removed as needed, and changing the primary component carrier may require UE 101 to undergo switching. In LAA, eLAA, and feLAA, some or all of the SCells may operate in an unlicensed spectrum (referred to as "LAA SCells"), and the LAA SCells are assisted by the PCells operating in a licensed spectrum. When a UE is configured with more than one LAA SCell, the UE may receive UL authorizations on the configured LAA SCells, indicating different PUSCH start positions within the same subframe.

[0062] The PDSCH carries user data and higher layer signaling to multiple UEs 101. The PDCCH carries, among other information, information about the transport format and resource allocation related to the PDSCH channel. It may also inform the UE 101 about the transport format, resource allocation, and hybrid automatic repeat request (HARQ) information related to the uplink shared channel. Downlink scheduling (e.g., allocating control and shared channel resource blocks to UE 101b within a cell) may be performed at any of the RAN nodes 111 based on channel quality information fed back from any of the UEs 101. Downlink resource allocation information may be sent on the PDCCH for (e.g., allocated to) each of the UEs 101.

[0063] PDCCH uses control channel elements (CCE) to transmit control information. Before being mapped to resource elements, the PDCCH complex-valued symbols can first be organized into quadruplets, which can then be arranged using a sub-block interleaver for rate matching. In some examples, one or more of these CCEs can be used to transmit each PDCCH, where each CCE can correspond to four sets of nine physical resource elements, collectively referred to as resource element groups (REGs). Four quadrature phase shift keying (QPSK) symbols can be mapped to each REG. Depending on the size of the downlink control information (DCI) and channel conditions, one or more CCEs can be used to transmit the PDCCH. In LTE, there may be four or more different PDCCH formats defined with different numbers of CCEs (e.g., aggregation levels, L=1, 2, 4, or 8).

[0064] Some implementations may use the concept of resource allocation for control channel information, which is an extension of the above concept. For example, some implementations may utilize an enhanced PDCCH (EPDCCH) that uses PDSCH resources for control information transmission. One or more enhanced CCEs (ECCEs) may be used to transmit EPDCCH. Similar to the above, each ECCE may correspond to nine sets of four physical resource elements, collectively referred to as enhanced REGs (EREGs). In some examples, ECCEs may have other numbers of EREGs.

[0065] The RAN nodes 111 are configured to communicate with each other using an interface 112. In an example, such as when the system 100 is an LTE system (e.g., when the core network 120 is an evolved packet core (EPC) network), the interface 112 may be an X2 interface 112. The X2 interface may be defined between two or more RAN nodes 111 (e.g., two or more eNBs, etc.) connected to the EPC 120, or between two eNBs connected to the EPC 120, or both. In some examples, the X2 interface may include an X2 user plane interface (X2-U) and an X2 control plane interface (X2-C). The X2-U may provide a flow control mechanism for user packets transmitted over the X2 interface, and may be used to transmit information about the delivery of user data between eNBs. For example, X2-U may provide specific sequence number information about user data transmitted from the primary eNB to the secondary eNB; information about successful in-sequence delivery of PDCP protocol data units (PDUs) from the secondary eNB to the UE 101 for user data; information about PDCP PDUs that were not delivered to the UE 101; information about the current minimum expected buffer size at the secondary eNB for transmitting user data to the UE; etc. X2-C may provide intra-LTE access mobility functions, including context transfer from a source eNB to a target eNB, or user plane transmission control; load management functions; inter-cell interference coordination functions; etc.

[0066] In some examples, such as when the system 100 is a 5G NR system (e.g., when the core network 120 is a 5G core network), the interface 112 may be an Xn interface 112. The Xn interface may be defined between two or more RAN nodes 111 (e.g., two or more gNBs, etc.) connected to the 5G core network 120, between a RAN node 111 (e.g., a gNB) and an eNB connected to the 5G core network 120, or between two eNBs connected to the 5G core network 120, or a combination of the above. In some examples, the Xn interface may include an Xn user plane (Xn-U) interface and an Xn control plane (Xn-C) interface. The Xn-U may provide non-guaranteed delivery of user plane PDUs and support / provide data forwarding and flow control functions. The Xn-C may provide management and error handling functions for managing functions of the Xn-C interface; mobility support for UE 101 in a connected mode (e.g., CM-CONNECTED), including functions for managing UE mobility in a connected mode between one or more RAN nodes 111; and the like. Mobility support may include context transfer from the old (source) serving RAN node 111 to the new (target) serving RAN node 111, and control of the user plane tunnel between the old (source) serving RAN node 111 and the new (target) serving RAN node 111. The protocol stack of Xn-U may include a transport network layer built on an Internet Protocol (IP) transport layer, and a GPRS Tunneling Protocol (GTP-U) layer for carrying user plane PDUs on top of a user datagram protocol (UDP) or IP layer or both. The Xn-C protocol stack may include an application layer signaling protocol (referred to as the Xn Application Protocol (Xn-AP)) and a transport network layer built on a stream control transmission protocol (SCTP). SCTP may be on top of the IP layer and may provide guaranteed delivery of application layer messages. In the transport IP layer, point-to-point transport is used to deliver signaling PDUs. In other specific implementations, the Xn-U protocol stack or the Xn-C protocol stack or both may be the same or similar to the user plane and / or control plane protocol stacks shown and described herein.

[0067] RAN 110 is shown as being communicatively coupled to a core network 120 (referred to as "CN 120"). CN 120 includes one or more network elements 122 configured to provide various data and telecommunication services to customers / subscribers (e.g., users of UE 101) connected to CN 120 using RAN 110. Components of CN 120 may be implemented in one physical node or separate physical nodes and may include components for reading and executing instructions from a machine-readable or computer-readable medium (e.g., a non-transitory machine-readable storage medium). In some examples, network function virtualization (NFV) may be used to virtualize some or all of the network node functions described herein using executable instructions stored in one or more computer-readable storage media, as will be described in further detail below. A logical instance of CN 120 may be referred to as a network slice, and a logical instance of a portion of CN 120 may be referred to as a network sub-slice. NFV architecture and infrastructure may be used to virtualize one or more network functions onto physical resources comprising a combination of industry-standard server hardware, storage hardware, or switches (alternatively performed by proprietary hardware). In other words, a NFV system may be used to perform virtual or reconfigurable implementations, or both, of one or more network components or functions.

[0068] In general, the application server 130 may be an element that provides applications that use IP bearer resources with the core network (e.g., UMTS packet service (PS) domain, LTE PS data service, etc.). The application server 130 may also be configured to utilize the CN 120 to support one or more communication services for the UE 101 (e.g., VoIP sessions, PTT sessions, group communication sessions, social network services, etc.).

[0069] In some examples, CN 120 may be a 5G core network (referred to as "5GC 120"), and RAN 110 may be connected to CN 120 using a next generation interface 113. In some examples, next generation interface 113 may be divided into two parts: a next generation user plane (NG-U) interface 114, which carries traffic data between multiple RAN nodes 111 and a user plane function (UPF); and an S1 control plane (NG-C) interface 115, which is a signaling interface between the RAN node 111 and an access and mobility management function (AMF).

[0070] In some examples, CN 120 may be an EPC (referred to as “EPC 120” or the like), and RAN 110 may be connected to CN 120 using an S1 interface 113. In some examples, S1 interface 113 may be divided into two parts: an S1 user plane (S1-U) interface 114, which carries traffic data between RAN node 111 and a serving gateway (S-GW); and an S1-MME interface 115, which is a signaling interface between RAN node 111 and a mobility management entity (MME).

[0071] Figure 2 An example of a platform 300 (or "device 300") is shown. In some examples, the computer platform 300 may be suitable for use as a UE 101, 201, 301, an application server 130, or any other component or device discussed herein. The platform 300 may include any combination of the components shown in the examples. The components (or portions thereof) of the platform 300 may be implemented as integrated circuits (ICs), discrete electronic devices, or other modules, logic, hardware, software, firmware, or combinations thereof adapted in the computer platform 300, or as components otherwise incorporated within a chassis of a larger system. Figure 2 The block diagram is intended to show a high-level view of the components of platform 300. However, in some examples, platform 300 may include fewer, additional, or alternative components, or include Figure 2 Different arrangements of components are shown.

[0072] The application circuit 305 includes circuits such as, but not limited to, one or more processors (or processor cores), cache memory, and one or more of an LDO, an interrupt controller, a serial interface (such as SPI), I2C or a general programmable serial interface module, an RTC, a timer (including an interval timer and a watchdog timer), a general I / O, a memory card controller (such as an SD MMC or similar controller), a USB interface, a MIPI interface, and a JTAG test access port. The processor (or core) of the application circuit 305 may be coupled to or may include a memory / storage element, and may be configured to execute instructions stored in the memory or storage device to enable various applications or operating systems to run on the system 300. In some examples, the memory or storage element may be an on-chip memory circuit, which may include any suitable volatile or non-volatile memory, such as DRAM, SRAM, EPROM, EEPROM, flash memory, solid-state memory, or a combination thereof, etc.

[0073] The processor of application circuit 305 may include, for example, one or more processor cores, one or more application processors, one or more GPUs, one or more RISC processors, one or more ARM processors, one or more CISC processors, one or more DSPs, one or more FPGAs, one or more PLDs, one or more ASICs, one or more microprocessors or controllers, multi-threaded processors, ultra-low voltage processors, embedded processors, some other known processing elements, or any suitable combination thereof. In some examples, application circuit 305 may include or may be a dedicated processor / controller for performing techniques described herein.

[0074] As an example, the processor of the application circuit 305 may include an Apple A series processor. The processor of the application circuit 305 may also be one or more of the following: based on Architecture Core TM Processors such as Quark TM 、Atom TM , i3, i5, i7 or MCU class processors, or available from Santa Clara, CA company( Another such processor is Intel Corporation, Santa Clara, CA); Advanced Micro Devices (AMD) Processor or Accelerated Processing Unit (APU); from Snapdragon by Technologies, Inc. TM Processor, Texas Instruments, Open Multimedia ApplicationsPlatform(OMAP) TM processors; MIPS-based designs from MIPS Technologies, Inc., such as the MIPS Warrior M-class, Warrior I-class, and Warrior P-class processors; ARM-based designs licensed from ARM Holdings, Ltd., such as the ARM Cortex-A, Cortex-R, and Cortex-M series processors; etc. In some implementations, the application circuit 305 can be part of a system on a chip (SoC), in which the application circuit 305 and other components are formed as a single integrated circuit.

[0075] Additionally or alternatively, the application circuit 305 may include circuits such as, but not limited to, one or more field programmable devices (FPDs) such as FPGAs; programmable logic devices (PLDs), such as complex PLDs (CPLDs), high capacity PLDs (HCPLDs); ASICs, such as structured ASICs; programmable SoCs (PSoCs), or combinations thereof, etc. In some examples, the application circuit 305 may include logic blocks or logic structures, and other interconnected resources that may be programmed to perform various functions such as the processes, methods, functions described herein. In some examples, the application circuit 305 may include memory cells (e.g., erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, static memory (e.g., static random access memory (SRAM) or antifuse)) for storing logic blocks, logic structures, data or other data in a lookup table (LUT), etc.

[0076] Baseband circuit 310 may be implemented, for example, as a solder-in substrate including one or more integrated circuits, a single packaged integrated circuit soldered to a main circuit board, or a multi-chip module containing two or more integrated circuits.

[0077] The RFEM 315 may include a millimeter wave (mmWave) RFEM and one or more sub-millimeter wave radio frequency integrated circuits (RFICs). In some examples, the one or more sub-millimeter wave RFICs may be physically separated from the mmWave RFEM. The RFIC may include connections to one or more antennas or antenna arrays, and the RFEM may be connected to multiple antennas. In some examples, both mmWave and sub-millimeter wave radio functions may be implemented in the same physical RFEM 315 incorporating both mmWave antennas and sub-millimeter waves.

[0078] The memory circuit 320 may include any number and type of memory devices for providing a fixed amount of system memory. As an example, the memory circuit 320 may include one or more of the following: volatile memory, such as random access memory (RAM), dynamic RAM (DRAM) or synchronous dynamic RAM (SDRAM); and non-volatile memory (NVM), such as high-speed electrically erasable memory (commonly referred to as flash memory), phase change random access memory (PRAM), or magnetoresistive random access memory (MRAM) or a combination thereof. The memory circuit 320 may be developed according to the Joint Electron Device Engineering Council (JEDEC) based low power double data rate (LPDDR) design such as LPDDR2, LPDDR3, LPDDR4, etc. The memory circuit 320 may be implemented as one or more of the following: a solder-in package integrated circuit, a single die package (SDP), a dual die package (DDP) or a quad die package (Q17P), a socketed memory module, a dual in-line memory module (DIMM) including a micro DIMM or a mini DIMM, or soldered to a motherboard using a ball grid array (BGA). In a low-power implementation, memory circuit 320 may be an on-chip memory or register associated with application circuit 305. To provide persistent storage of information such as data, applications, operating systems, etc., memory circuit 320 may include one or more mass storage devices, which may include, for example, a solid-state disk drive (SSDD), a hard disk drive (HDD), a micro HDD, a resistive change memory, a phase change memory, a holographic memory, or a chemical memory. In some examples, computer platform 300 may be combined with a memory device obtained from and Three-dimensional (3D) cross-point (XPOINT) memory.

[0079] The removable memory circuit 323 may include devices, circuits, housings, casings, ports or sockets, etc., for coupling portable data storage devices to the platform 300. These portable data storage devices may be used for mass storage and may include, for example, flash memory cards (e.g., secure digital (SD) cards, micro SD cards, xD picture cards), as well as USB flash drives, optical disks, or external HDDs, or combinations thereof, etc.

[0080] The platform 300 may further include an interface circuit (not shown) for connecting external devices to the platform 300. External devices connected to the platform 300 using the interface circuit include a sensor circuit 321 and an electromechanical component (EMC) 322, and a removable memory device coupled to a removable memory circuit 323.

[0081] Sensor circuit 321 includes a device, module, or subsystem whose purpose is to detect events or changes in its environment and send information about the detected events (e.g., sensor data) to one or more other devices, modules, or subsystems. Examples of such sensors include: an inertial measurement unit (IMU), such as an accelerometer, gyroscope, or magnetometer; a microelectromechanical system (MEMS) or nanoelectromechanical system (NEMS) including a three-axis accelerometer, a three-axis gyroscope, or a magnetometer; a liquid level sensor; a flow sensor; a temperature sensor (e.g., a thermistor); a pressure sensor; a barometric pressure sensor; a gravity meter; an altimeter; an image capture device (e.g., a camera or a lensless aperture); a light detection and ranging (LiDAR) sensor; a proximity sensor (e.g., an infrared radiation detector, etc.), a depth sensor, an ambient light sensor, an ultrasonic transceiver; a microphone or other audio capture device, or a combination thereof, etc.

[0082] The EMC 322 includes devices, modules, or subsystems whose purpose is to enable the platform 300 to change its state, position, or orientation or to move or control a mechanism, system, or subsystem. In addition, the EMC 322 can be configured to generate and send messages or signaling to other components of the platform 300 to indicate the current state of the EMC 322. Examples of EMC 322 include electromechanical components such as one or more power switches, relays (such as electromechanical relays (EMRs) and / or solid-state relays (SSRs)), actuators (e.g., valve actuators), audible sound generators, visual warning devices, motors (e.g., DC motors or stepper motors), wheels, thrusters, propellers, claws, clamps, hooks, or combinations thereof. In some examples, the platform 300 is configured to operate one or more EMCs 322 based on one or more capture events, instructions, or control signals received from a service provider or client, or both.

[0083] In some examples, the interface circuit may connect the platform 300 to the positioning circuit 345. The positioning circuit 345 includes a circuit for receiving and decoding signals transmitted or broadcast by a positioning network of a GNSS. Examples of GNSS may include the United States' GPS, Russia's GLONASS, the European Union's Galileo system, China's Beidou navigation satellite system, a regional navigation system or a GNSS augmentation system (e.g., NAVIC), Japan's QZSS, France's DORIS, etc. The positioning circuit 345 includes various hardware elements (e.g., including hardware devices such as switches, filters, amplifiers, antenna elements, etc. for facilitating OTA communications) to communicate with components of the positioning network such as navigation satellite constellation nodes. In some examples, the positioning circuit 345 may include a micro PNT IC that uses a master timing clock to perform position tracking or estimation without GNSS assistance. The positioning circuit 345 may also be part of or interact with the baseband circuit 410 or RFEM 315 or both to communicate with nodes and components of the positioning network. Positioning circuitry 345 may also provide data (eg, position data, time data) to application circuitry 305, which may use the data to synchronize operations with various infrastructures (eg, radio base stations) for use in turn-by-turn navigation applications, and the like.

[0084] In some examples, the interface circuit may connect the platform 300 with a near field communication (NFC) circuit 340. The NFC circuit 340 is configured to provide contactless short-range communication based on the radio frequency identification (RFID) standard, wherein a magnetic field is sensed to enable communication between the NFC circuit 340 and an NFC-enabled device (e.g., an "NFC touch point") external to the platform 300. The NFC circuit 340 includes an NFC controller coupled to an antenna element and a processor coupled to the NFC controller. The NFC controller may be a chip or IC that provides NFC functionality to the NFC circuit 340 by executing an NFC controller firmware and an NFC stack. The NFC stack may be executed by the processor to control the NFC controller, and the NFC controller firmware may be executed by the NFC controller to control the antenna element to transmit a short-range RF signal. The RF signal may power a passive NFC tag (e.g., a microchip embedded in a sticker or wristband) to transfer stored data to the NFC circuit 340, or initiate data transfer between the NFC circuit 340 and another active NFC device (e.g., a smart phone or an NFC-enabled POS terminal) close to the platform 300.

[0085] The driver circuit 346 may include software and hardware elements for controlling specific devices embedded in, attached to, or otherwise communicatively coupled to the platform 300. The driver circuit 346 may include various drivers to allow other components of the platform 300 to interact with or control various input / output (I / O) devices that may be present in or connected to the platform 300. For example, the driver circuit 346 may include: a display driver for controlling and allowing access to a display device, a touch screen driver for controlling and allowing access to a touch screen interface of the platform 300, a sensor driver for obtaining sensor readings of the sensor circuit 321 and controlling and allowing access to the sensor circuit 321, an EMC driver for obtaining an actuator position of the EMC 322 or controlling and allowing access to the EMC 322, a camera driver for controlling and allowing access to an embedded image capture device, and an audio driver for controlling and allowing access to one or more audio devices.

[0086] A power management integrated circuit (PMIC) 325 (also referred to as “power management circuit 325”) may manage power provided to various components of the platform 300. Specifically, the PMIC 325 may control power selection, voltage scaling, battery charging, or DC-DC conversion with respect to the baseband circuit 310. The PMIC 325 may be included when the platform 300 is capable of being powered by a battery 330, for example, when the device is included in a UE 101, 201, 301.

[0087] In some examples, the PMIC 325 may control or otherwise be part of various power saving mechanisms of the platform 300. For example, if the platform 300 is in the RRC_Connected state, in which the platform is still connected to the RAN node because it expects to receive traffic soon, then after a period of inactivity, the platform may enter a state known as discontinuous reception mode (DRX). During this state, the platform 300 may be powered off for short time intervals, thereby saving power. If there is no data traffic activity for a longer period of time, the platform 300 may transition to the RRC_Idle state, in which the device is disconnected from the network and no operations such as channel quality feedback or switching are performed. This may allow the platform 300 to enter a very low power state, in which the platform periodically wakes up to listen to the network and then powers off again. In some examples, the platform 300 may not receive data in the RRC_Idle state, but must transition back to the RRC_Connected state to receive data. Additional power saving modes may prevent the device from using the network for longer than the paging interval (ranging from a few seconds to a few hours). During this time, the device may not be able to connect to the network and may lose power completely. Any data sent during this time may be significantly delayed, and it is assumed that the delay is acceptable.

[0088] Battery 330 may power platform 300, but in some examples, platform 300 may be deployed in a fixed location and may have a power source coupled to a power grid. Battery 330 may be a lithium-ion battery, a metal-air battery such as a zinc-air battery, an aluminum-air battery, or a lithium-air battery, etc. In some examples, such as in V2X applications, battery 330 may be a typical lead-acid car battery.

[0089] In some examples, the battery 330 may be a "smart battery" that includes or is coupled to a battery management system (BMS) or a battery monitoring integrated circuit. The BMS may be included in the platform 300 to track the state of charge (SoCh) of the battery 330. The BMS may be used to monitor other parameters of the battery 330, such as the state of health (SoH) and state of function (SoF) of the battery 330 to provide fault prediction. The BMS may transmit information of the battery 330 to the application circuit 305 or other components of the platform 300. The BMS may also include an analog-to-digital (ADC) converter that allows the application circuit 305 to directly monitor the voltage of the battery 330 or the current from the battery 330. The battery parameters may be used to determine actions that the platform 300 may perform, such as transmission frequency, network operation, sensing frequency, etc.

[0090] A power block or other power source coupled to the grid can be coupled to the BMS to charge the battery 330. In some examples, the power block 330 can be replaced with a wireless power receiver to obtain power wirelessly, for example, through a loop antenna in the computer platform 300. In these examples, a wireless battery charging circuit can be included in the BMS. The specific charging circuit selected can depend on the size of the battery 330 and therefore the current required. Charging can be performed using the aviation fuel standard published by the Aviation Fuel Alliance, the Qi wireless charging standard published by the Wireless Power Alliance, or the Rezence charging standard published by the Wireless Power Alliance.

[0091] The user interface circuit 350 includes various input / output (I / O) devices present in or connected to the platform 300, and includes one or more user interfaces designed to implement user interaction with the platform 300 or peripheral component interfaces designed to implement interaction with peripheral components of the platform 300. The user interface circuit 350 includes input device circuits and output device circuits. The input device circuit includes any physical or virtual means for accepting input, including one or more physical or virtual buttons (e.g., a reset button), a physical keyboard, a keypad, a mouse, a touchpad, a touch screen, a microphone, a scanner, or a headset, or a combination thereof, etc. The output device circuit includes any physical or virtual means for displaying information or otherwise conveying information (such as sensor readings, actuator positions, or other information). The output device circuitry may include any number or combination of audio or visual displays, including one or more simple visual outputs or indicators (e.g., binary state indicators (e.g., light emitting diodes (LEDs)), multi-character visual outputs, or more complex outputs, such as a display device or touch screen (e.g., a liquid crystal display (LCD), an LED display, a quantum dot display, or a projector), where the output of characters, graphics, or multimedia objects is generated or produced by the operation of the platform 300. The output device circuitry may also include a speaker or other audio emitting device, or a printer. In some examples, the sensor circuitry 321 may be used as an input device circuit (e.g., an image capture device or a motion capture device) and one or more EMCs may be used as an output device circuit (e.g., an actuator for providing tactile feedback). In another example, an NFC circuit may be included to read an electronic tag or connect to another NFC-enabled device, the NFC circuitry including an NFC controller and a processing device coupled to an antenna element. The peripheral component interface may include, but is not limited to, a non-volatile memory port, a USB port, an audio jack, or a power interface.

[0092] Although not shown, the components of the platform 300 can communicate with each other using a suitable bus or interconnect (IX) technology, which can include any number of technologies, including ISA, EISA, PCI, PCIx, PCIe, a time-triggered protocol (TTP) system, a FlexRay system, or any number of other technologies. The bus or IX can be a proprietary bus or IX, for example, used in a SoC-based system. Other bus or IX systems can be included, such as an I2C interface, an SPI interface, a point-to-point interface, and a power bus, etc.

[0093] Figure 3 Various protocol functions that can be implemented in a wireless communication device are shown. Specifically, Figure 3 An arrangement 500 is included that shows the interconnection between various protocol layers / entities. Various protocol layers and entities operating in conjunction with the 5G NR system standard and the LTE system standard are provided. Figure 3 The following description, but Figure 3 Some or all aspects of the invention may also be applicable to other wireless communication network systems.

[0094] In addition to other higher layer functions not shown, the protocol layers of arrangement 500 may also include one or more of PHY 510, MAC 520, RLC 530, PDCP 540, SDAP 547, RRC 555, and NAS layer 557. These protocol layers may include one or more service access points (e.g., Figure 3 Items 559, 556, 550, 549, 545, 535, 525 and 515).

[0095] PHY 510 can send and receive physical layer signals 505, which can be received from or sent to one or more other communication devices. Physical layer signals 505 may include one or more physical channels, such as those discussed herein. PHY 510 may also perform link adaptation or adaptive modulation and coding (AMC), power control, cell search (e.g., for initial synchronization and switching purposes) and other measurements used by higher layers (e.g., RRC 555). PHY 510 may further perform error detection on transmission channels, forward error correction (FEC) encoding and decoding of transmission channels, modulation and demodulation of physical channels, interleaving, rate matching, mapping to physical channels, and MIMO antenna processing. In some examples, an instance of PHY 510 may use one or more PHY-SAP 515 to process requests from an instance of MAC 520 and provide instructions thereto. In some examples, requests and instructions transmitted using PHY-SAP 515 may include one or more transmission channels.

[0096] An instance of MAC 520 may process requests from and provide indications to an instance of RLC 530 using one or more MAC-SAPs 525. These requests and indications transmitted using MAC-SAPs 525 may include one or more logical channels. MAC 520 may perform mapping between logical channels and transport channels, multiplexing MAC service data units (SDUs) from one or more logical channels onto transport blocks (TBs) to be delivered to PHY 510 using transport channels, demultiplexing MAC SDUs from TBs delivered from PHY 510 using transport channels to one or more logical channels, multiplexing MAC SDUs onto TBs, scheduling information reporting, error correction via HARQ, and logical channel prioritization.

[0097] An instance of RLC 530 may process requests from an instance of PDCP 540 and provide indications thereto using one or more radio link control service access points (RLC-SAPs) 535. These requests and indications transmitted using RLC-SAPs 535 may include one or more RLC channels. RLC 530 may operate in a variety of operating modes, including: transparent mode (TM), unacknowledged mode (UM), and acknowledged mode (AM). RLC 530 may perform transmission of upper layer protocol data units (PDUs), error correction through automatic repeat request (ARQ) for AM data transmission, and concatenation, segmentation, and reassembly of RLC SDUs for UM and AM data transmission. RLC 530 may also perform resegmentation of RLC data PDUs for AM data transmission, reorder RLC data PDUs for UM and AM data transmission, detect duplicate data for UM and AM data transmission, discard RLC SDUs for UM and AM data transmission, detect protocol errors for AM data transmission, and perform RLC re-establishment.

[0098] An instance of PDCP 540 may process requests from an instance of RRC 555 or an instance of SDAP 547, or both, using one or more Packet Data Convergence Protocol Service Access Points (PDCP-SAP) 545 and provide indications thereto. These requests and indications transmitted using PDCP-SAP 545 may include one or more radio bearers. PDCP 540 may perform header compression and decompression of IP data, maintain PDCP sequence numbers (SNs), perform in-order delivery of higher layer PDUs when lower layers are reestablished, eliminate duplication of lower layer SDUs when lower layers are reestablished for radio bearers mapped on RLC AM, encrypt and decrypt control plane data, perform integrity protection and integrity verification on control plane data, control timer-based data discard, and perform security operations (e.g., encryption, decryption, integrity protection, or integrity verification).

[0099] An instance of SDAP 547 may process requests from one or more higher layer protocol entities using one or more SDAP-SAPs 549 and provide instructions thereto. These requests and instructions transmitted using SDAP-SAPs 549 may include one or more QoS flows. SDAP 547 may map QoS flows to data radio bearers (DRBs) and vice versa, and may also mark QoS flow identifiers (QFIs) in DL and UL packets. A single SDAP entity 547 may be configured for a separate PDU session. In the UL direction, NG-RAN 110 may control the mapping of QoS flows to DRBs in two different ways (reflective mapping or explicit mapping). For reflective mapping, SDAP 547 of UE 101 may monitor the QFI of the DL packets of each DRB, and may apply the same mapping to packets flowing in the UL direction. For DRBs, SDAP 547 of UE 101 may map UL packets belonging to a QoS flow corresponding to the QoS flow ID and PDU session observed in the DL packets of the DRB. To implement the reflective mapping, the NG-RAN 310 may mark the DL packets with a QoS flow ID over the Uu interface. The explicit mapping may involve the RRC 555 configuring the SDAP 547 with explicit mapping rules of QoS flows to DRBs, which may be stored and followed by the SDAP 547. In some examples, the SDAP 547 may be used only in NR implementations and may not be used in LTE implementations.

[0100] The RRC 555 may configure aspects of one or more protocol layers using one or more management service access points (M-SAPs), which may include one or more instances of PHY 510, MAC 520, RLC 530, PDCP 540, and SDAP 547. In some examples, an instance of the RRC 555 may process requests from one or more NAS entities 557 using one or more RRC-SAPs 556 and provide instructions thereto. The main services and functions of the RRC 555 may include broadcasting of system information (e.g., included in a master information block (MIB) or system information block (SIB) related to NAS), broadcasting of system information related to the access stratum (AS), paging, establishment, maintenance, and release of an RRC connection between the UE 101 and the RAN 110 (e.g., RRC connection paging, RRC connection establishment, RRC connection modification, and RRC connection release), establishment, configuration, maintenance, and release of point-to-point radio bearers, security functions including key management, inter-RAT mobility, and measurement configuration for UE measurement reporting. The MIB and SIB may include one or more information elements, each of which may include separate data fields or data structures.

[0101] NAS 557 may form the highest layer of the control plane between UE 101 and AMF 321. NAS 557 may support the mobility and session management procedures of UE 101 to establish and maintain an IP connection between UE 101 and P-GW in the LTE system.

[0102] In some examples, one or more protocol entities of arrangement 500 may be implemented in UE 101, RAN node 111, AMF 321 in NR implementation or MME 221 in LTE implementation, UPF 302 in NR implementation or S-GW 222 and P-GW 223 in LTE implementation, etc., for control plane or user plane communication protocol stacks between the aforementioned devices. In some examples, one or more protocol entities that may be implemented in one or more of UE 101, gNB 111, AMF 321, etc. may communicate with corresponding peer protocol entities that may be implemented in or on another device (using services of corresponding lower layer protocol entities to perform such communication). In some examples, the gNB-CU of gNB 111 may host the gNB's RRC 555, SDAP 547, and PDCP 540 that control operations of one or more gNB-DUs, and the gNB-DUs of gNB 111 may each host the RLC 530, MAC 520, and PHY 510 of gNB 111.

[0103] In some examples, the control plane protocol stack may include, in order from the highest layer to the lowest layer, NAS 557, RRC 555, PDCP 540, RLC 530, MAC 520, and PHY 510. In this example, upper layer 560 may be built on top of NAS 557, which includes IP layer 561, SCTP 562, and application layer signaling protocol (AP) 563.

[0104] In some examples such as NR implementations, AP 563 may be an NG application protocol layer (NGAP or NG-AP) 563 for an NG interface 113 defined between an NG-RAN node 111 and an AMF 321, or AP 563 may be an Xn application protocol layer (XnAP or Xn-AP) 563 for an Xn interface 112 defined between two or more RAN nodes 111.

[0105] The NG-AP 563 may support the functionality of the NG interface 113 and may include an elementary procedure (EP). The NG-AP EP may be an interaction unit between the NG-RAN node 111 and the AMF 321. The NG-AP 563 services may include two groups: UE-associated services (e.g., services related to the UE 101) and non-UE-associated services (e.g., services related to the entire NG interface instance between the NG-RAN node 111 and the AMF 321). These services may include functions such as, but not limited to: a paging function for sending a paging request to the NG-RAN node 111 involved in a specific paging area; a UE context management function for allowing the AMF 321 to establish, modify or release the UE context in the AMF 321 and the NG-RAN node 111; a mobility function for the UE 101 in ECM-CONNECTED mode, for intra-system HO to support mobility within the NG-RAN, and for inter-system HO to support mobility from / to the EPS system; a NAS signaling transport function for transporting or rerouting NAS messages between the UE 101 and the AMF 321; a NAS node selection function for determining the association between the AMF 321 and the UE 101; an NG interface management function for setting up the NG interface and monitoring errors over the NG interface; a warning message transport function for providing a means to transport a warning message using the NG interface or to cancel an ongoing warning message broadcast; a function for utilizing the CN 120 is a configuration transmission function for requesting and transmitting RAN configuration information (eg, SON information, or performance measurement (PM) data) between two RAN nodes 111; or a combination thereof, etc.

[0106] The XnAP 563 may support the functions of the Xn interface 112 and may include XnAP basic mobility procedures and XnAP global procedures. The XnAP basic mobility procedures may include procedures for handling UE mobility within the NG RAN 111 (or E-UTRAN 210), such as handover preparation and cancellation procedures, SN state transfer procedures, UE context retrieval and UE context release procedures, RAN paging procedures, or procedures related to dual connectivity, etc. The XnAP global procedures may include procedures not related to a specific UE 101, such as Xn interface setup and reset procedures, NG-RAN update procedures, or cell activation procedures, etc.

[0107] In an LTE specific implementation, the AP 563 may be an S1 application protocol layer (S1-AP) 563 for the S1 interface 113 defined between the E-UTRAN node 111 and the MME, or the AP 563 may be an X2 application protocol layer (X2AP or X2-AP) 563 for the X2 interface 112 defined between two or more E-UTRAN nodes 111.

[0108] The S1 application protocol layer (S1-AP) 563 may support the functionality of the S1 interface, and similar to the NG-AP discussed previously, the S1-AP may include an S1-AP EP. The S1-AP EP may be an interaction unit between the E-UTRAN node 111 and the MME 221 within the LTE CN 120. The S1-AP 563 services may include two groups: UE-associated services and non-UE-associated services. The functions performed by these services include, but are not limited to: E-UTRAN Radio Access Bearer (E-RAB) management, UE capability indication, mobility, NAS signaling transmission, RAN Information Management (RIM), and configuration transmission.

[0109] The X2AP 563 may support the functions of the X2 interface 112 and may include an X2AP basic mobility procedure and an X2AP global procedure. The X2AP basic mobility procedure may include a procedure for handling UE mobility within the E-UTRAN 120, such as a handover preparation and cancellation procedure, an SN state transfer procedure, a UE context retrieval and a UE context release procedure, a RAN paging procedure, or a procedure related to dual connectivity, etc. The X2AP global procedure may include a procedure not related to a specific UE 101, such as an X2 interface setup and reset procedure, a load indication procedure, an error indication procedure, or a cell activation procedure, etc.

[0110] The SCTP layer (alternatively referred to as the SCTP / IP layer) 562 may provide guaranteed delivery of application layer messages (e.g., NGAP or XnAP messages in NR implementations, or S1-AP or X2AP messages in LTE implementations). The SCTP 562 may ensure reliable delivery of signaling messages between the RAN node 111 and the AMF 321 / MME 221 based in part on the IP protocol supported by the IP 561. The Internet Protocol layer (IP) 561 may be used to perform packet addressing and routing functions. In some implementations, the IP layer 561 may deliver and transmit PDUs using point-to-point transport. In this regard, the RAN node 111 may include L2 and L1 layer communication links (e.g., wired or wireless) with the MME / AMF to exchange information.

[0111] In some examples, the user plane protocol stack may include, in order from the highest layer to the lowest layer, SDAP 547, PDCP 540, RLC 530, MAC 520, and PHY 510. The user plane protocol stack may be used for communication between UE 101, RAN node 111, and UPF 302 in a NR implementation, or communication between S-GW 222 and P-GW 223 in a LTE implementation. In this example, the upper layer 551 may be built on top of the SDAP 547 and may include a user datagram protocol (UDP) and IP security layer (UDP / IP) 552, a general packet radio service (GPRS) tunneling protocol layer (GTP-U) 553 for the user plane, and a user plane PDU layer (UP PDU) 563.

[0112] The transport network layer 554 (also referred to as the "transport layer") may be built on top of the IP transport, and the GTP-U 553 may be used on top of the UDP / IP layer 552 (including the UDP layer and the IP layer) to carry the user plane PDU (UP-PDU). The IP layer (also referred to as the "Internet layer") may be used to perform packet addressing and routing functions. The IP layer may assign IP addresses to user data packets, for example, in any of the IPv4, IPv6, or PPP formats.

[0113] GTP-U 553 may be used to carry user data within the GPRS core network and between the radio access network and the core network. For example, the transmitted user data may be packets in any of the IPv4, IPv6 or PPP formats. UDP / IP 552 may provide checksums for data integrity, port numbers for addressing different functions at the source and destination, and encryption and authentication of selected data flows. The RAN node 111 and the S-GW 222 may exchange user plane data using a protocol stack including an L1 layer (e.g., PHY 510), an L2 layer (e.g., MAC 520, RLC 530, PDCP 540 and / or SDAP 547), a UDP / IP layer 552, and a GTP-U 553 using an S1-U interface. The S-GW 222 and the P-GW 223 may exchange user plane data using a protocol stack including an L1 layer, an L2 layer, a UDP / IP layer 552, and a GTP-U 553 using an S5 / S8a interface. As previously discussed, the NAS protocol may support mobility of UE 101 and session management procedures to establish and maintain an IP connection between UE 101 and P-GW 223 .

[0114] In addition, despite Figure 3Not shown, but an application layer may exist above AP 563 and / or transport network layer 554. The application layer may be a layer where a user of UE 101, RAN node 111, or other network element interacts with a software application, for example, executed by application circuitry 205 or application circuitry 305, respectively. The application layer may also provide one or more interfaces for the software application to interact with a communication system (such as baseband circuitry 410) of UE 101 or RAN node 111. In some examples, the IP layer or the application layer, or both, may provide functionality the same or similar to layers 5 to 7 of the Open Systems Interconnection (OSI) model, or portions thereof (e.g., OSI layer 7—application layer, OSI layer 6—presentation layer, and OSI layer 5—session layer).

[0115] Figure 4 is a block diagram illustrating components for reading instructions from a machine-readable or computer-readable medium (e.g., a non-transitory machine-readable storage medium) and performing any one or more of the techniques described herein. Specifically, Figure 4 A schematic diagram of hardware resources 600 is shown, including one or more processors (or processor cores) 610, one or more memory / storage devices 620, and one or more communication resources 630, each of which may be communicatively coupled using a bus 640. For specific implementations in which node virtualization (e.g., NFV) is utilized, a hypervisor 602 may be executed to provide an execution environment for one or more network slices or sub-slices to utilize the hardware resources 600.

[0116] Processor 610 may include processor 612 and processor 614. Processor 610 may be, for example, a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a DSP such as a baseband processor, an ASIC, an FPGA, a radio frequency integrated circuit (RFIC), another processor (including those discussed herein), or any suitable combination thereof.

[0117] The memory / storage device 620 may include main memory, disk storage, or any suitable combination thereof. The memory / storage device 620 may include, but is not limited to, any type of volatile or non-volatile memory, such as dynamic random access memory (DRAM), static random access memory (SRAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or solid-state storage, or a combination thereof.

[0118] The communication resources 630 may include an interconnect or network interface component or other suitable device to communicate with one or more peripheral devices 604 or one or more databases 606 using the network 608. For example, the communication resources 630 may include a wired communication component (e.g., for coupling using USB), a cellular communication component, an NFC component, (or Low power consumption) components, components and other communication components.

[0119] The instructions 650 may include software, programs, applications, applet, applications, or other executable code for causing at least any one of the processors 610 to perform any one or more of the methods discussed herein. The instructions 650 may reside completely or partially in at least one of the processors 610 (e.g., in a cache memory of the processor), the memory / storage device 620, or any suitable combination thereof. In addition, any portion of the instructions 650 may be transmitted to the hardware resources 600 from any combination of the peripheral devices 604 or the database 606. Thus, the memory of the processor 610, the memory / storage device 620, the peripheral devices 604, and the database 606 are examples of computer-readable and machine-readable media.

[0120] Figure 5 An example of a video-based point cloud coding (V-PCC) architecture 700 is shown. The V-PCC architecture can be used with traditional video nodes such as H.264 / AVC and H.265 / HEVC. The architecture 700 includes a series of modules for processing a point cloud frame 702 as input and generating a compressed bitstream 750 representing volumetric video data as output. The operation of the V-PCC architecture 700 is subsequently described in detail. Typically, the 3D geometry and attribute data of the point cloud are converted into a set of 2D patches. The set of 2D patches is then packed into an image. The image is compressed, such as with any existing or future image or video codec. For example, the video codec can include MPEG-4 Advanced Video Coding (AVC), High Efficiency Video Coding (HEVC), AV1, etc.

[0121] The V-PCC architecture 700 is configured to receive an input point cloud frame 702 and generate patch data 708 through a patch data generation process described in further detail later. The patch data 708 is sent to a plurality of modules that process the patch data (and other data) to generate corresponding substreams, including a patch substream 722, an attribute substream 718, a geometry substream 742, and an occupancy substream 734. The substreams 722, 718, 742, and 734 are multiplexed by a multiplexer 724 to generate a compressed bitstream 750 that includes all data for streaming volumetric video. The generation of each of the substreams 722, 718, 742, and 734 can be referred to as a substream generation process. Typically, the modules of the architecture 700 for generating each of the substreams 722, 718, 742, and 734 act in parallel, such that the substreams are generated in a parallel process. However, in some cases, modules of different processes send and receive data to each other, and the processes are partially serial, as described subsequently.

[0122] The V-PCC architecture utilizes a patch-based approach to segment a point cloud into a set of clusters (or patches). The input cloud frame 702 data is used to generate 3D patches through a 3D patch generation module 704. Typically, patches are mapped to a predefined set of 2D planes by orthogonal projection without automatic blocking and with limited distortion. The V-PCC architecture is configured to generate a time-coherent, low-distortion single mapping. The mapping is used to assign each point of the 3D point cloud to a cell of a 2D grid. The mapping between the point cloud and the regular 2D grid is then obtained by packing the projected patches through a patch packing module 706 in a patch packing process. The patch packing module 706 generates patch data 708, which is sent to each substream generation process for generating substreams 722, 718, 742, and 734.

[0123] The patch packing process is constrained to ensure that there is no overlap between patches. In addition, the bounding box of any patch represented in T×T blocks (where T is the packing block size) should not overlap with any T×T block belonging to a previously encoded patch. Such constraints make it possible to determine for each T×T block of the packing grid which patch it belongs to by analyzing the 2D bounding boxes of all patches.

[0124] The T×T patches are then processed in parallel to generate point cloud geometry and attributes. For each cell of the T×T patch, the corresponding pixel in the occupancy map is used to determine whether the cell is full or empty. If the cell is full, a 3D point is generated following two different procedures depending on the type of patch.

[0125] V-PCC supports the concept of regular patches, which use the patch projection process described previously. For regular patches, the 3D point Cartesian coordinates are calculated by combining the depth information stored in the geometry image with the 2D position of the cell, the 3D offset of the patch, and the 2D projection plane. The attribute value associated with the reconstructed point is obtained by sampling the 2D attribute frame at the same grid position.

[0126] The patch data 708 is used by four processes to generate a compressed bitstream 750. The first process described is the patch substream process. The patch data 708 generated to reconstruct a 3D point cloud from 2D geometry, attribute, and occupancy video is compressed. The patch data 708 is encoded in a V-PCC patch sequence substream, such as by a patch sequence compression module 720. The V-PCC architecture 700 introduces a codec that is specifically optimized to handle a patch substream 722, which occupies a relatively small amount of the total bitstream (e.g., less than 5%). The patch substream 722 is sent to a multiplexer 724 to be combined with the attribute substream 718, the geometry substream 742, and the occupancy substream 734.

[0127] The second process described is an occupancy map generation process. Patch data 708 is sent to an occupancy map image generation module 728. Module 728 generates an occupancy map 730 that indicates which voxels are occupied. Occupancy map 730 is processed by a video compression module 732 to generate an occupancy substream 734. Substream 734 includes a data stream indicating occupancy information of a compressed bitstream 750. During a reconstruction process (described later), video compression module 732 generates a reconstructed occupancy map 733 that is used for image filling in the attribute subprocess and the geometry subprocess.

[0128] The third process is a geometry sub-process in which geometry data (e.g., of a patch) of a frame is determined and a geometry sub-stream 742 is generated. The patch data 708 is sent to a geometry image generation module 726 to generate geometry data 736. The geometry data 736 is compressed by a video compression module 740 (which may be a dedicated codec for geometry data) after image padding 738, which may be used during the reconstruction process with data from the reconstructed occupancy map 733. The video compression module 740 generates a geometry sub-stream 742 associated with a corresponding sequence of patches of the patch sub-stream 722 and the occupancy map 730 data of the occupancy sub-stream 734.

[0129] The fourth process is an attribute process in which attribute values ​​are determined for attributes associated with the patch sequence and the geometry and occupancy data. The attribute image generation module 710 receives the patch data 708 and receives smoothed geometry data 748 from the smoothing module 746 during reconstruction. The smoothing module 746 in turn receives the reconstructed geometry image 744, the reconstructed occupancy map 733, and the patch data 708 to generate the smoothed geometry data 748.

[0130] The attribute image generation module 710 sends the attribute data to the image filling module 712, which may receive the occupancy map 730 during the reconstruction process. The padded attribute data 714 is sent to the video compression module 716, which may be similar or identical to the video compression modules 732 and / or 740 (e.g., using a shared codec), or may be completely different from the modules 732 and 740. The compression module 716 generates an attribute substream 718 for the attribute values ​​including the associated patch sequence data of the patch substream 722, the geometry data of the geometry substream 742, and the occupancy map data 730 of the occupancy substream 734.

[0131] In general, additional information is used to synchronize and link the video and patch substreams, such as synchronization data, and is also signaled in the bitstream 750. The V-PCC bitstream 750 is then formed by concatenating the various encoded information (e.g., occupancy map substream 734, geometry substream 742, attribute substream 718, and patch sequence substream 722) into a single stream 750. This is done by encapsulating these substreams into V-PCC data units, each of which consists of a header and a payload.

[0132] The V-PCC unit header describes the V-PCC unit type. Generally speaking, five different unit types are supported. The sequence parameter set (SPS) unit type describes the entire V-PCC bitstream and its subcomponents. The remaining unit types include occupancy video, geometry video, attribute video, and patch sequence data units, which encapsulate occupancy map, geometry, attribute, and patch sequence substreams respectively.

[0133] The V-PCC decoding process is divided into two stages, including a bitstream decoding process and a reconstruction process. The bitstream decoding process takes the V-PCC compressed bitstream 750 as input and outputs decoded occupancy, geometry, and attribute 2D video frames, as well as patch information associated with each frame. The reconstruction process uses the patch information to convert the 2D video frames into a set of reconstructed 3D point cloud frames. The reconstruction process requires resampling the occupancy, geometry, and attribute video sequence at the nominal 2D resolution specified in the SPS. The resampled video is then used for a 3D reconstruction process, which includes geometry and attribute reconstruction and geometry and attribute smoothing.

[0134] For V-PCC-based bitstreams, the RTP-based streaming framework includes the following features. The framework includes a dedicated codec bitstream (e.g., HEVC bitstream). The dedicated bitstreams each carry information encoded in one of the substreams 722, 718, 742, or 734, including occupancy map data 730, geometry data 736, and attribute data 718. The dedicated RTP stream includes each codec / HEVC bitstream so that there is a dedicated RTP stream for occupancy map, geometry, and attribute information. The dedicated RTP stream uses a specific RTP payload format for the codec (e.g., an RTP payload format for HEVC). As previously described, dedicated RTP streams 722, 718, 742, and 734 are multiplexed into a single RTP stream 750 of the V-PCC video. The RTP header extension for the multiplexed RTP stream can carry sequence parameter set (SPS) unit type information. In some implementations, the SPS data is part of one of the codec bitstreams or a separate bitstream, such as an SEI message for an HEVC bitstream. In another embodiment, each of the dedicated RTP streams is negotiated and transmitted separately as a different media type. For example, a first media type is used for geometry data, a second media type is used for attribute data, a third media type is used for occupancy map data, and so on. In addition, another RTP stream may carry a compressed patch sequence substream 722 and may also be multiplexed into a single RTP stream for V-PCC video. Alternatively, the information may be carried as part of an RTP header extension (as part of the multiplexed RTP stream). In one embodiment, the HEVC SEI message for each dedicated HEVC bitstream may carry additional metadata about each encoded information, including occupancy map data 730, geometry data 736, and attribute data 714.

[0135] Figure 6 8 shows an example of an overview of a receiver architecture 800. The receiver architecture 800 is configured to receive a V-PCC bit stream (e.g., Figure 5 The spherical video is reconstructed from the received RTP stream obtained after the RTP stream of the bitstream 750 is received. The architecture 800 represents a set of logical receiver functions, including a real-time protocol (RTP) receiver 804, a HEVC decoder 808, a point cloud reconstruction module 814, and a viewport rendering module 818.

[0136] Based on one or more received RTP media streams, a device (e.g., Figures 1 to 4The UE 101 or other network device described in the embodiment of the present invention is configured to receive and parse, decrypt the primary RTP stream 806 (at the receiver 804) and / or feed the primary RTP stream to the HEVC decoder 808. The HEVC decoder 808 obtains a decoder output signal (referred to as texture data 810), as well as decoder metadata 812. In this context, the texture data 810 may include occupancy maps, geometry data, and attribute data, depending on the HEVC bitstream. The decoder metadata 812 includes SEI messages, such as information carried in V-PCC specific SEI messages, for use in the rendering stage. Specifically, the point cloud reconstruction function 814 can use the decoder metadata 812 and the decoded texture information about the occupancy map, geometry, and attributes to generate a point cloud (or a portion thereof) based on the decoded output signal (e.g., texture data 810). Viewport data is then generated from the point cloud signal 816 (or a portion thereof) and rendered (e.g., by a rendering module 818) by considering the viewport position information from the sensor, display characteristics, and possible other metadata (such as initial viewport information). These sensors may include any positioning sensors or other sensors, such as gyroscopes, magnetic trackers, and the like.

[0137] Viewport-dependent processing can be supported for point-to-point session and multi-party conference scenarios, and can be achieved by sending RTCP feedback or RTP header extension messages with the desired viewport information from the receiver, and then encoding and sending the corresponding viewport via the IP Multimedia Core Network Subsystem (IMS) (MTSI) sender or by a media gateway (e.g., MRF) through the multimedia telephony service. It is expected to deliver a resolution higher than the desired viewport for the viewport-independent method. The transmitted RTP stream from the sender can also include actual viewport or coverage information, such as in the RTP header extension message, because the point cloud video generated, encoded and streamed by the sender can cover a larger area than the desired viewport.

[0138] For the negotiation of the RTP stream for carrying the V-PCC bitstream, the procedures specified in IETF RFC 7798, RTP Payload Format for HEVC, which was published in March 2016 and is incorporated herein by reference in its entirety, may be reused. Additional SDP attributes may also be defined instead as follows. Based on the above, an SDP framework for immersive video exchange needs to be developed to negotiate codec support, SEI messages for decoder rendering metadata, and RTP / RTCP signaling required for viewport dependency handling. These capabilities can be negotiated separately, but in order to simplify SDP exchange and avoid fragmentation of these capabilities, it would be more preferable to specify one or more MTSI client profiles and develop an SDP framework based on these profiles.

[0139] Figures 5 and 6The encoding and decoding architectures 700, 800 may be composed of the previously described Figures 1 to 4 For example, the encoding and decoding architectures 700 and 800 may be implemented on a UE, an access node (AN), a networking device, a next generation node (gNB), etc.

[0140] Figure 7 and Figure 8 Exemplary processes 900 and 1000 for streaming volumetric point cloud content based on session description protocol and real-time protocol according to various embodiments herein are shown. In some embodiments, Figures 1 to 6 The electronic device, network, system, chip or component of some other figures or a portion or specific implementation thereof may be configured to perform one or more processes, techniques or methods or a portion thereof as described herein. Figure 7 Such a process 900 is depicted in FIG. Process 900 may be performed by a network element, a UE, or a base station. For a specific implementation, the process is performed by a network element (e.g., a gNB, an access node (AN), etc.).

[0141] Figure 7 The process 900 shown in includes generating (902) a plurality of codec bitstreams, each codec bitstream configured to carry encoded information including one of occupancy map data, geometry data, or attribute data for volumetric video. The process includes generating (904) a dedicated RTP stream for each of the plurality of codec bitstreams, the dedicated RTP stream including an RTP payload format based on the corresponding codec bitstream of the RTP stream. The process includes multiplexing (906) each dedicated RTP stream into a resulting RTP stream of the volumetric video.

[0142] In some implementations, process 900 includes performing video-based point cloud coding (V-PCC) using the resulting RTP stream. In some embodiments, each codec bitstream includes a high efficiency video coding (HEVC) bitstream. In some embodiments, process 900 includes generating an additional dedicated RTP stream including a compressed patch sequence substream. Process 900 includes multiplexing the additional RTP stream into the resulting RTP stream for the V-PCC video.

[0143] In some embodiments, process 900 includes generating an RTP header extension of the resulting RTP stream, the RTP header extension being configured to include a compressed patch sequence substream. In some embodiments, process 900 includes generating one or more Session Description Protocol (SDP) attributes to indicate the ability to carry point cloud video as part of the resulting RTP stream. In some specific implementations, the one or more SDP attributes of the resulting RTP stream indicate at least one of the following: the ability to carry encoded point cloud information about occupancy map data, geometry data, attribute data, and patch sequence data in a dedicated RTP stream; the ability to carry a dedicated RTP stream about point cloud information such as occupancy map data, geometry data, attribute data, and patch sequence data in a single multiplexed RTP stream; the ability to carry sequence parameter set (SPS) unit type information in an RTP header extension; the ability to carry point cloud metadata information as part of the RTP payload format; and the ability to support codec capabilities to meet minimum quality requirements for point cloud video.

[0144] In some embodiments, the RTP payload format includes at least one Supplementary Information Enhancement (SEI) message, wherein the data carried in the point cloud video specific SEI message is used at the receiver for the rendering stage. In some specific implementations, the one or more SDP attributes indicate at least one of the following: the ability to signal the desired viewport using an RTCP feedback message; and the ability to signal the actual transmitted viewport using an RTP header extension message. In some embodiments, each dedicated RTP stream includes a different media type.

[0145] Figure 8 A process 1000 for receiving an RTP bitstream and generating a volumetric video from the RTP bitstream is shown. The process 1000 includes demultiplexing (1002) an RTP stream containing a V-PCC bitstream. The process (1004) includes obtaining a decoder output signal and decoder metadata. The process 1000 includes generating (1006) a point cloud signal based on the decoded output signal and the decoder metadata. The process 1000 includes generating (1008) a viewpoint signal from the point cloud signal.

[0146] The techniques described herein may be performed by an apparatus implemented or employed in one or more types of network components, user equipment, or both. In some implementations, one or more non-transitory computer-readable media include instructions for causing an electronic device to perform one or more of the techniques described herein when the instructions are executed by one or more processors of an electronic device. An apparatus may include one or more processors and one or more computer-readable media, the computer-readable media including instructions that, when executed by the one or more processors, cause the one or more processors to perform one or more of the techniques described herein.

[0147] In different specific implementations, the methods described herein can be implemented in software, hardware, or a combination thereof. In addition, the order of the blocks of the method can be changed, and various elements can be added, reordered, combined, omitted, modified, etc. Various modifications and changes can be made, which will be apparent to those skilled in the art who benefit from this disclosure. The various specific implementations described herein are intended to be illustrative and not restrictive. Many variations, modifications, additions, and improvements are possible. Therefore, multiple examples can be provided for the components described herein as a single example. The boundaries between various components, operations, and data repositories are arbitrary to a certain extent, and specific operations are shown in the context of a specific exemplary configuration. Other allocations of functions are contemplated, and they may fall within the scope of the appended claims. Finally, the structure and function presented as discrete components in the exemplary configuration may be implemented as a combined structure or component.

[0148] The methods described herein may be implemented in a circuit, such as one or more of an integrated circuit, a logic circuit, a processor (shared, dedicated, or group) and / or a memory (shared, dedicated, or group), an ASIC, a field programmable device (FPD) (e.g., a field programmable gate array (FPGA), a programmable logic device (PLD), a complex PLD (CPLD), a high capacity PLD (HCPLD), a structured ASIC, or a programmable SoC), a digital signal processor (DSP), or some combination thereof. Examples of processors may include Apple A series processors, Architecture Core TM Processors, such as Intel processors, ARM processors, AMD processors, and Qualcomm processors. Other types of processors are possible. In some specific implementations, the circuit can execute one or more software or firmware programs to provide at least some of the functions. The term "circuit" can also refer to a combination of one or more hardware elements and a program code for executing the functions of the program code (or a combination of circuits used in an electrical or electronic system). In these embodiments, the combination of hardware elements and program codes can be referred to as a specific type of circuit. The circuit can also include a radio circuit, such as a transmitter, a receiver, or a transceiver.

[0149] As described above, some aspects of the subject matter of this specification include the collection and use of data from various sources to improve the services that mobile devices can provide to users. This disclosure contemplates that, in some cases, the collected data may identify a specific location or address based on device usage. Such personal information data may include location-based data, addresses, subscriber account identifiers, or other identifying information.

[0150] The disclosure also contemplates that entities responsible for the collection, analysis, disclosure, transmission, storage or other purposes of such personal information data will comply with established privacy policies and / or privacy practices. Specifically, such entities should implement and adhere to the use of privacy policies and practices that are recognized as meeting or exceeding the industry or government requirements for maintaining the privacy and security of personal information data. For example, personal information from users should be collected for the legal and reasonable purposes of the entity, and not shared or sold outside these legal purposes. In addition, such collection should only be carried out after the user's informed consent. In addition, such entities should take any necessary steps to safeguard and protect access to such personal information data, and ensure that others who can access personal information data comply with their privacy policies and procedures. In addition, such entities can subject themselves to third-party assessments to prove that they comply with widely accepted privacy policies and practices.

[0151] In the case of an advertisement delivery service, the present disclosure also contemplates implementations in which users selectively block the use or access of personal information data. That is, the present disclosure contemplates providing hardware elements and / or software elements to prevent or block access to such personal information data. For example, in the case of an advertisement delivery service, the technology of the present invention may be configured to allow a user to choose to "opt in" or "opt out" of participating in the collection of personal information data during registration for the service.

[0152] Thus, while the present disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, the present disclosure also contemplates that various embodiments may also be implemented without access to such personal information data. That is, various embodiments of the present technology will not fail to function properly due to the lack of all or a portion of such personal information data. For example, content may be selected and delivered to a user by inferring preferences based on non-personal information data or an absolute minimum amount of personal information, such as content requested by a device associated with the user, other non-personal information available to a content delivery service, or publicly available information.

[0153] As used herein, the term "user equipment" or "UE" refers to a device that has radio communication capabilities and can describe a remote user of network resources in a communication network. In addition, the terms "user equipment" or "UE" may be considered synonymous and may be referred to as a client, mobile phone, mobile device, mobile terminal, user terminal, mobile unit, mobile station, mobile user, subscriber, user, remote station, access agent, user agent, receiver, radio equipment, reconfigurable radio equipment, reconfigurable mobile device, etc. In addition, the term "user equipment" or "UE" may include any type of wireless / wired device or any computing device that includes a wireless communication interface.

[0154] As used herein, the term "network element" refers to physical or virtualized equipment and / or infrastructure for providing wired or wireless communication network services. The term "network element" may be considered synonymous with and / or referred to as a networked computer, networking hardware, network equipment, network node, router, switch, hub, bridge, radio network controller, RAN equipment, RAN node, gateway, server, virtualized VNF, NFVI, etc.

[0155] As used herein, the term "computer system" refers to any type of interconnected electronic devices, computer devices, or components thereof. Additionally, the terms "computer system" and / or "system" may refer to various components of a computer that are communicatively coupled to one another. Furthermore, the terms "computer system" and / or "system" may refer to multiple computer devices and / or multiple computing systems that are communicatively coupled to one another and configured to share computing and / or networking resources.

[0156] As used herein, the terms "appliance", "computer appliance", etc. refer to a computer device or computer system with program code (e.g., software or firmware) that is specifically designed to provide specific computing resources. A "virtual appliance" is a virtual machine image to be implemented by a device equipped with a hypervisor that virtualizes or emulates a computer appliance or is otherwise dedicated to providing specific computing resources.

[0157] As used herein, the term "resource" refers to a physical or virtual device, a physical or virtual component within a computing environment, and / or a physical or virtual component within a specific device, such as a computer device, a mechanical device, a memory space, a processor / CPU time and / or a processor / CPU usage rate, a processor and accelerator load, a hardware time or usage rate, a power supply, an input / output operation, a port or a network socket, a channel / link allocation, throughput, memory usage rate, storage, a network, a database and an application, a unit of work, etc. "Hardware resources" may refer to computing, storage and / or network resources provided by physical hardware elements. "Virtualized resources" may refer to computing, storage and / or network resources provided by a virtualized infrastructure to an application, a device, a system, etc. The term "network resources" or "communication resources" may refer to resources that a computer device / system can access via a communication network. The term "system resources" may refer to any kind of shared entity that provides a service, and may include computing resources and / or network resources. System resources may be considered as a set of coherent functions, network data objects or services that can be accessed through a server, wherein such system resources reside on a single host or multiple hosts and can be clearly identified.

[0158] As used herein, the term "channel" refers to any tangible or intangible transmission medium for transmitting data or data streams. The term "channel" may be synonymous and / or equivalent to "communication channel", "data communication channel", "transmission channel", "data transmission channel", "access channel", "data access channel", "link", "data link", "carrier", "radio frequency carrier" and / or any other similar terms representing a path or medium through which data is transmitted. In addition, the term "link" as used herein refers to a connection between two devices for transmitting and receiving information over a RAT.

[0159] As used herein, the terms "instantiate," "instantiate," and the like refer to the creation of an instance. "Instance" also refers to a specific occurrence of an object, which may occur, for example, during the execution of program code.

[0160] The terms "coupled," "communicatively coupled," and their derivatives are used herein. The term "coupled" may mean that two or more elements are in direct physical or electrical contact with each other, may mean that two or more elements are in indirect contact with each other but still cooperate or interact with each other, and / or may mean that one or more other elements are coupled or connected between the elements said to be coupled to each other. The term "directly coupled" may mean that two or more elements are in direct contact with each other. The term "communicatively coupled" may mean that two or more elements may be in contact with each other by means of communication, including through a wire or other interconnect connection, through a wireless communication channel or link, etc.

[0161] It is understood that the use of personally identifiable information should be subject to privacy policies and practices that are generally recognized to meet or exceed industry or government requirements for maintaining user privacy. Specifically, personally identifiable information data should be managed and processed to minimize the risk of unintentional or unauthorized access or use, and the nature of the authorized use should be clearly stated to users.

[0162] The term "information element" refers to a structural element that contains one or more fields. The term "field" refers to the individual contents of an information element, or a data element that contains the contents.

[0163] The term "SMTC" refers to the SSB-based measurement timing configuration configured by SSB-MeasurementTimingConfiguration.

[0164] The term "SSB" refers to SS / PBCH block.

[0165] The term "primary cell" refers to an MCG cell operating on a primary frequency, where the UE either performs an initial connection establishment procedure or initiates a connection re-establishment procedure.

[0166] The term "primary SCG cell" refers to an SCG cell in which a UE performs random access when reconfiguration is performed using a synchronization procedure for DC operation.

[0167] The term "secondary cell" refers to a cell that provides additional radio resources on top of a special cell for a UE configured with CA.

[0168] The term "secondary cell group" refers to a subset of serving cells including a PSCell for a UE configured with DC and zero or more secondary cells.

[0169] The term "serving cell" refers to a primary cell for a UE in RRC_CONNECTED without CA / DC configured, where there is only one serving cell including the primary cell.

[0170] The term "serving cell" refers to a cell group including a special cell for a UE configured with CA / DC and in RRC_CONNECTED and all secondary cells.

[0171] The term "special cell" refers to the PCell of the MCG or the PSCell of the SCG for DC operation; otherwise, the term "special cell" refers to the PCell.

[0172] A plurality of specific implementations have been described. However, it should be understood that various modifications may be made. Elements in one or more specific implementations may be combined, deleted, modified or supplemented to form other specific implementations. As another example, the logic flow shown in the accompanying drawings does not require the specific order or sequential order shown to achieve the desired result. In addition, other steps may be provided or steps may be eliminated from the described process, and other components may be added to the described system or removed from the described system. Therefore, other specific implementations are within the scope of the following claims.

[0173] Example

[0174] Embodiment 1 may include an RTP streaming framework having one or more of the following. The RTP streaming framework may include a dedicated codec bitstream (e.g., an HEVC bitstream) that carries each of the encoded information, such as occupancy maps, geometry, and attributes. The RTP streaming framework may include a dedicated RTP stream containing each codec / HEVC bitstream, such that there will be a dedicated RTP stream for occupancy maps, geometry, and attribute information.

[0175] Embodiment 2 may include an RTP streaming framework as described in Embodiment 1 or some other embodiments herein, wherein each dedicated RTP stream uses a specific RTP payload format for the corresponding codec (e.g., an RTP payload format for HEVC). The dedicated RTP stream is multiplexed into a single RTP stream of the V-PCC video. The RTP header extension for the multiplexed RTP stream is configured to carry sequence parameter set (SPS) unit type information. The SPS may otherwise be part of a codec bitstream in a codec bitstream, such as an SEI message for an HEVC bitstream.

[0176] Embodiment 3 may include an RTP streaming framework as described in Embodiment 2 or another embodiment herein, wherein each dedicated RTP stream in the dedicated RTP stream is separately negotiated and transmitted as a different media type, such as a media type for geometry, a media type for attributes, a media type for occupancy maps, and the like.

[0177] Embodiment 4 may include an RTP streaming framework as described in Embodiment 3 or another embodiment herein, wherein another RTP stream may be configured to carry a compressed patch sequence substream and also multiplexed into a single RTP stream for V-PCC video.

[0178] Embodiment 5 may include an RTP streaming framework as described in Embodiment 4 or another embodiment herein, wherein a compressed patch sequence substream may be carried as part of an RTP header extension as part of a multiplexed RTP stream.

[0179] Embodiment 6 may include an RTP streaming framework as described in Embodiment 5 or another embodiment herein, wherein the HEVC SEI message for each of the dedicated HEVC bitstreams carries additional metadata about each encoded information, such as occupancy maps, geometry, and attributes.

[0180] Embodiment 7 may include one or more SDP attributes as described in Embodiment 1 to indicate the ability to carry point cloud video as part of an RTP stream, including (but not limited to) one or more of the following: the ability to carry encoded point cloud information about occupancy maps, geometry, attributes, and patch sequences in a dedicated RTP stream, the dedicated RTP stream having a corresponding codec payload format (e.g., an HEVC payload format) and possibly having a different media type (e.g., a media type for geometry); the ability to carry a dedicated RTP stream about point cloud information such as occupancy maps, geometry, attributes, and patch sequences in a single multiplexed RTP stream; the ability to carry sequence parameter set (SPS) unit type information in an RTP header extension; the ability to carry point cloud metadata information as part of the RTP payload format, for example, using supplementary information enhancement (SEI) messages, such as information carried in a point cloud video specific SEI message and then used at the receiver for the rendering stage; the ability to support advanced codec capabilities required to provide sufficiently high quality of point cloud video such as HEVC Main 10 Profile, Main Tier, Level 5.1.

[0181] Embodiment 8 may include a new SDP attribute to indicate viewport-dependent handling capabilities of point cloud video carried as part of an RTP stream, including one or more of: the ability to signal the desired viewport using RTCP feedback messages, as described in detail in PCT / US2017 / 020041, entitled “REGION OF INTEREST SIGNALING FOR STREAMING THREE-DIMENSIONAL VIDEO INFORMATION,” filed on February 28, 2017 (hereinafter referred to as “PCT / US2017 / 020041”), the entire contents of which are incorporated herein by reference; and the ability to signal the actual transmitted viewport using RTP header extension messages, as described in detail in PCT / US2017 / 020041.

[0182] Embodiment 9 may include SDP attributes as described in Embodiment 7 or some other embodiments herein, wherein two or more RTP streams may be negotiated accordingly.

[0183] Embodiment 10 may include a single SDP attribute that combines the capabilities of Embodiment 7 and Embodiment 8 or some other embodiments herein.

[0184] Embodiment 11 may include SDP attributes as described in Embodiment 8 or some other embodiments herein, wherein one RTP stream is used for base point cloud video and another RTP stream is a viewport-optimized RTP stream having high-quality point cloud video corresponding to a desired viewport.

[0185] Embodiment 12 may include a method of RTP streaming, the method comprising: generating a dedicated codec bitstream (e.g., an HEVC bitstream) to carry information including, for example, encoding of an occupancy map, geometry, and attributes; generating a dedicated RTP stream including the codec / HEVC bitstream, such as a dedicated RTP stream for occupancy map, geometry, and attribute information, the dedicated RTP stream using a specific RTP payload format for the codec (e.g., an RTP payload format for HEVC); and multiplexing the dedicated RTP stream into a single RTP stream for V-PCC video.

[0186] Embodiment 13 may include a method as described in Embodiment 12 or some other embodiments herein, further comprising: generating another RTP stream to carry the compressed patch sequence substream; and multiplexing the other RTP stream into a single RTP stream for V-PCC video.

[0187] Embodiment 14 may include a method as described in Embodiment 12 or some other embodiments herein, further comprising: generating an RTP header extension of the multiplexed RTP stream to carry the compressed patch sequence sub-stream.

[0188] Embodiment 15 may include a method comprising: generating one or more SDP attributes to indicate a capability to carry point cloud video as part of an RTP stream.

[0189] Embodiment 16 may include a method as described in Embodiment 15 or some other embodiments herein, wherein the one or more SDP attributes indicate: the ability to carry encoded point cloud information about occupancy maps, geometry, attributes, and patch sequences in a dedicated RTP stream having a corresponding codec payload format (e.g., an HEVC payload format) and optionally having a different media type (e.g., a media type for geometry); the ability to carry a dedicated RTP stream about point cloud information such as occupancy maps, geometry, attributes, and patch sequences in a single multiplexed RTP stream; the ability to carry sequence parameter set (SPS) unit type information in an RTP header extension; the ability to carry point cloud metadata information as part of the RTP payload format, such as using a supplementary information enhancement (SEI) message, wherein the information carried in the point cloud video specific SEI message is used at the receiver for the rendering stage; or the ability to support advanced codec capabilities required to provide sufficiently high quality of point cloud video such as HEVC Main 10 Profile, Main Tier, Level 5.1.

[0190] Embodiment 17 may include a method comprising generating an SDP attribute to indicate viewport-dependent processing capabilities of a point cloud video carried as part of an RTP stream.

[0191] Embodiment 18 may include a method as described in Embodiment 17 or some other embodiments herein, wherein an SDP attribute is used to indicate: the ability to use an RTCP feedback message to signal a desired viewport, as described in detail in P112382PCT; or the ability to use an RTP header extension message to signal an actual transmitted viewport, as described in P112382PCT.

[0192] Example 19 may include an apparatus comprising means for performing one or more elements of the method described in or related to any of Examples 1-18, or any other method or process described herein.

[0193] Embodiment 20 may include one or more non-transitory computer-readable media, which include instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform one or more elements of a method described in or related to any of Embodiments 1-18 or any other method or process described herein.

[0194] Embodiment 21 may include an apparatus comprising logic components, modules, or circuits for performing one or more elements of the method described in or related to any of Embodiments 1 to 18, or any other method or process described herein.

[0195] Example 22 may include methods, techniques or processes as described or related to any of Examples 1-18, or portions or parts thereof.

[0196] Embodiment 23 may include a device comprising: one or more processors and one or more computer-readable media, wherein the one or more computer-readable media include instructions that, when executed by the one or more processors, cause the one or more processors to perform a method, technique, or process, or portion thereof, as described in or related to any one of Embodiments 1-18.

[0197] Embodiment 24 may include a signal as described or associated with any one of Embodiments 1 to 18, or a portion or part thereof.

[0198] Embodiment 25 may include a datagram, packet, frame, segment, protocol data unit (PDU) or message as described or associated with any of Embodiments 1 to 18, or a portion or part thereof, or otherwise described in the present disclosure.

[0199] Embodiment 26 may include a signal encoded with data as described or associated with any one of Embodiments 1 to 18, or a portion or part thereof, or as otherwise described in this disclosure.

[0200] Embodiment 27 may include a signal encoded with a datagram, packet, frame, segment, protocol data unit (PDU) or message as described or associated with any of Embodiments 1 to 18, or a portion or part thereof, or otherwise described in the present disclosure.

[0201] Embodiment 28 may include an electromagnetic signal carrying computer-readable instructions, wherein execution of the computer-readable instructions by one or more processors will cause the one or more processors to perform a method, technique, or process as described or related to any one of Embodiments 1 to 18, or a portion thereof.

[0202] Embodiment 29 may include a computer program comprising instructions, wherein execution of the program by a processing element causes the processing element to perform a method, technique, or process as described or related to any one of Embodiments 1 to 18, or a portion thereof.

[0203] Embodiment 30 may include signals in a wireless network as shown and described herein.

[0204] Embodiment 31 may include a method of communicating in a wireless network as shown and described herein.

[0205] Embodiment 32 may include a system for providing wireless communications as shown and described herein.

[0206] Embodiment 33 may include an apparatus for providing wireless communications as shown and described herein.

[0207] Unless explicitly stated otherwise, any of the above examples may be combined with any other example (or combination of examples). The foregoing description of one or more specific implementations provides illustration and description, but is not intended to be exhaustive or to limit the scope of the embodiments to the precise form disclosed. Modifications and variations are possible in view of the above teachings or may be obtained from the practice of the various embodiments.

[0208] Unless explicitly stated otherwise, any of the above examples may be combined with any other example (or combination of examples). The foregoing description of one or more specific implementations provides illustration and description, but is not intended to be exhaustive or to limit the scope of the embodiments to the precise form disclosed. Modifications and variations are possible in view of the above teachings or may be obtained from the practice of the various embodiments.

Claims

1. A method for real-time protocol RTP data streaming transmission, the method comprising: receiving, from a receiver in an RTP extension message, data indicating a desired viewport associated with the receiver; generating a plurality of codec bitstreams, each codec bitstream configured to include encoded information including one of occupancy map data, geometry data, or attribute data for the volumetric video; For each of the plurality of codec bitstreams, generating a dedicated RTP stream, the dedicated RTP stream comprising an RTP payload format based on the corresponding codec bitstream for the dedicated RTP stream; multiplexing each dedicated RTP stream into a resulting RTP stream for the volumetric video; as well as The resulting RTP stream is sent to the receiver, the resulting RTP stream including data representing an actual viewport of the volumetric video in the resulting RTP stream, the actual viewport corresponding to the desired viewport.

2. The method according to claim 1, further comprising: Video-based point cloud coding V-PCC is performed using the resulting RTP stream. 3 . The method of claim 1 , wherein each codec bitstream comprises a High Efficiency Video Coding (HEVC) bitstream.

4. The method according to claim 1, further comprising: generating an additional dedicated RTP stream comprising a compressed patch sequence substream; as well as The additional dedicated RTP stream is multiplexed into the resulting RTP stream for video-based point cloud coding V-PCC video.

5. The method according to claim 1, further comprising: An RTP header extension of the resulting RTP stream is generated, wherein the RTP header extension is configured to include the compressed patch sequence sub-stream.

6. The method according to claim 1, further comprising: One or more Session Description Protocol (SDP) attributes are generated to indicate a capability to carry point cloud video as part of the resulting RTP stream.

7. The method of claim 6, wherein the one or more SDP attributes for the resulting RTP stream indicate at least one of the following: The ability to carry encoded point cloud information about occupancy map data, geometry data, attribute data, and patch sequence data in a dedicated RTP stream; The capability of carrying said dedicated RTP stream regarding point cloud information including occupancy map data, geometry data, attribute data and patch sequence data in a single multiplexed RTP stream; The ability to carry sequence parameter set (SPS) unit type information in the RTP header extension; The ability to carry point cloud metadata information as part of the RTP payload format; and Ability to support codec capabilities that meet minimum quality requirements for point cloud video.

8. The method according to claim 7, wherein the RTP payload format includes at least one supplementary information enhancement SEI message, wherein the data carried in the point cloud video specific SEI message is used at the receiver for the rendering stage.

9. The method of claim 6, wherein the one or more SDP attributes indicate at least one of the following: The ability to signal the desired viewport using Real-time Control Protocol RTCP feedback messages; and The capabilities of the actual transmitted viewport are signaled using an RTP header extension message.

10. The method of claim 1, wherein each dedicated RTP stream comprises a different media type.

11. The method according to any one of claims 1 to 10, wherein the method is performed by a user equipment UE, a networking device or an access node AN.

12. A user equipment (UE) configured for real-time protocol (RTP) data streaming transmission, the UE comprising: A processing circuit, the processing circuit being configured to: receiving, from a receiver in an RTP extension message, data indicating a desired viewport associated with the receiver; generating a plurality of codec bitstreams, each codec bitstream configured to carry encoded information, the encoded information comprising one of occupancy map data, geometry data, or attribute data for the volumetric video; generating a dedicated RTP stream for each of the plurality of codec bitstreams, the dedicated RTP stream comprising an RTP payload format based on the corresponding codec bitstream for the dedicated RTP stream; as well as multiplexing each dedicated RTP stream into a resulting RTP stream for the volumetric video; and A baseband circuit coupled to the processing circuit, the baseband circuit configured to generate the resulting RTP stream for transmission to the receiver of the networked device, the resulting RTP stream transmitted to the receiver including data representing an actual viewport of the volumetric video in the resulting RTP stream, the actual viewport corresponding to the desired viewport.

13. The UE according to claim 12, wherein the processing circuit is further configured to: Video-based point cloud coding V-PCC is performed using the resulting RTP stream.

14. The UE of claim 12, wherein each codec bitstream comprises a High Efficiency Video Coding (HEVC) bitstream.

15. The UE according to claim 12, wherein the processing circuit is further configured to: generating an additional dedicated RTP stream, the additional dedicated RTP stream comprising a compressed patch sequence substream; and The additional dedicated RTP stream is multiplexed into the resulting RTP stream for video-based point cloud coding V-PCC video.

16. The UE according to claim 12, wherein the processing circuit is further configured to: An RTP header extension of the resulting RTP stream is generated, wherein the RTP header extension is configured to include the compressed patch sequence sub-stream.

17. The UE according to claim 12, wherein the processing circuit is further configured to: One or more Session Description Protocol (SDP) attributes are generated to indicate a capability to carry point cloud video as part of the resulting RTP stream.

18. The UE of claim 17, wherein the one or more SDP attributes for the resulting RTP stream indicate at least one of the following: The ability to carry encoded point cloud information about occupancy map data, geometry data, attribute data, and patch sequence data in a dedicated RTP stream; The capability of carrying said dedicated RTP stream regarding point cloud information including occupancy map data, geometry data, attribute data and patch sequence data in a single multiplexed RTP stream; The ability to carry sequence parameter set (SPS) unit type information in the RTP header extension; The ability to carry point cloud metadata information as part of the RTP payload format; and Ability to support codec capabilities that meet minimum quality requirements for point cloud video.

19. The UE of claim 18, wherein the RTP payload format comprises at least one supplementary information enhancement SEI message, wherein the data carried in the point cloud video specific SEI message is used at a receiver for a rendering stage.

20. The UE of claim 17, wherein the one or more SDP attributes indicate at least one of the following: The ability to signal the desired viewport using Real-time Control Protocol RTCP feedback messages; and The capabilities of the actual transmitted viewport are signaled using an RTP header extension message.

21. The UE of claim 12, wherein each dedicated RTP stream comprises a different media type.

22. A method for real-time protocol RTP data streaming transmission, the method comprising: Transmitting, by a receiver in an RTP extension message, data indicating a desired viewport associated with the receiver; receiving, from a sender in an RTP extension message, data indicating an actual viewport associated with a volumetric video in an RTP stream representing the volumetric video, and wherein the actual viewport corresponds to the desired viewport; demultiplexing the RTP stream representing the volumetric video into a plurality of coded bitstreams; decoding texture data from the plurality of coded bitstreams, the texture data comprising occupancy map data, geometry data, and attribute data for the volumetric video; generating a point cloud reconstruction for the volumetric video from the texture data; as well as The volumetric video is rendered having the actual viewport or a viewport within the actual viewport based on the point cloud reconstruction.

23. The method according to claim 22, further comprising: One or more supplemental information enhancement (SEI) messages are decoded from the plurality of codec bitstreams, the one or more SEI messages comprising metadata for rendering the volumetric video.

24. The method of claim 22, further comprising decoding patch sequence data from the plurality of codec bitstreams, wherein generating the point cloud reconstruction is based on the patch sequence data.

25. The method of claim 22, wherein rendering the volumetric video comprises: receiving sensor data indicative of a position of a display from a sensor associated with a device configured to display the volumetric video; as well as The volumetric video is rendered based on the sensor data indicative of the location.

26. The method of claim 22, wherein each codec bitstream comprises a High Efficiency Video Coding (HEVC) bitstream.

27. The method of claim 22, further comprising: An RTP header extension of the RTP stream is received, the RTP header extension being configured to include a compressed patch sequence sub-stream.

28. The method of claim 22, further comprising: Data including one or more Session Description Protocol (SDP) attributes is received to indicate a capability to carry point cloud video as part of the RTP stream.

29. The method of claim 28, wherein the one or more SDP attributes for the RTP stream indicate at least one of the following: The ability to carry encoded point cloud information about occupancy map data, geometry data, attribute data, and patch sequence data in a dedicated RTP stream; The capability of carrying said dedicated RTP stream regarding point cloud information including occupancy map data, geometry data, attribute data and patch sequence data in a single multiplexed RTP stream; The ability to carry sequence parameter set (SPS) unit type information in the RTP header extension; The ability to carry point cloud metadata information as part of the RTP payload format; and Ability to support codec capabilities that meet minimum quality requirements for point cloud video.

30. The method of claim 28, wherein the one or more SDP attributes indicate at least one of: The ability to signal the desired viewport using Real-time Control Protocol RTCP feedback messages; and The capabilities of the actual transmitted viewport are signaled using an RTP header extension message.

31. The method of claim 22, wherein each codec bitstream comprises a different media type.

32. The method according to any one of claims 22 to 31, wherein the method is performed by a user equipment UE, a networking device or an access node AN.

Images