Immersive teleconferencing and telepresence

Abstract

The invention relates to immersive teleconferencing and telepresence. Embodiments may relate to a user equipment (UE) configured to determine a real-time transport protocol (RTP) media stream and a supplemental information enhancement (SEI) message, the real-time transport protocol media stream including visualization data relating to a plurality of images of a location taken simultaneously, and the supplemental information enhancement message for displaying at least a portion of the visualization data; the UE is further configured to visually display the portion of the visualization data to a user of the user equipment based on the visualization data and the SEI message. Other embodiments are also described or claimed to be protected.

Description

[0001] This application is a divisional application of the invention patent application with application date of June 25, 2020, application number 202080058984.2, and invention name “Immersive Remote Conferencing and Remote Presentation”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This disclosure claims the benefit of priority to U.S. Provisional Patent Application No. 62 / 866,488, filed on June 25, 2019, and entitled “SESSION DESCRIPTION PROTOCOL (SDP)-BASED METHODS FOR IMMERSIVE TELECONFERENCING AND TELEPRESENCE.” The above-described patent application is incorporated herein by reference in its entirety. Background Art

[0004] With the increase of cellular broadband, telepresence and teleconferencing are becoming more and more common. In some use cases, multiple users can join a meeting from a remote location via a user equipment (UE) such as a cell phone, tablet, headset, or some other device. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 Depicted is a simplified architecture associated with a first example use case of various embodiments herein.

[0006] Figure 2 Depicted is a simplified alternative architecture associated with a second example use case of various embodiments herein.

[0007] Figure 3 Depicted is an overview of possible receiver architectures according to various embodiments herein.

[0008] Figure 4 Depicted are exemplary process flows associated with embodiments herein.

[0009] Figure 5 An exemplary architecture of a network system according to various embodiments is shown.

[0010] Figure 6 Examples of platforms (or "devices") are shown according to various embodiments.

[0011] Figure 7 Exemplary components of a baseband circuit and a radio front end module (RFEM) are shown according to various embodiments. DETAILED DESCRIPTION

[0012] In the following detailed description, reference is made to the accompanying drawings forming a part of the present invention, wherein like numbers represent like parts throughout the accompanying drawings, and in the accompanying drawings, embodiments in which the subject matter of the present disclosure may be practiced are shown by way of example. It should be understood that other embodiments may be used and structural or logical changes may be made without departing from the scope of the present disclosure. Therefore, the following detailed description will not have a limiting meaning.

[0013] For purposes of this disclosure, the phrase "A or B" means (A), (B), or (A and B). For purposes of this disclosure, the phrase "A, B, or C" means (A), (B), (C), (A and B), (A and C), (B and C), or (A, B, and C). The description may use the phrases "in one embodiment" or "in multiple embodiments," which may each refer to one or more of the same or different embodiments. In addition, the terms "comprising," "including," "having," and the like are used synonymously with the embodiments of the present disclosure.

[0014] The term "coupled" and its derivatives may be used herein. "Coupled" may refer to one or more of the following. "Coupled" may refer to two or more elements that are in direct physical or electrical contact. However, "coupled" may also refer to two or more elements that are in indirect contact with each other, but still cooperate or interact with each other, and may mean that one or more other elements are coupled or connected between the elements referred to as being coupled to each other. The term "directly coupled" may refer to two or more elements that are in direct contact.

[0015] Various operations may be described as multiple discrete operations in turn, in a manner that is most helpful in understanding the claimed subject matter. However, the order of description should not be construed as to imply that these operations are necessarily order dependent.

[0016] In general, embodiments herein relate to SDP-based and real-time transport protocol (RTP)-based processes to support immersive teleconferencing and telepresence. For example, embodiments may relate to SDP processes for negotiating immersive media exchange capabilities. Embodiments herein may provide many benefits, such as simplifying interoperability and SDP processing.

[0017] Figure 1 A simplified architecture 100 is depicted in relation to a first example use case of embodiments herein. It should be noted that this use case is described with respect to a remote conference with two remote individuals 105a and 105b, but in other embodiments, the architecture may include more or fewer individuals. Specifically, in Figure 1In the example embodiment, a group of colleagues may be meeting in conference room 110. The conference room may include a conference table (for participants who are actually present), a camera, and a view screen. For the purposes of this disclosure, the camera will be described as a "360-degree camera" that is capable of capturing video of a full 360-degree field of view around the camera. Specifically, the camera may include multiple separate cameras or lenses that are capable of capturing video at different angles or fields of view relative to the camera. However, it should be understood that in other embodiments, the camera may not be a full 360-degree camera, but may be configured to capture a field of view that is less than a full 360-degree field of view.

[0018] Two other individuals at 105a and 105b may not be in the same location as conference room 110 and may wish to join the meeting via teleconference. For example, participants in conference room 110 may use a screen to display shared presentations and / or video streams from individuals 105a and 105b. In this example use case, individual 105a may join a video conference from a remote location (such as their home) using a UE with a head-mounted display (HMD) and a camera that captures their video. Individual 105a may enjoy a 360-degree view of conference room 110 by, for example, turning its head. Individual 105b may join the meeting from a remote location (such as an airport) using a UE such as a mobile phone. Individual 105b may also enjoy a 360-degree view of the conference room on its mobile phone screen by turning the phone, and may further use its mobile camera to capture its own video.

[0019] In this example use case, individuals 105a and 105b may be able to see the screen in conference room 110 as part of a 360-degree video. They may also choose to focus on one or more of the various incoming video streams (e.g., camera feeds of presentations or other remote participants) using their own display devices.

[0020] Generally, the above use cases can be implemented in two possible configurations. The first configuration is as described above with respect to Figure 1 In this configuration, communications between conference room 110 and various individuals 105a and 105b (or other individuals that may join using one or more UEs) may be established without the support of a media-aware network element such as a conference server. In this embodiment, communications such as audio or video may be transmitted directly between conference room 110 and individuals 105a / 105b via session audio / video stream 125. Specifically, the session audio or video stream may provide audio or video data (e.g., data of individual 105a or 105b) between the conference room and the UE of individual 105a / 105b.

[0021] In addition, the individual 105a / 105b may provide viewport-related information 115 to the conference room 110, which in turn may provide a viewport-dependent audio or video stream at 120. As used herein, the term "viewport" may relate to the field of view of the conference room 110 displayed by the UE. For example, the term "viewport" may relate to the zoom level, a specific direction, etc. of the conference room 110. More specifically, an individual such as the individual 105a / 105b may change its viewport through a variety of mechanisms, such as interaction with the UE (e.g., swiping on the UE's touch screen or some other interaction), rotation of the UE, and commands such as verbal commands, gestures, etc. The change may be transmitted via the viewport-related information 115. In response, the viewport-dependent audio or video stream 120 may be changed to include corresponding changes to the video information (such as the field of view, zoom level, stereo aspects of the audio stream, etc.) provided to the UE of the individual 105a / 105b.

[0022] Figure 2 A simplified alternative architecture 200 is depicted in connection with a second example use case of various embodiments herein. Similar to architecture 100, architecture 200 may include conference room 210 and individuals 205a / 205b, which may be similar to conference room 110 and individuals 105a and 105b. In the second case, the call is established using network functions, which may be performed by a media resource function (MRF) or a media control unit (MCU). Specifically, the MRF / MCU may be provided by a conference server 230, which may be communicatively located between individuals 205a / 205b and conference room 210. In some embodiments, conference server 230 may be an Internet Protocol (IP) Multimedia Subsystem (IMS) server operable to provide MRF or MCU functions.

[0023] In this example use case, server 230 may receive viewport independent stream 240 from conference room 210. Specifically, server 230 may receive audio or video data that is not based on a particular viewport direction. Server 230 may also be communicatively coupled with conference room 210 to provide session audio or visual data stream 235, which may be generally similar to the above description of the session audio or visual data stream. Figure 1 Stream 125 is described.

[0024] The server 230 may also be communicatively coupled to the individuals 205a and 205b via data streams such as a viewport-dependent stream 220, viewport-related information 215, and a session audio / video stream 225, which may be similar to the viewport-dependent stream 120, the viewport-related information 115, and the session audio / video stream 125, respectively.

[0025] In general, use cases of architecture 100 or 200 may enable individuals 205a / 205b joining teleconferences and telepresence conferences to gain an immersive experience through two-way audio and one-way immersive video, e.g., a remote user participating in a conference wearing an HMD may send audio and optional 2D video (e.g., presentations, screen sharing, and / or capture of the user themselves), but receive stereo or immersive voice / audio and immersive video captured by an omnidirectional camera in a conference room connected to a fixed network.

[0026] It should be understood that these architectures are exemplary architectures, and other embodiments may include aspects of both architectures or additional aspects. For example, in some embodiments, an individual's UE may be directly coupled to a conference room, while another individual's UE may be communicatively coupled to a server. In some embodiments, when an individual's UE may be directly coupled to a conference room to obtain a session audio or visual data stream, the individual's UE may be communicatively coupled to a server to obtain viewport-related information. It should also be understood that, as used herein, the concept of "coupled to a conference room" is used to describe a communicative coupling with, for example, a server or other electronic device that is providing or receiving audio or video data from a speaker or camera in a conference room.

[0027] The architecture 100 or 200 may have many exemplary features. One such feature may be that multiple single-user participants may be allowed, such as individuals 105a / 105b / 205a / 205b. Communications between single users may be structured as multimedia telephony services for IMS (MTSI) or telepresence communications as defined by the 3rd Generation Partnership Project (3GPP) specification. In other embodiments, communications may be structured as multi-stream MTSI (MSMTSI). In embodiments using MSMTSI, media data may be transmitted in separate media streams.

[0028] Another such feature may be the presentation of a single 360 ​​degree camera at each location (e.g., in a respective one of conference rooms 110 or 210) in a multi-party conference scenario involving multiple physical locations. As described above, the camera may capture multiple two-dimensional images. The images may then be "stitched" together or combined into an image with a wider (e.g., 360 degree) field of view. Various embodiments may use: in-camera stitching, where the images are stitched together by the camera itself; or network-based stitching, where the images are stitched together by a server (such as server 230).

[0029] In the case of camera stitching, the stitched immersive video may be sent from conference room 210 to a server, e.g., in viewport-independent stream 240, and then sent from server 230 to individual 205a or 205b, e.g., via viewport-dependent stream 220, session audio / video stream 225, or both. If the use case is a one-to-one session conference between conference room 210 and a user (e.g., individual 105a / 105b / 205a / 205b), a server such as server 230 acting as a media gateway may not be needed.

[0030] In the use case of network-based stitching, the various two-dimensional images may be sent from conference room 210 to server 230, for example, in viewport-independent streams 240. The server may perform decoding, stitching, and re-encoding of the images to produce an immersive video (e.g., a video or image with a larger field of view), which is then distributed to remote participants as described above.

[0031] In some embodiments, it may be desirable for various elements of the architecture 100 or 200 (such as cameras or electronics in conference room 110 / 210, UEs of individuals 105a / 105b / 205a / 205b, or server 230) to support MTSI or IMS telepresence codecs, protocols, or transport capabilities associated with encoding, delivery, and consumption of immersive voice / audio and immersive video. It may also be desirable for elements such as cameras or electronics in conference room 110 or 210 or server 230 to be able to send viewport-dependent streams, viewport-independent streams, or both. It may also be desirable to build an architecture so that changes in viewport direction are delivered and relevant viewport-dependent streams are updated in a manner that reduces or eliminates delay-prone signaling (such as SIP renegotiation). It may also be desirable for the architecture to establish a suitable coordinate system to serve as a standard way of communicating viewport direction between various elements of the architecture 100 or 200.

[0032] In general, embodiments of the present invention may include implementations of 3GPP Technical Specification (TS) 26.114 v16.2.0 (June 2019) for immersive teleconferencing. Figure 4 .1. In addition, the following can be observed.

[0033] For in-camera stitching, the stitched immersive video can be sent from the conference room to a conference server (e.g., MSMTSIMRF) or directly to remote participants (e.g., one-to-one sessions) in one or more RTP streams (e.g., established via SDP). If block or sub-picture based delivery optimization is used, multiple RTP streams can be used. In these embodiments, the RTP streams can be, for example, Figure 1 or Figure 2 The viewport-dependent stream 120 / 220 or the viewport-independent stream 240.

[0034] For network-based stitching, multiple RTP streams may be established between the conference server and the conference room (e.g., using MSMTSI, via SDP), for example, as multiple viewport-independent streams 240, each of which may carry a specific two-dimensional image or video capture. These RTP streams may then be sent from the conference room to the conference server, for example, in viewport-dependent streams 220, and the conference server may perform decoding, stitching, and re-encoding to produce one or more RTP streams containing immersive video, which may then be distributed to remote participants (e.g., again via MSMTSI). If block- or sub-picture-based delivery optimization is used, multiple RTP streams may be used for immersive video.

[0035] Figure 3 Depicted is an overview of possible receiver architectures according to various embodiments herein. Specifically, Figure 3 An overview of possible receiver architectures is provided that reconstruct spherical video in an MTSI or IMS telepresence UE, such as may be used by an individual 105a / 105b / 205a / 205b, etc. It will be appreciated that Figure 3 It may not represent an actual implementation, but can be considered to depict a logical set of receiver functions. Figure 3 Some or all of the elements may be implemented in a baseband receiver. In other embodiments, the various elements may be implemented in or by a processor or element of a radio frequency (RF) transceiver, or elsewhere within the UE.

[0036] Initially, the RTP receiver 300 may receive one or more RTP streams 301. The RTP streams 301 may be received, for example, from a server (such as the server 230) or a conference room (such as the conference room 110 or 210 described above). More specifically, the one or more RTP streams 301 may be received in a viewport-dependent stream (such as the viewport-dependent stream 120 or 220). Based on the one or more received RTP streams 301, the RTP receiver 300 may parse, possibly decrypt, or otherwise process the one or more RTP streams 301 to generate an elementary stream 303, which is provided to a high-efficiency video coding (HEVC) decoder 305. The HEVC decoder 305 may obtain a decoder output signal from the elementary stream 303. The decoder output signal may include image data, which may be referred to herein as "texture" 307. The HEVC decoder 305 may also obtain decoder metadata 309. The decoder metadata 309 may include one or more supplemental information enhancement (SEI) messages (e.g., information carried in an omni-directional video-specific SEI message) for use by the UE in the rendering stage. In particular, decoder metadata 309 may be used by texture-sphere mapping function 310 to generate spherical video 311 (or a portion thereof) based on the decoded output signal (e.g., texture 307). Viewport rendering module 315 may then generate a viewport from spherical video signal 311 (or a portion thereof) by taking into account viewport position information from a sensor, display characteristics, and other possible metadata such as initial viewport information. The rendered signal may then be provided to a display of a UE, such as a touch screen, an eyepiece, or some other display, so that a user of the UE may view the rendered image.

[0037] For 360-degree video, potential solutions may consider one or more of the following principles. Specifically, the RTP stream 301 may contain an HEVC bitstream with omni-directional video-specific SEI messages. The omni-directional video-specific SEI messages may be similar to those defined in the International Organization for Standardization / International Electrotechnical Commission (ISO / IEC) standard 23008-2 (2017). The elementary stream 303 may be based on the Omni-directional Media Format (OMAF) specification ISO / IEC 23090-2 (2019) section 10.1.2.2.

[0038] Typically, the SEI message in the elementary stream 303 with decoder rendering metadata may include various information. For example, the SEI message may include partition encapsulation information, for example, carrying a partition encapsulation format indication and any coverage restrictions. The SEI message may also include projection mapping information indicating the projection format used, such as equirectangular projection (ERP) or cubic map projection (CMP). The SEI message may also include information related to padding, for example, whether there is padding or guard bands in the encapsulated picture. The SEI message may also include information related to the frame encapsulation arrangement, which indicates the frame encapsulation format used for stereoscopic content. The SEI message may also include content pre-rotation information indicating the amount of spherical rotation (if any) applied to the spherical signal before projection and partition encapsulation on the encoder side.

[0039] The decoder metadata information 309 associated with the relevant SEI message may then be used to render an output signal, such as a decoded picture or "texture" 307. The decoder metadata 309 may be used when performing rendering operations such as partition packing, projective demapping, and rotation to create spherical content for each eye of the user (if necessary).

[0040] In some embodiments, viewport-dependent processing may be supported for both point-to-point sessions and multi-party conference scenarios. For example, in viewport-related information 115 or 215, viewport-dependent processing may be implemented by sending RTP control protocol (RTCP) feedback or RTP header extension messages with desired viewport information from an MTSI receiver (e.g., an individual UE, such as individual 105a / 105b / 205a / 205b). The corresponding viewport-dependent information may then be encoded and sent by the MTSI sender (e.g., conference room 110 in viewport-dependent stream 120) or by the server 230 (e.g., in viewport-dependent stream 220).

[0041] This processing flow can provide higher resolution for the desired viewport than the viewport-independent approach.In general, viewport-dependent processing based on tile and sub-picture coding can be based on RTP / RTCP-based protocols supported by MTSI and IMS-based remote presentation.

[0042] In order to achieve video quality that is considered acceptable for viewport-dependent virtual reality (VR) services, it may be desirable for the video codecs supporting VR in MTSI and IMS telepresence to be consistent with OMAF, 3GPP TS 26.118v 15.1.0 (December 2018), or both. It may be desirable for MTSI client (such as may be used by a UE) and MTSI gateway (such as may be used by server 230) codec requirements to be consistent with these recommended video codec requirements for VR support.

[0043] The process associated with negotiation of SEI messages for transporting decoder rendering metadata may be similar to the process described in the RTP payload format for HEVC in Internet Engineering Task Force (IETF) Request for Comments (RFC) 7798 (March 2016). In particular, the process may involve exposing SEI messages related to decoder rendering metadata for omnidirectional media in the SDP using the "sprop-sei" parameter, which may allow the transmission of one or more SEI messages describing bitstream characteristics. When present, the decoder may rely on the bitstream characteristics described in the SEI messages throughout the session. In some embodiments, both the MTSI client and the MTSI gateway may support the RTP payload format for VR support.

[0044] As described above, SEI messages may be present in the decoder metadata 309. SEI messages may include one or more of the following: equirectangular projection SEI message; cubic projection SEI message; sphere rotation SEI message; and partition packing SEI message. For stereoscopic video support in one-to-one video telephony scenarios or multi-party video conferencing scenarios, support for a subset of frame packing arrangement SEI messages may be required.

[0045] Figure 4 Depicted is an exemplary process flow associated with embodiments herein. It should be appreciated that for in-camera stitching, the stitched immersive video may be sent from a conference room (e.g., conference room 110 or 210) to a server (e.g., server 230) or directly to a remote participant (e.g., to an individual 105a / 105b / 205a / 205b) in one or more RTP streams as described above (e.g., established via SDP). Additionally, in some embodiments, multiple RTP streams may be used if block or sub-picture based delivery optimization is used.

[0046] Figure 4 There may be three communication elements in the process flow of . Specifically, the process flow may include an individual UE at 405, which may be similar to, for example, the UE of individual 105a / 105b / 205a / 205b, etc. The process flow may also include a server 430, which may be similar to, for example, server 230. The process flow may also include an electronic device at a conference room 410, which may be similar to, for example, conference room 110 or 210.

[0047] Initially, at 402, the UE at 405 may send an SDP offer to the server 430 indicating immersive media capabilities including 360-degree video support. In some embodiments, the UE at 405 may also include viewport-dependent processing capabilities in the SDP offer, such as based on various embodiments described herein. If viewport-dependent processing is proposed, the SDP offer at 402 may include two or more RTP streams, such as one RTP stream for basic 360-degree video and another viewport-optimized RTP stream with high-quality 360-degree video corresponding to a desired viewport.

[0048] At 404, server 430 may respond to UE 405 with an SDP answer confirming the immersive media capabilities of server 430, including 360-degree video support. In some embodiments, server 430 may also include an indication of viewport-dependent processing capabilities in the SDP answer at 404. If viewport-dependent processing is accepted or included, the SDP answer from server 430 may include multiple RTP streams.

[0049] At 406, conference room 410 may provide viewport independence information to server 430, and then at 408, server 430 may stream the RTP media stream with the immersive media including the 360-degree video to the UE at 405. The 360-degree video transmission may be based on the RTP payload format of HEVC carrying SEI messages.

[0050] In some embodiments, at 410, the UE at 405 may signal the desired viewport information using a dedicated RTCP feedback message. In this embodiment, at 412, the server 430 may respond to the UE at 405. Information on the actual transmitted viewport may also be included in the RTP media stream. If two RTP streams are negotiated, the viewport optimized RTP stream containing the high quality 360 degree video may contain this information.

[0051] It should be understood that this embodiment is intended as an exemplary embodiment, and other embodiments may include more or fewer elements, elements in a different order than depicted, etc. It should also be understood that although the elements herein are described as being associated with 360-degree video, in other embodiments the media streams may be associated with a narrower field of view as described above.

[0052] Figure 5An exemplary architecture of a system 500 of a network according to various embodiments is shown. The following description is provided for an exemplary system 500 operating in conjunction with LTE system standards and 5G or NR system standards provided by 3GPP technical specifications. However, the exemplary embodiments are not limited in this regard, and the embodiments may be applied to other networks that benefit from the principles described herein, such as future 3GPP systems (e.g., sixth generation (6G)) systems, IEEE 802.16 protocols (e.g., WMAN, WiMAX, etc.), and the like.

[0053] like Figure 5 As shown, system 500 includes UE 501a and UE 501b (collectively referred to as "UE 501"), which can be similar to, for example, the UE of individual 105a / 105b / 205a / 205c or the UE described at 405. In this example, UE 501 is shown as a smart phone (e.g., a handheld touch screen mobile computing device capable of connecting to one or more cellular networks), but may also include any mobile or non-mobile computing device, such as a consumer electronic device, a mobile phone, a smart phone, a feature phone, a tablet computer, a wearable computer device, a personal digital assistant (PDA), a pager, a wireless handheld device, a desktop computer, a laptop computer, an in-vehicle infotainment (IVI), an in-car entertainment (ICE) device, a dashboard, a head-up display (HUD) device, an on-board diagnostic (OBD) device, a dashtop mobile equipment (DME), a mobile data terminal (MDT), an electronic engine management system (EEMS), an electronic / engine control unit (ECU), an electronic / engine control module (ECM), an embedded system, a microcontroller, a control module, an engine management system (EMS), a networked or "smart" appliance, an MTC device, an M2M, an IoT device, etc.

[0054] In some embodiments, any of the UEs 501 may be an IoT UE, which may include a network access layer designed for low-power IoT applications that utilize short-term UE connections. The IoT UE may utilize technologies such as M2M or MTC to exchange data with an MTC server or device via PLMN, ProSe or D2D communications, sensor networks, or IoT networks. M2M or MTC data exchanges may be machine-initiated data exchanges. The IoT network describes interconnected IoT UEs, which may include uniquely identifiable embedded computing devices (within the Internet infrastructure) with short-lived connections. The IoT UE may execute background applications (e.g., keep-alive messages, status updates, etc.) to facilitate connectivity to the IoT network.

[0055] UE 501 may be configured to connect, e.g., be communicatively coupled to RAN 510. In an embodiment, RAN 510 may be an NG RAN or 5G RAN, E-UTRAN, or a legacy RAN, such as UTRAN or GERAN. As used herein, the term "NG RAN" or the like may refer to RAN 510 operating in NR or 5G system 500, while the term "E-UTRAN" or the like may refer to RAN 510 operating in LTE or 4G system 500. UE 501 utilizes connections (or channels) 503 and 504, respectively, each of which includes a physical communication interface or layer (discussed in further detail below).

[0056] In this example, connections 503 and 504 are shown as air interfaces to achieve communication coupling, and may be consistent with a cellular communication protocol, such as a GSM protocol, a CDMA network protocol, a PTT protocol, a POC protocol, a UMTS protocol, a 3GPP LTE protocol, a 5G protocol, a NR protocol, and / or any other communication protocol discussed herein. In an embodiment, the UE 501 may directly exchange communication data via the ProSe interface 505. The ProSe interface 505 may alternatively be referred to as a SL interface 505, and may include one or more logical channels, including but not limited to PSCCH, PSSCH, PSDCH, and PSBCH.

[0057] UE 501b is shown as being configured to access AP 506 (also referred to as "WLAN node 506", "WLAN 506", "WLAN terminal 506", "WT 506", etc.) via connection 507. Connection 507 may include a local wireless connection, such as a connection consistent with any IEEE 802.11 protocol, wherein AP 506 will include wireless fidelity Router. In this example, the AP 506 shown is connected to the Internet without being connected to the core network of the wireless system (described in further detail below). In various embodiments, the UE 501b, the RAN 510, and the AP 506 may be configured to utilize LWA operation and / or LWIP operation. The LWA operation may involve the UE 501b in the RRC_CONNECTED state being configured by the RAN nodes 511a-b to utilize the radio resources of LTE and WLAN. The LWIP operation may involve the UE 501b using the WLAN radio resources (e.g., connection 507) via an IPsec protocol tunnel to authenticate and encrypt packets (e.g., IP packets) sent through the connection 507. IPsec tunneling may include encapsulating the entire original IP packet and adding a new packet header, thereby protecting the original header of the IP packet.

[0058] The RAN 510 includes one or more AN nodes or RAN nodes 511a and 511b (collectively referred to as "multiple RAN nodes 511" or "RAN node 511") that enable connections 503 and 504. As used herein, the terms "access node", "access point", etc. may describe equipment that provides radio baseband functions for data and / or voice connections between a network and one or more users. These access nodes may be referred to as BS, gNB, RAN node, eNB, NodeB, RSU, TRxP or TRP, etc., and may include ground stations (e.g., terrestrial access points) or satellite stations that provide coverage within a geographic area (e.g., a cell). As used herein, the terms "NG RAN node" and the like may refer to a RAN node 511 (e.g., a gNB) operating in an NR or 5G system 500, while the terms "E-UTRAN node" and the like may refer to a RAN node 511 (e.g., an eNB) operating in an LTE or 4G system 500. According to various embodiments, the RAN node 511 may be implemented as one or more of dedicated physical devices such as a macrocell base station and / or a low power (LP) base station for providing a femtocell, picocell or other similar cell with a smaller coverage area, smaller user capacity or higher bandwidth than a macrocell.

[0059] In some embodiments, all or part of the multiple RAN nodes 511 may be implemented as one or more software entities running on a server computer as part of a virtual network that may be referred to as a CRAN and / or a virtual baseband unit pool (vBBUP). In these embodiments, the CRAN or vBBUP may implement RAN functional partitioning such as PDCP partitioning, where the RRC and PDCP layers are operated by the CRAN / vBBUP, while other L2 protocol entities are operated by individual RAN nodes 511; MAC / PHY partitioning, where the RRC, PDCP, RLC, and MAC layers are operated by the CRAN / vBBUP, and the PHY layer is operated by individual RAN nodes 511; or "lower PHY" partitioning, where the RRC, PDCP, RLC, MAC layer, and upper portions of the PHY layer are operated by the CRAN / vBBUP, and the lower portions of the PHY layer are operated by individual RAN nodes 511. This virtualization framework allows idle processor cores of multiple RAN nodes 511 to execute other virtualized applications. In some specific implementations, separate RAN nodes 511 may represent a single RAN node 511 connected to the network via a separate F1 interface ( Figure 5In some implementations, the gNB-DU may include one or more remote radio heads or RFEMs, and the gNB-CU may be operated by a server (not shown) located in the RAN 510 or by a server pool in a manner similar to CRAN / vBBUP. In addition or alternatively, one or more of the RAN nodes 511 may be a next generation eNB (ng-eNB), which is a RAN node that provides E-UTRA user plane and control plane protocol terminals to multiple UEs 501 and is connected to the 5GC via an NG interface (discussed below).

[0060] In a V2X scenario, one or more of the RAN nodes 511 may be an RSU or act as an RSU. The term "roadside unit" or "RSU" may refer to any traffic infrastructure entity used for V2X communication. The RSU may be implemented in or by a suitable RAN node or a stationary (or relatively stationary) UE, wherein an RSU implemented in or by a UE may be referred to as a "UE-type RSU", an RSU implemented in or by an eNB may be referred to as an "eNB-type RSU", an RSU implemented in or by a gNB may be referred to as a "gNB-type RSU", and so on. In one example, the RSU is a computing device coupled to an RF circuit located on the roadside that provides connectivity support to a passing vehicle UE 501 (vUE 501). The RSU may also include internal data storage circuitry for storing intersection map geometry, traffic statistics, media, and applications / software for sensing and controlling ongoing vehicle and pedestrian traffic. The RSU may operate on the 5.9 GHz Direct Short Range Communication (DSRC) band to provide extremely low latency communications required for high-speed events, such as collision avoidance, traffic warnings, etc. In addition or alternatively, the RSU may operate on the cellular V2X band to provide the aforementioned low latency communications as well as other cellular communication services. In addition or alternatively, the RSU may operate as a Wi-Fi hotspot (2.4 GHz band) and / or provide connectivity to one or more cellular networks to provide uplink and downlink communications. Some or all of the computing device and the RSU's RF circuitry may be packaged in a weather-resistant enclosure suitable for outdoor installation, and may include a network interface controller to provide a wired connection (e.g., Ethernet) to a traffic signal controller and / or backhaul network.

[0061] Any of the multiple RAN nodes 511 may serve as an endpoint for the air interface protocol and may be the first point of contact for multiple UEs 501. In some embodiments, any of the multiple RAN nodes 511 may perform various logical functions of the RAN 510, including but not limited to the functions of a radio network controller (RNC), such as radio bearer management, uplink and downlink dynamic radio resource management and data packet scheduling, and mobility management.

[0062] In an embodiment, UE 501 may be configured to communicate with each other or with any AN node in RAN node 511 using OFDM communication signals on a multi-carrier communication channel according to various communication techniques, such as but not limited to OFDMA communication techniques (e.g., for downlink communication) or SC-FDMA communication techniques (e.g., for uplink and ProSe or sidelink communication), but the scope of the embodiment is not limited in this respect. OFDM signals may include multiple orthogonal subcarriers.

[0063] In some embodiments, a downlink resource grid may be used for downlink transmissions from any of the RAN nodes 511 to the UE 501, while uplink transmissions may utilize similar techniques. The grid may be a time-frequency grid, referred to as a resource grid or a time-frequency resource grid, which is a physical resource in the downlink in each time slot. For OFDM systems, such a time-frequency plane representation is a common practice, which makes wireless resource allocation intuitive. Each column and each row of the resource grid corresponds to an OFDM symbol and an OFDM subcarrier, respectively. The duration of the resource grid in the time domain corresponds to a time slot in a radio frame. The smallest time-frequency unit in the resource grid is represented as a resource element. Each resource grid includes a plurality of resource blocks, which describe the mapping of certain physical channels to resource elements. Each resource block includes a set of resource elements; in the frequency domain, this may represent the minimum amount of resources that can currently be allocated. Such resource blocks are used to transmit several different physical downlink channels.

[0064] According to various embodiments, UE 501 and RAN node 511 communicate data (e.g., transmit data and receive data) through a licensed medium (also referred to as "licensed spectrum" and / or "licensed frequency band") and an unlicensed shared medium (also referred to as "unlicensed spectrum" and / or "unlicensed frequency band"). The licensed spectrum may include channels operating in a frequency range of approximately 400 MHz to approximately 3.8 GHz, and the unlicensed spectrum may include a 5 GHz frequency band.

[0065] To operate in the unlicensed spectrum, the UE 501 and the RAN node 511 may operate using LAA, eLAA, and / or feLAA mechanisms. In these implementations, the UE 501 and the RAN node 511 may perform one or more known medium sensing operations and / or carrier sensing operations to determine whether one or more channels in the unlicensed spectrum are unavailable or otherwise occupied before transmitting in the unlicensed spectrum. The medium / carrier sensing operations may be performed according to a listen-before-talk (LBT) protocol.

[0066] LBT is a mechanism by which equipment (e.g., UE 501, RAN node 511, etc.) senses a medium (e.g., a channel or carrier frequency) and transmits when the medium is sensed to be idle (or when a particular channel in the medium is sensed to be unoccupied). The medium sensing operation may include a CCA that utilizes at least an ED to determine whether other signals are present on the channel in order to determine whether the channel is occupied or idle. The LBT mechanism allows cellular / LAA networks to coexist with existing systems in unlicensed spectrum and with other LAA networks. ED may include sensing RF energy over a period of time on an expected transmission band and comparing the sensed RF energy to a predefined or configured threshold.

[0067] Typically, existing systems in the 5 GHz band are WLANs based on IEEE 802.11 technology. WLANs use a contention-based channel access mechanism called CSMA / CA. Here, when a WLAN node (e.g., a mobile station (MS) such as UE 501, AP 506, etc.) intends to transmit, the WLAN node may first perform CCA before transmission. In addition, in the case where more than one WLAN node senses the channel as idle and transmits at the same time, a backoff mechanism is used to avoid conflicts. The backoff mechanism may be a counter randomly introduced within the CWS that increases exponentially when a conflict occurs and is reset to a minimum value when the transmission is successful. The LBT mechanism designed for LAA is somewhat similar to the CSMA / CA of WLAN. In some specific implementations, the LBT process for a DL or UL transmission burst (including PDSCH or PUSCH transmission) may have a LAA contention window of variable length between X and Y ECCA slots, where X and Y are the minimum and maximum values ​​of the CWS of LAA. In one example, the minimum CWS for LAA transmissions may be 9 microseconds (μs); however, the size of the CWS and MCOT (eg, transmission burst) may be based on government regulatory requirements.

[0068] The LAA mechanism is built on the CA technology of the LTE-Advanced system. In CA, each aggregated carrier is called a CC. A CC can have a bandwidth of 1.4, 3, 5, 10, 15, or 20 MHz, and up to five CCs can be aggregated, so the maximum aggregated bandwidth is 100 MHz. In an FDD system, the number of aggregated carriers can be different for DL ​​and UL, where the number of UL CCs is equal to or lower than the number of DL component carriers. In some cases, each CC may have a different bandwidth from other CCs. In a TDD system, the number of CCs and the bandwidth of each CC are typically the same for DL ​​and UL.

[0069] CA also includes individual serving cells to provide individual CCs. The coverage of the serving cells may be different, for example, because CCs on different frequency bands will experience different path losses. The primary serving cell or PCell may provide the PCC for both UL and DL, and may handle activities related to RRC and NAS. Other serving cells are called SCells, and each SCell may provide individual SCCs for both UL and DL. SCCs may be added and removed as needed, and changing PCCs may require the UE 501 to undergo switching. In LAA, eLAA, and feLAA, some or all of the SCells may operate in an unlicensed spectrum (referred to as "LAA SCells"), and the LAA SCells are assisted by the PCells operating in the licensed spectrum. When a UE is configured with more than one LAA SCell, the UE may receive UL grants on the configured LAA SCells, indicating different PUSCH start positions within the same subframe.

[0070] The PDSCH carries user data and higher layer signaling to multiple UEs 501. The PDCCH carries, among other information, information about the transport format and resource allocation related to the PDSCH channel. It can also inform the UE 501 about the transport format, resource allocation, and HARQ information related to the uplink shared channel. Typically, downlink scheduling (allocation of control and shared channel resource blocks to UE 501b within a cell) can be performed on any of the RAN nodes 511 based on channel quality information fed back from any of the UEs 501. Downlink resource allocation information can be sent on the PDCCH for (e.g., allocated to) each of the multiple UEs 501.

[0071] PDCCH uses CCE to transmit control information. Before being mapped to resource elements, the PDCCH complex-valued symbols can first be organized into quadruplets, which can then be arranged using a sub-block interleaver for rate matching. One or more of these CCEs can be used to transmit each PDCCH, where each CCE can correspond to nine sets of four physical resource elements, respectively, called REGs. Four quadrature phase shift keying (QPSK) symbols can be mapped to each REG. Depending on the size of the DCI and channel conditions, one or more CCEs can be used to transmit the PDCCH. There may be four or more different PDCCH formats defined in LTE with different numbers of CCEs (e.g., aggregation levels, L=1, 2, 4, or 8).

[0072] Some embodiments may use the concept of resource allocation for control channel information, which is an extension of the above concept. For example, some embodiments may utilize EPDCCH that uses PDSCH resources for control information transmission. One or more ECCEs may be used to transmit EPDCCH. Similar to the above, each ECCE may correspond to nine sets of four physical resource elements, referred to as EREG. In some cases, ECCE may have other numbers of EREGs.

[0073] The RAN nodes 511 may be configured to communicate with each other via an interface 512. In an embodiment where the system 500 is an LTE system (e.g., when the CN 520 is an EPC), the interface 512 may be an X2 interface 512. The X2 interface may be defined between two or more RAN nodes 511 (e.g., two or more eNBs, etc.) connected to the EPC 520, and / or between two eNBs connected to the EPC 520. In some specific implementations, the X2 interface may include an X2 user plane interface (X2-U) and an X2 control plane interface (X2-C). The X2-U may provide a flow control mechanism for user packets transmitted over the X2 interface, and may be used to transmit information about the delivery of user data between eNBs. For example, the X2-U may provide specific sequence number information about user data transmitted from the MeNB to the SeNB; information about the successful in-sequence delivery of PDCP PDUs from the SeNB to the UE 501 for user data; information about PDCP PDUs that are not delivered to the UE 501; information about the current minimum expected buffer size at the SeNB for transmitting user data to the UE; and the like. X2-C can provide intra-LTE access mobility functions, including context transfer from source eNB to target eNB, user plane transmission control, etc.; load management functions; and inter-cell interference coordination functions.

[0074] In an embodiment where the system 500 is a 5G or NR system, the interface 512 may be an Xn interface 512. The Xn interface is defined between two or more RAN nodes 511 (e.g., two or more gNBs, etc.) connected to the 5GC 520, between a RAN node 511 (e.g., a gNB) and an eNB connected to the 5GC 520, and / or between two eNBs connected to the 5GC 520. In some specific implementations, the Xn interface may include an Xn user plane (Xn-U) interface and an Xn control plane (Xn-C) interface. The Xn-U may provide non-guaranteed delivery of user plane PDUs and support / provide data forwarding and flow control functions. The Xn-C may provide management and error handling functions for managing the functions of the Xn-C interface; mobility support for the UE 501 in a connected mode (e.g., CM-CONNECTED) includes functions for managing UE mobility in a connected mode between one or more RAN nodes 511. The mobility support may include context transfer from the old (source) serving RAN node 511 to the new (target) serving RAN node 511; and control of the user plane tunnel between the old (source) serving RAN node 511 and the new (target) serving RAN node 511. The protocol stack of Xn-U may include a transport network layer built on an IP transport layer, and a GTP-U layer on top of a UDP and / or IP layer for carrying user plane PDUs. The Xn-C protocol stack may include an application layer signaling protocol (referred to as the Xn Application Protocol (Xn-AP)) and a transport network layer built on SCTP. SCTP may be on top of the IP layer and may provide guaranteed delivery of application layer messages. In the transport IP layer, point-to-point transport is used to deliver signaling PDUs. In other specific implementations, the Xn-U protocol stack and / or the Xn-C protocol stack may be the same or similar to the user plane and / or control plane protocol stacks shown and described herein.

[0075] RAN 510 is shown as being communicatively coupled to a core network—in this embodiment, communicatively coupled to a core network (CN) 520. CN 520 may include a plurality of network elements 522 configured to provide various data and telecommunication services to customers / users (e.g., users of UE 501) connected to CN 520 via RAN 510. The components of CN 520 may be implemented in one physical node or separate physical nodes, including components for reading and executing instructions from a machine-readable or computer-readable medium (e.g., a non-transitory machine-readable storage medium). In some embodiments, NFV may be used to virtualize any or all of the above-mentioned network node functions via executable instructions stored in one or more computer-readable storage media (described in further detail below). A logical instance of CN 520 may be referred to as a network slice, and a logical instance of a portion of CN 520 may be referred to as a network sub-slice. NFV architecture and infrastructure may be used to virtualize one or more network functions onto physical resources comprising a combination of industry-standard server hardware, storage hardware, or switches (alternatively performed by proprietary hardware). In other words, the NFV system may be used to perform a virtual or reconfigurable implementation of one or more EPC components / functions.

[0076] Generally speaking, the application server 530 may be an element that provides applications that use IP bearer resources with the core network (e.g., UMTS PS domain, LTE PS data services, etc.). The application server 530 may also be configured to support one or more communication services for the UE 501 via the EPC 520 (e.g., VoIP sessions, PTT sessions, group communication sessions, social network services, etc.).

[0077] In an embodiment, CN 520 may be a 5GC (referred to as "5GC 520" or the like), and RAN 510 may be connected to CN 520 via an NG interface 513. In an embodiment, NG interface 513 may be divided into two parts: an NG user plane (NG-U) interface 514, which carries traffic data between RAN node 511 and UPF; and an S1 control plane (NG-C) interface 515, which is a signaling interface between RAN node 511 and AMF.

[0078] In an embodiment, CN 520 may be a 5G CN (referred to as "5GC 520", etc.), while in other embodiments, CN 520 may be an EPC. In the case where CN 520 is an EPC (referred to as "EPC 520", etc.), RAN 510 may be connected to CN 520 via an S1 interface 513. In an embodiment, S1 interface 513 may be divided into two parts: an S1 user plane (S1-U) interface 514, which carries traffic data between RAN node 511 and S-GW; and an S1-MME interface 515, which is a signaling interface between RAN node 511 and MME.

[0079] Figure 6 An example of a platform 600 (or "device 600") according to various embodiments is shown. In an embodiment, the computer platform 600 may be suitable for use as a UE 501, an application server 530, a UE of an individual 105a / 105b / 205a / 205b, or a UE at 405, and / or any other element / device discussed herein. The platform 600 may include any combination of components shown in the example. The components of the platform 600 may be implemented as an integrated circuit (IC), part of an IC, a discrete electronic device, or other modules, logic, hardware, software, firmware, or a combination thereof adapted in the computer platform 600, or implemented as components otherwise incorporated within the chassis of a larger system. Figure 6 The block diagram is intended to show a high-level view of the components of computer platform 600. However, some of the components shown may be omitted, additional components may be present, and different arrangements of the components shown may occur in other implementations.

[0080] The application circuit 605 includes circuits such as, but not limited to, one or more processors (or processor cores), cache memory, and one or more of an LDO, an interrupt controller, a serial interface (such as SPI), I2C or a general programmable serial interface module, an RTC, a timer (including an interval timer and a watchdog timer), a general I / O, a memory card controller (such as a secure digital (SD) MMC or similar controller), a USB interface, a MIPI interface, and a JTAG test access port. The processor (or core) of the application circuit 605 may be coupled to or may include a memory / storage element and may be configured to execute instructions stored in the memory / storage device to enable various applications or operating systems to run on the system 600. In some specific implementations, the memory / storage element may be an on-chip memory circuit that may include any suitable volatile and / or non-volatile memory, such as DRAM, static random access memory (SRAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, solid-state memory, and / or any other type of memory device technology, such as those discussed herein.

[0081] As an example, the processor of the application circuit 605 may include a processor based on Architecture TM Processors such as Quark TM 、Atom TM , i3, i5, i7 or MCU class processors, or available from Santa Clara, CA The processor of application circuit 605 may also be one or more of the following: Advanced Micro Devices (AMD) Processor or Accelerated Processing Unit (APU); from Inc.'s A5-A9 processors, Snapdragon by Technologies, Inc. TM Processor, Texas Instruments, Open Multimedia Applications Platform(OMAP) TMprocessors; MIPS-based designs from MIPS Technologies, Inc., such as the MIPS Warrior M-class, Warrior I-class, and Warrior P-class processors; ARM-based designs licensed from ARM Holdings, Ltd., such as the ARM Cortex-A, Cortex-R, and Cortex-M series processors; etc. In some implementations, the application circuit 605 can be part of a system on a chip (SoC), in which the application circuit 605 and other components are formed as a single integrated circuit or a single package, such as company( Edison Corporation v or Galileo TM SoC board.

[0082] Additionally or alternatively, the application circuit 605 may include circuits such as, but not limited to, one or more field programmable devices (FPDs) such as FPGAs, etc.; programmable logic devices (PLDs) such as complex PLDs (CPLDs), high capacity PLDs (HCPLDs), etc.; ASICs such as structured ASICs, etc.; programmable SoCs (PSoCs); etc. In such embodiments, the circuits of the application circuit 605 may include logic blocks or logic structures, and other interconnected resources that may be programmed to perform various functions such as the processes, methods, functions, etc. of the various embodiments discussed herein. In such embodiments, the circuits of the application circuit 605 may include memory cells such as EPROM, EEPROM, flash memory, static memory (e.g., SRAM, anti-fuse, etc.) for storing logic blocks, logic structures, data, etc. in a lookup table (LUT), etc.

[0083] Baseband circuit 610 may be implemented, for example, as a solder-in substrate including one or more integrated circuits, a single packaged integrated circuit soldered to a main circuit board, or a multi-chip module containing two or more integrated circuits. Figure 7 The various hardware electronic components of baseband circuit 610 are discussed.

[0084] The RFEM 615 may include a millimeter wave (mmWave) RFEM and one or more sub-millimeter wave radio frequency integrated circuits (RFICs). In some implementations, the one or more sub-millimeter wave RFICs may be physically separated from the mmWave RFEM. The RFIC may include connections to one or more antennas or antenna arrays (see, e.g., below). Figure 7Antenna array 711), and the RFEM can be connected to multiple antennas. In an alternative implementation, both millimeter wave and sub-millimeter wave radio functions can be implemented in the same physical RFEM 615 that combines both millimeter wave antennas and sub-millimeter waves.

[0085] The memory circuit 620 may include any number and type of memory devices for providing a fixed amount of system memory. For example, the memory circuit 620 may include one or more of the following: volatile memory, including random access memory (RAM), dynamic RAM (DRAM) and / or synchronous dynamic RAM (SDRAM); and non-volatile memory (NVM), including high-speed electrically erasable memory (commonly referred to as flash memory), phase change random access memory (PRAM), magnetoresistive random access memory (MRAM), etc. The memory circuit 620 may be developed according to the Joint Electron Device Engineering Council (JEDEC) based low power double data rate (LPDDR) design such as LPDDR2, LPDDR3, LPDDR4, etc. The memory circuit 620 may be implemented as one or more of the following: a solder-in package integrated circuit, a single die package, a dual die package (DDP) or a quad die package (Q17P), a socketed memory module, a dual in-line memory module (DIMM) including a micro DIMM or a mini DIMM, and / or soldered to a motherboard via a ball grid array (BGA). In a LP implementation, the memory circuit 620 may be an on-chip memory or register associated with the application circuit 605. To provide persistent storage of information such as data, applications, operating systems, etc., the memory circuit 620 may include one or more mass storage devices, which may include, among others, a solid state disk drive (SSDD), a hard disk drive (HDD), a micro HDD, a resistive change memory, a phase change memory, a holographic memory, or a chemical memory. For example, the computer platform 600 may be combined with a memory device obtained from and Three-dimensional (3D) cross-point (XPOINT) memory.

[0086] Removable memory circuitry 623 may include devices, circuitry, housings / casings, ports or receptacles, etc., for coupling portable data storage devices to platform 600. These portable data storage devices may be used for mass storage purposes, and may include, for example, flash memory cards (e.g., SD cards, micro SD cards, xD picture cards, etc.), as well as USB flash drives, optical disks, external HDDs, etc.

[0087] The platform 600 may further include an interface circuit (not shown) for connecting external devices to the platform 600. External devices connected to the platform 600 via the interface circuit include a sensor circuit 621 and an electromechanical component (EMC) 622, and a removable memory device coupled to a removable memory circuit 623.

[0088] Sensor circuitry 621 comprises a device, module, or subsystem that is intended to detect events or changes in its environment, and to send information about the detected events (sensor data) to some other device, module, subsystem, etc. Examples of such sensors include, among others: an inertial measurement unit (IMU) including an accelerometer, a gyroscope, and / or a magnetometer; a microelectromechanical system (MEMS) or a nanoelectromechanical system (NEMS) including a three-axis accelerometer, a three-axis gyroscope, and / or a magnetometer; a level sensor; a flow sensor; a temperature sensor (e.g., a thermistor); a pressure sensor; a barometric pressure sensor; a gravity meter; an altimeter; an image capture device (e.g., a camera or a lensless aperture); a light detection and ranging (LiDAR) sensor; a proximity sensor (e.g., an infrared radiation detector, etc.), a depth sensor, an ambient light sensor, an ultrasonic transceiver; a microphone or other similar audio capture device; etc.

[0089] EMC 622 includes devices, modules or subsystems that are intended to enable platform 600 to change its state, position and / or orientation or to move or control mechanisms or (sub) systems. In addition, EMC 622 can be configured to generate messages / signaling and send messages / signaling to other components of platform 600 to indicate the current state of EMC 622. Examples of EMC 622 include one or more power switches, relays (including electromechanical relays (EMR) and / or solid-state relays (SSR)), actuators (e.g., valve actuators, etc.), audible sound generators, visual warning devices, motors (e.g., DC motors, stepper motors, etc.), wheels, thrusters, propellers, claws, clamps, hooks and / or other similar EMCs. In an embodiment, platform 600 is configured to operate one or more EMC 622 based on one or more capture events and / or instructions or control signals received from service providers and / or various clients.

[0090] In some specific implementations, the interface circuit may connect the platform 600 to the positioning circuit 645. The positioning circuit 645 includes a circuit for receiving and decoding signals transmitted / broadcasted by the positioning network of the GNSS. Examples of navigation satellite constellations (or GNSS) may include the GPS of the United States, the GLONASS of Russia, the Galileo system of the European Union, the Beidou navigation satellite system of China, regional navigation systems or GNSS augmentation systems (e.g., NAVIC, QZSS of Japan, DORIS of France, etc.), etc. The positioning circuit 645 includes various hardware elements (e.g., including hardware devices such as switches, filters, amplifiers, antenna elements, etc. for facilitating OTA communication) to communicate with components of the positioning network such as navigation satellite constellation nodes. In some embodiments, the positioning circuit 645 may include a micro PNT IC that uses a master timing clock to perform position tracking / estimation without GNSS assistance. The positioning circuit 645 may also be part of the baseband circuit 610 and / or the RFEM 615 or interact with it to communicate with nodes and components of the positioning network. Positioning circuitry 645 may also provide position data and / or time data to application circuitry 605 , which may use the data to synchronize operations with various infrastructure (eg, radio base stations) for use in turn-by-turn navigation applications, and the like.

[0091] In some implementations, the interface circuit may connect the platform 600 with a near field communication (NFC) circuit 640. The NFC circuit 640 is configured to provide contactless short-range communication based on the radio frequency identification (RFID) standard, where a magnetic field is sensed to enable communication between the NFC circuit 640 and an NFC-enabled device (e.g., an "NFC touch point") external to the platform 600. The NFC circuit 640 includes an NFC controller coupled to an antenna element and a processor coupled to the NFC controller. The NFC controller may be a chip / IC that provides NFC functionality to the NFC circuit 640 by executing an NFC controller firmware and an NFC stack. The NFC stack may be executed by the processor to control the NFC controller, and the NFC controller firmware may be executed by the NFC controller to control the antenna element to transmit a short-range RF signal. The RF signal may power a passive NFC tag (e.g., a microchip embedded in a sticker or wristband) to transfer stored data to the NFC circuit 640, or initiate data transfer between the NFC circuit 640 and another active NFC device (e.g., a smartphone or an NFC-enabled POS terminal) proximate to the platform 600.

[0092] The driver circuit 646 may include software and hardware elements for controlling specific devices embedded in, attached to, or otherwise communicatively coupled to the platform 600. The driver circuit 646 may include various drivers to allow other components of the platform 600 to interact with or control various input / output (I / O) devices that may be present in or connected to the platform 600. For example, the driver circuit 646 may include: a display driver for controlling and allowing access to a display device, a touch screen driver for controlling and allowing access to a touch screen interface of the platform 600, a sensor driver for obtaining sensor readings of the sensor circuit 621 and controlling and allowing access to the sensor circuit 621, an EMC driver for obtaining an actuator position of the EMC 622 and / or controlling and allowing access to the EMC 622, a camera driver for controlling and allowing access to an embedded image capture device, and an audio driver for controlling and allowing access to one or more audio devices.

[0093] A power management integrated circuit (PMIC) 625 (also referred to as “power management circuit 625”) may manage power provided to various components of platform 600. Specifically, PMIC 625 may control power selection, voltage scaling, battery charging, or DC-DC conversion relative to baseband circuit 610. PMIC 625 may be typically included when platform 600 is capable of being powered by battery 630, for example, when the device is included in UE 501.

[0094] In some embodiments, the PMIC 625 may control or otherwise be part of various power saving mechanisms of the platform 600. For example, if the platform 600 is in the RRC_Connected state, in which the platform is still connected to the RAN node because it expects to receive traffic soon, then after a period of inactivity, the platform may enter a state known as discontinuous reception mode (DRX). During this state, the platform 600 may be powered off for short time intervals, thereby saving power. If there is no data traffic activity for an extended period of time, the platform 600 may transition to the RRC_Idle state, in which the device is disconnected from the network and no operations such as channel quality feedback, handover, etc. are performed. The platform 600 enters a very low power state and performs paging, in which the device wakes up periodically again to listen to the network and then powers off again. The platform 600 may not receive data in this state; in order to receive data, the platform must transition back to the RRC_Connected state. Additional power saving modes may prevent the device from using the network for longer than the paging interval (ranging from a few seconds to a few hours). During this time, the device is completely unable to connect to the network and can be completely powered off. Any data sent during this time will be significantly delayed, and it is assumed that the delay is acceptable.

[0095] The battery 630 can power the platform 600, but in some examples, the platform 600 can be mounted in a fixed location and can have a power source coupled to a power grid. The battery 630 can be a lithium-ion battery, a metal-air battery such as a zinc-air battery, an aluminum-air battery, a lithium-air battery, etc. In some implementations, such as in V2X applications, the battery 630 can be a typical lead-acid car battery.

[0096] In some implementations, the battery 630 may be a "smart battery" that includes or is coupled to a battery management system (BMS) or a battery monitoring integrated circuit. The BMS may be included in the platform 600 to track the state of charge (SoCh) of the battery 630. The BMS may be used to monitor other parameters of the battery 630, such as the state of health (SoH) and state of function (SoF) of the battery 630 to provide fault prediction. The BMS may transmit information about the battery 630 to the application circuit 605 or other components of the platform 600. The BMS may also include an analog-to-digital (ADC) converter that allows the application circuit 605 to directly monitor the voltage of the battery 630 or the current from the battery 630. The battery parameters may be used to determine actions that the platform 600 may perform, such as transmission frequency, network operation, sensing frequency, etc.

[0097] A power block or other power source coupled to the grid can be coupled to the BMS to charge the battery 630. In some examples, the power block can be replaced with a wireless power receiver to obtain power wirelessly, for example, through a loop antenna in the computer platform 600. In these examples, a wireless battery charging circuit can be included in the BMS. The specific charging circuit selected can depend on the size of the battery 630 and therefore the current required. Charging can be performed using the aviation fuel standard published by the Aviation Fuel Alliance, the Qi wireless charging standard published by the Wireless Power Alliance, or the Rezence charging standard published by the Wireless Power Alliance.

[0098] The user interface circuit 650 includes various input / output (I / O) devices present in or connected to the platform 600, and includes one or more user interfaces designed to implement user interaction with the platform 600 and / or a peripheral component interface designed to implement interaction with the peripheral components of the platform 600. The user interface circuit 650 includes input device circuits and output device circuits. The input device circuit includes any physical or virtual means for accepting input, including, in particular, one or more physical or virtual buttons (e.g., a reset button), a physical keyboard, a keypad, a mouse, a touchpad, a touch screen, a microphone, a scanner, a headset, etc. The output device circuit includes any physical or virtual means for displaying information or otherwise conveying information (such as sensor readings, actuator positions, or other similar information). The output device circuitry may include any number and / or combination of audio or visual displays, including, among other things, one or more simple visual outputs / indicators (e.g., binary status indicators (e.g., light emitting diodes (LEDs)) and multi-character visual outputs, or more complex outputs such as display devices or touch screens (e.g., liquid crystal displays (LCDs), LED displays, quantum dot displays, projectors, etc.), where outputs of characters, graphics, multimedia objects, etc. are generated or produced by the operation of the platform 600. The output device circuitry may also include speakers or other audio emitting devices, printers, etc. In some embodiments, the sensor circuitry 621 may be used as an input device circuitry (e.g., an image capture device, a motion capture device, etc.) and one or more EMCs may be used as output device circuitry (e.g., an actuator for providing tactile feedback, etc.). In another example, an NFC circuit may be included to read an electronic tag and / or connect to another NFC-enabled device, the NFC circuitry including an NFC controller and a processing device coupled to an antenna element. The peripheral component interfaces may include, but are not limited to, non-volatile memory ports, USB ports, audio jacks, power interfaces, etc.

[0099] Although not shown, the components of platform 600 can communicate with each other using a suitable bus or interconnect (IX) technology, which can include any number of technologies, including ISA, EISA, PCI, PCIx, PCIe, time-triggered protocol (TTP) systems, FlexRay systems, or any number of other technologies. The bus / IX can be a proprietary bus / IX, for example, used in a SoC-based system. Other bus / IX systems can be included, such as an I2C interface, an SPI interface, a point-to-point interface, and a power bus, etc.

[0100] Figure 7 7 shows exemplary components of a baseband circuit 710 and an RFEM 715 according to various embodiments. The baseband circuit 710 corresponds to Figure 6 Baseband circuit 610. RFEM 715 corresponds to Figure 6RFEM 615. As shown, RFEM 715 may include RF circuit 706, front end module (FEM) circuit 708, and antenna array 711 coupled together at least as shown.

[0101] The baseband circuit 710 includes circuits and / or control logic components that are configured to execute various radio / network protocols and radio control functions that enable communication with one or more radio networks via the RF circuit 706. The radio control functions may include, but are not limited to, signal modulation / demodulation, encoding / decoding, RF shifting, etc. In some embodiments, the modulation / demodulation circuit of the baseband circuit 710 may include fast Fourier transform (FFT), precoding, or constellation mapping / demapping functions. In some embodiments, the encoding / decoding circuit of the baseband circuit 710 may include convolution, tail-biting convolution, turbo, Viterbi, or low-density parity check (LDPC) encoder / decoder functions. The implementation of the modulation / demodulation and encoder / decoder functions is not limited to these examples and may include other suitable functions in other embodiments. The baseband circuit 710 is configured to process baseband signals received from the receive signal path of the RF circuit 706 and generate baseband signals for the transmit signal path of the RF circuit 706. The baseband circuit 710 is configured to communicate with the application circuit 605 (see Figure 6 ) to generate and process baseband signals and control the operation of RF circuit 706. Baseband circuit 710 may handle various radio control functions.

[0102] The aforementioned circuits and / or control logic components of the baseband circuit 710 may include one or more single-core or multi-core processors. For example, the one or more processors may include a 3G baseband processor 704A, a 4G / LTE baseband processor 704B, a 5G / NR baseband processor 704C, or some other baseband processors 704D for other existing generations, generations under development or generations to be developed in the future (e.g., the sixth generation (6G), etc.). In other embodiments, some or all of the functions of the baseband processors 704A-704D may be included in a module stored in the memory 704G and executed via a central processing unit (CPU) 704E. In other embodiments, some or all of the functions of the baseband processors 704A-704D may be provided as a hardware accelerator (e.g., FPGA, ASIC, etc.) loaded with an appropriate bitstream or logic block stored in a corresponding memory unit. In various embodiments, the memory 704G may store program code of a real-time OS (RTOS), which, when executed by the CPU 704E (or other baseband processor), will enable the CPU 704E (or other baseband processor) to manage resources of the baseband circuit 710, schedule tasks, etc. Examples of RTOS may include: Operating System Embedded (OSE) provided TM , by Mentor Nucleus RTOS provided TM , by Mentor Versatile Real-Time Executive (VRTX) provided by Express Provided by ThreadX TM ,Depend on FreeRTOS and REX OS provided by Open Kernel (OK) OKL4 provided, or any other suitable RTOS, such as those discussed herein. In addition, the baseband circuit 710 includes one or more audio digital signal processors (DSPs) 704F. The audio DSP 704F includes elements for compression / decompression and echo cancellation, and may include other suitable processing elements in other embodiments.

[0103] In some embodiments, each of processors 704A-704E includes a corresponding memory interface to send data to / receive data from memory 704G. Baseband circuit 710 may also include one or more interfaces for communicatively coupling to other circuits / devices, such as an interface for sending / receiving data to / from a memory external to baseband circuit 710; an application circuit interface for sending / receiving data to / from application circuit 605; and an interface for communicating with other circuits / devices. Figure 7 RF circuit 706 to send data / receive data from the RF circuit RF circuit interface; for receiving data from one or more wireless hardware elements (e.g., NFC components, Low power consumption components, components, etc.) to send data / receive data from these wireless hardware elements; and a power management interface for sending power or control signals to / receiving power or control signals from the PMIC 625.

[0104] In an alternative embodiment (which may be combined with the above embodiment), the baseband circuit 710 includes one or more digital baseband systems, which are coupled to each other and to the CPU subsystem, the audio subsystem and the interface subsystem via an interconnect subsystem. The digital baseband subsystem may also be coupled to the digital baseband interface and the mixed signal baseband subsystem via another interconnect subsystem. Each of the interconnect subsystems may include a bus system, a point-to-point connector, a network on chip (NOC) structure and / or some other suitable bus or interconnect technology, such as those discussed herein. The audio subsystem may include a DSP circuit, a buffer memory, a program memory, a voice processing accelerator circuit, a data converter circuit such as an analog-to-digital converter circuit and a digital-to-analog converter circuit, an analog circuit including one or more of an amplifier and a filter, and / or other similar components. In one aspect of the present disclosure, the baseband circuit 710 may include a protocol processing circuit having one or more control circuit instances (not shown) to provide control functions for the digital baseband circuit and / or the RF circuit (e.g., RFEM 715).

[0105] although Figure 7 Not shown, but in some embodiments, the baseband circuit 710 includes various processing devices (e.g., a "multi-protocol baseband processor" or "protocol processing circuit") to operate one or more wireless communication protocols and various processing devices to implement PHY layer functions. In these embodiments, the PHY layer functions include the aforementioned radio control functions. In these embodiments, the protocol processing circuit operates or implements various protocol layers / entities of one or more wireless communication protocols. In a first example, when the baseband circuit 710 and / or the RF circuit 706 are part of a millimeter wave communication circuit or some other suitable cellular communication circuit, the protocol processing circuit may operate LTE protocol entities and / or 5G / NR protocol entities. In the first example, the protocol processing circuit will operate MAC, RLC, PDCP, SDAP, RRC, and NAS functions. In a second example, when the baseband circuit 710 and / or the RF circuit 706 are part of a Wi-Fi communication system, the protocol processing circuit may operate one or more IEEE-based protocols. In the second example, the protocol processing circuit will operate Wi-Fi MAC and logical link control (LLC) functions. The protocol processing circuit may include one or more memory structures (e.g., 704G) for storing program code and data for operating protocol functions, and one or more processing cores for executing program code and performing various operations using data. The baseband circuit 710 may also support radio communications of more than one wireless protocol.

[0106] The various hardware elements of the baseband circuit 710 discussed herein may be implemented as, for example, a solder-in substrate including one or more integrated circuits (ICs), a single packaged integrated circuit soldered to a main circuit board, or a multi-chip module containing two or more ICs. In one example, the components of the baseband circuit 710 may be appropriately combined in a single chip or a single chipset, or disposed on the same circuit board. In another example, some or all of the components of the baseband circuit 710 and the RF circuit 706 may be implemented together, such as, for example, a SoC or a system-in-package (SiP). In another example, some or all of the components of the baseband circuit 710 may be implemented as a separate SoC communicatively coupled to the RF circuit 706 (or multiple instances of the RF circuit 706). In yet another example, some or all of the components of the baseband circuit 710 and the application circuit 605 may be implemented together as a separate SoC (e.g., a "multi-chip package") mounted to the same circuit board.

[0107] In some embodiments, the baseband circuit 710 can provide communications compatible with one or more radio technologies. For example, in some embodiments, the baseband circuit 710 can support communications with E-UTRAN or other WMANs, WLANs, WPANs. Embodiments in which the baseband circuit 710 is configured to support radio communications of more than one wireless protocol can be referred to as multi-mode baseband circuits.

[0108] The RF circuit 706 may enable communication with a wireless network through a non-solid medium using modulated electromagnetic radiation. In various embodiments, the RF circuit 706 may include switches, filters, amplifiers, etc. to facilitate communication with the wireless network. The RF circuit 706 may include a receive signal path, which may include circuits for down-converting RF signals received from the FEM circuit 708 and providing baseband signals to the baseband circuit 710. The RF circuit 706 may also include a transmit signal path, which may include circuits for up-converting baseband signals provided by the baseband circuit 710 and providing an RF output signal for transmission to the FEM circuit 708.

[0109] In some embodiments, the receive signal path of the RF circuit 706 may include a mixer circuit 706a, an amplifier circuit 706b, and a filter circuit 706c. In some embodiments, the transmit signal path of the RF circuit 706 may include a filter circuit 706c and a mixer circuit 706a. The RF circuit 706 may also include a synthesizer circuit 706d, which is used to synthesize the frequency used by the mixer circuit 706a of the receive signal path and the transmit signal path. In some embodiments, the mixer circuit 706a of the receive signal path may be configured to down-convert the RF signal received from the FEM circuit 708 based on the synthesized frequency provided by the synthesizer circuit 706d. The amplifier circuit 706b may be configured to amplify the down-converted signal, and the filter circuit 706c may be a low pass filter (LPF) or a band pass filter (BPF), which is configured to remove unwanted signals from the down-converted signal to generate an output baseband signal. The output baseband signal may be provided to the baseband circuit 710 for further processing. In some embodiments, the output baseband signal may be a zero frequency baseband signal, although this is not required.In some embodiments, the mixer circuit 706a of the receive signal path may include a passive mixer, although the scope of the embodiments is not limited in this respect.

[0110] In some embodiments, mixer circuit 706a of the transmit signal path may be configured to up-convert an input baseband signal based on a synthesized frequency provided by synthesizer circuit 706d to generate an RF output signal for FEM circuit 708. The baseband signal may be provided by baseband circuit 710 and may be filtered by filter circuit 706c.

[0111] In some embodiments, the mixer circuit 706a of the receive signal path and the mixer circuit 706a of the transmit signal path may include two or more mixers and may be arranged for quadrature down-conversion and quadrature up-conversion, respectively. In some embodiments, the mixer circuit 706a of the receive signal path and the mixer circuit 706a of the transmit signal path may include two or more mixers and may be arranged for image suppression (e.g., Hartley image suppression). In some embodiments, the mixer circuit 706a of the receive signal path and the mixer circuit 706a of the transmit signal path may be arranged for direct down-conversion and direct up-conversion, respectively. In some embodiments, the mixer circuit 706a of the receive signal path and the mixer circuit 706a of the transmit signal path may be configured for superheterodyne operation.

[0112] In some embodiments, the output baseband signal and the input baseband signal may be analog baseband signals, although the scope of the embodiments is not limited in this respect. In some alternative embodiments, the output baseband signal and the input baseband signal may be digital baseband signals. In these alternative embodiments, the RF circuit 706 may include an analog-to-digital converter (ADC) and a digital-to-analog converter (DAC) circuit, and the baseband circuit 710 may include a digital baseband interface to communicate with the RF circuit 706.

[0113] In some dual-mode embodiments, separate radio IC circuits may be provided to process signals for each spectrum, although the scope of the embodiments is not limited in this respect.

[0114] In some embodiments, synthesizer circuit 706d may be a fractional-N synthesizer or a fractional-N / N+1 synthesizer, but the scope of the embodiments is not limited in this respect, as other types of frequency synthesizers may also be suitable. For example, synthesizer circuit 706d may be a delta-sigma synthesizer, a frequency multiplier, or a synthesizer including a phase-locked loop with a frequency divider.

[0115] Synthesizer circuit 706d may be configured to synthesize an output frequency based on the frequency input and the divider control input for use by mixer circuit 706a of RF circuit 706. In some embodiments, synthesizer circuit 706d may be a fractional-N / N+1 synthesizer.

[0116] In some embodiments, the frequency input may be provided by a voltage controlled oscillator (VCO), although this is not required. The divider control input may be provided by baseband circuitry 710 or application circuitry 605 according to the desired output frequency. In some embodiments, the divider control input (e.g., N) may be determined from a lookup table based on the channel indicated by application circuitry 605.

[0117] The synthesizer circuit 706d of the RF circuit 706 may include a frequency divider, a delay locked loop (DLL), a multiplexer, and a phase accumulator. In some embodiments, the frequency divider may be a dual-mode frequency divider (DMD), and the phase accumulator may be a digital phase accumulator (DPA). In some embodiments, the DMD may be configured to divide the input signal by N or N+1 (e.g., based on a carry) to provide a fractional division ratio. In some example embodiments, the DLL may include a cascaded, tunable, delay element, a phase detector, a charge pump, and a D-type flip-flop set. In these embodiments, the delay element may be configured to divide the VCO cycle into Nd equal phase groups, where Nd is the number of delay elements in the delay line. In this way, the DLL provides negative feedback to help ensure that the total delay through the delay line is one VCO cycle.

[0118] In some embodiments, the synthesizer circuit 706d can be configured to generate a carrier frequency as an output frequency, while in other embodiments, the output frequency can be a multiple of the carrier frequency (e.g., twice the carrier frequency, four times the carrier frequency) and can be used with a quadrature generator and divider circuit to generate multiple signals with multiple different phases relative to each other at the carrier frequency. In some embodiments, the output frequency can be the LO frequency (fLO). In some embodiments, the RF circuit 706 can include an IQ / polarity converter.

[0119] The FEM circuitry 708 may include a receive signal path that may include circuitry configured to operate on RF signals received from the antenna array 711, amplify the received signals, and provide an amplified version of the received signals to the RF circuitry 706 for further processing. The FEM circuitry 708 may also include a transmit signal path that may include circuitry configured to amplify signals provided for transmission by the RF circuitry 706 for transmission by one or more antenna elements in the antenna array 711. In various embodiments, amplification by the transmit or receive signal paths may be accomplished only in the RF circuitry 706, only in the FEM circuitry 708, or in both the RF circuitry 706 and the FEM circuitry 708.

[0120] In some embodiments, the FEM circuit 708 may include a TX / RX switch to switch between transmit mode and receive mode operation. The FEM circuit 708 may include a receive signal path and a transmit signal path. The receive signal path of the FEM circuit 708 may include an LNA to amplify the received RF signal and provide the amplified received RF signal as an output (e.g., to the RF circuit 706). The transmit signal path of the FEM circuit 708 may include a power amplifier (PA) for amplifying the input RF signal (e.g., provided by the RF circuit 706), and one or more filters for generating an RF signal for subsequent transmission by one or more antenna elements of the antenna array 711.

[0121] The antenna array 711 includes one or more antenna elements, each of which is configured to convert electrical signals into radio waves to travel through the air and to convert received radio waves into electrical signals. For example, the digital baseband signal provided by the baseband circuit 710 is converted into an analog RF signal (e.g., a modulated waveform), which is amplified and transmitted via the antenna elements of the antenna array 711 including one or more antenna elements (not shown). The antenna elements may be omnidirectional, directional, or a combination thereof. The antenna elements may be formed into a variety of arrangements as known and / or discussed herein. The antenna array 711 may include a microstrip antenna or a printed antenna fabricated on the surface of one or more printed circuit boards. The antenna array 711 may be formed as a patch of metal foil of various shapes (e.g., a patch antenna) and may be coupled to the RF circuit 706 and / or the FEM circuit 708 using a metal transmission line or the like.

[0122] The processor of the application circuit 605 and the processor of the baseband circuit 710 can be used to execute elements of one or more instances of the protocol stack. For example, the processor of the baseband circuit 710 can be used alone or in combination to perform layer 3, layer 2, or layer 1 functions, while the processor of the application circuit 605 can utilize data received from these layers (e.g., packet data) and further perform layer 4 functions (e.g., TCP and UDP layers). As mentioned herein, layer 3 may include an RRC layer. As mentioned herein, layer 2 may include a MAC layer, an RLC layer, and a PDCP layer. As mentioned herein, layer 1 may include a PHY layer of a UE / RAN node.

[0123] Example

[0124] Embodiment 1 may include a new SDP attribute to indicate the ability to transport 360 video as part of an RTP stream, including one or more of the following: the ability to carry immersive media metadata information as part of the RTP payload format, for example, using Supplementary Information Enhancement (SEI) messages, such as then using the information carried in the omnidirectional video specific SEI message for the rendering stage at the receiver.

[0125] Embodiment 2 may include a new SDP attribute to indicate viewport-dependent processing capabilities for transporting 360 video as part of an RTP stream, including one or more of: the ability to signal the expected viewport using RTCP feedback messages; and / or the ability to signal the actual transmitted viewport using RTP header extension messages.

[0126] Embodiment 3 may include SDP attributes according to embodiment 2 or some other embodiment herein, wherein two or more RTP streams may be negotiated accordingly.

[0127] Embodiment 4 may comprise a single SDP attribute that combines the capabilities of Embodiment 1 and Embodiment 2, or some other embodiment herein.

[0128] Embodiment 5 may include the SDP attributes described in accordance with Embodiment 3 or some other embodiment herein, wherein one RTP stream is for basic 360 video and the other is a viewport optimized RTP stream having high quality 360 video corresponding to a desired viewport.

[0129] Embodiment 6 may include the SDP attributes according to embodiment 1 or some other embodiment herein, wherein the omnidirectional video specific SEI message may include one or more of the following messages: an equirectangular projection SEI message, a cubic projection SEI message, a spherical rotation SEI message, and a partition encapsulation SEI message.

[0130] Embodiment 7 may include a method comprising: receiving an SDP proposal message, the SDP proposal message being used to indicate the capability of supporting 360 video in an RTP stream; and encoding an SDP response message for transmission based on the SDP proposal message, the SDP response message being used to indicate confirmation of these capabilities.

[0131] Embodiment 8 may include a method according to embodiment 7 or another embodiment herein, wherein the SDP offer message further includes viewport dependent processing capabilities.

[0132] Embodiment 9 may include a method according to embodiment 8 or another embodiment herein, wherein when the SDP answer message includes viewport-dependent processing capabilities, the SDP offer message includes two or more RTP streams.

[0133] Embodiment 10 may include a method according to embodiments 8 to 9 or another embodiment herein, wherein the capability for supporting 360 video and the viewport-dependent processing capability are indicated in the same SDP attribute in the SDP offer message.

[0134] Embodiment 11 may include a method according to embodiments 8 to 10 or another embodiment herein, wherein the SDP answer includes accepting viewport dependent processing capabilities.

[0135] Embodiment 12 may include a method according to embodiments 8 to 11 or another embodiment herein, further comprising encoding immersive media including 360 video for transmission based on an SDP offer message.

[0136] Embodiment 13 may include a method according to embodiment 8 to 12 or another embodiment herein, wherein the SDP offer message is received from the UE.

[0137] Embodiment 14 may include the method according to embodiment 8 to 13 or another embodiment herein, wherein the method is performed by a conference server or a portion thereof.

[0138] Embodiment 15 may include a method comprising: encoding an SDP proposal message for transmission, the SDP proposal message being used to indicate the capability of supporting 360 video in an RTP stream; and receiving an SDP response message based on the SDP proposal message, the SDP response message being used to indicate confirmation of these capabilities.

[0139] Embodiment 16 may include the method described in embodiment 15 or another embodiment herein, wherein the SDP proposal message further includes viewport-dependent processing capabilities.

[0140] Embodiment 17 may include the method of embodiment 16 or another embodiment herein, wherein when the SDP answer message includes viewport-dependent processing capabilities, the SDP offer message includes two or more RTP streams.

[0141] Embodiment 18 may include the method described in Embodiments 16 to 17 or another embodiment herein, wherein the capability for supporting 360 video and the viewport-dependent processing capability are indicated in the same SDP attribute in the SDP offer message.

[0142] Embodiment 19 may include the method of embodiments 16 to 18 or another embodiment herein, wherein the SDP answer includes accepting viewport-dependent processing capabilities.

[0143] Embodiment 20 may include the method of Embodiments 16 to 19 or another embodiment herein, further comprising receiving immersive media including 360 video based on the SDP offer message.

[0144] Embodiment 21 may include a method according to Embodiment 16 to 20 or another embodiment herein, wherein the SDP answer message is received from the conference server.

[0145] Embodiment 22 may include a method according to Embodiment 16 to 21 or another embodiment herein, wherein the method is performed by a UE or a portion thereof.

[0146] Embodiment 23 includes an electronic device comprising: a first circuit for decoding first visualization data associated with multiple images of a location captured simultaneously based on a first real-time transport protocol (RTP) stream; a second circuit for decoding an indication that a user equipment (UE) supports immersive viewing capabilities based on a session description protocol (SDP) proposal received from the UE; and a third circuit for transmitting second visualization data associated with the first visualization data via a second RTP stream based on the SDP proposal, wherein the second visualization data includes a supplemental information enhancement (SEI) message for displaying at least a portion of the second visualization data.

[0147] Embodiment 24 comprises an electronic device according to Embodiment 23, wherein the first RTP stream includes visualization data related to two or more images of the plurality of images.

[0148] Embodiment 25 includes an electronic device according to embodiment 23, wherein the first RTP stream includes visual data related to an image in the multiple images, and wherein the electronic device also includes: a fourth circuit, which is used to decode third visual data related to another image in the multiple images based on a third RTP stream; and a fifth circuit, which is used to splice the first visual data and the third visual data together.

[0149] Embodiment 26 includes an electronic device according to any one of Embodiments 23 to 25, wherein the SEI message is an equirectangular projection SEI message, a cubic projection SEI message, a sphere rotation SEI message, or a partition encapsulation SEI message.

[0150] Embodiment 27 includes an electronic device according to any one of embodiments 23 to 25, which also includes a sixth circuit for decoding an indication of a desired viewing direction of the first visualization data based on an RTP control protocol (RTCP) feedback message received from the UE.

[0151] Embodiment 28 includes the electronic device of Embodiment 27, wherein the second visualization data is a portion of the first visualization data based on the indication of the desired viewing direction.

[0152] Embodiment 29 includes the electronic device of any one of Embodiments 23 to 25, wherein the first, second, and third circuits are circuits of a processor.

[0153] Embodiment 30 includes an electronic device comprising: a first circuit for determining a basic stream based on a received real-time transport protocol (RTP) stream, the real-time transport protocol stream including visual data associated with multiple images of a location captured simultaneously; a second circuit for decoding the visual data and a supplementary information enhancement (SEI) message based on the basic stream; a third circuit for generating a mapping of the visual data to a field of view based on the visual data and the SEI message; and a fourth circuit for outputting data associated with the mapping of the visual data to the field of view to a display device.

[0154] Embodiment 31 includes the electronic device according to embodiment 30, wherein the electronic device is a user equipment (UE) of a 3rd Generation Partnership Project (3GPP) network, and wherein the UE includes a display device.

[0155] Embodiment 32 includes an electronic device according to embodiment 30, wherein the SEI message is an equirectangular projection SEI message, a cubic projection SEI message, a sphere rotation SEI message, or a partition packing SEI message.

[0156] Embodiment 33 includes an electronic device according to any one of Embodiments 30 to 32, further comprising a fifth circuit for facilitating transmission of an indication of a desired viewing direction of the first visualization data in an RTP control protocol (RTCP) feedback message.

[0157] Embodiment 34 includes an electronic device according to any one of Embodiments 30 to 32, wherein the RTP stream is a first RTP stream including first visual data related to a first image among the multiple images, and wherein the basic stream is also based on a decoded second RTP stream, and the decoded second RTP stream includes second visual data related to a second image among the multiple images.

[0158] Embodiment 35 includes an electronic device according to any one of Embodiments 30 to 32, wherein the RTP stream includes visualization data related to at least a first image and a second image of the plurality of images.

[0159] Embodiment 36 includes an electronic device according to any one of Embodiments 30 to 32, wherein the visual data is a high efficiency video coding (HEVC) bitstream including a SEI message.

[0160] Embodiment 37 includes a method, comprising: transmitting a session description protocol (SDP) proposal from a user equipment (UE) to an (IP) multimedia subsystem (IMS) server, the session description protocol proposal including an indication that the UE supports immersive video; determining, by the UE based on an SDP response received from the server, an indication that the server supports immersive video; determining, based on the SDP proposal and the SDP response, a real-time transport protocol (RTP) media stream and a supplementary information enhancement (SEI) message, the real-time transport protocol media stream including visual data associated with multiple images of a location taken simultaneously, and the supplementary information enhancement message being used to display at least a portion of the visual data; and the UE visually displaying the portion of the visual data to a user of the user equipment based on the visual data and the SEI message.

[0161] Embodiment 38 includes the method of embodiment 37, wherein the SDP offer includes an indication that the UE supports viewport dependent processing.

[0162] Embodiment 39 includes a method according to embodiment 37, wherein the SDP response includes an indication that the server supports viewport-dependent processing.

[0163] Embodiment 40 includes a method according to any one of Embodiments 37 to 39, further comprising transmitting an indication of a desired field of view of the visualization data from the UE to the server in an RTP Control Protocol (RTCP) feedback message.

[0164] Embodiment 41 comprises the method according to embodiment 40, wherein the visualization data of the RTP media stream is based on the indication of the desired field of view.

[0165] Embodiment 42 includes the method of any one of Embodiments 37 to 39, wherein the visualization data is based on an Omnidirectional Media Format (OMAF) video profile.

[0166] Embodiment 43 may include an apparatus comprising means for performing one or more elements of the method described in or related to any of Embodiments 1 to 42, or any other method or process described herein.

[0167] Embodiment 44 may include one or more non-transitory computer-readable media, which include instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform one or more elements of a method described or related to any of Embodiments 1 to 42 or any other method or process described herein.

[0168] Embodiment 45 may include an apparatus comprising logic components, modules, or circuits for performing one or more elements of a method as described or related to any of Embodiments 1 to 42, or any other method or process described herein.

[0169] Embodiment 46 may include methods, techniques or processes described or related to any one of Embodiments 1 to 42, or portions or components thereof.

[0170] Embodiment 47 may include a device comprising: one or more processors and one or more computer-readable media, wherein the one or more computer-readable media include instructions that, when executed by the one or more processors, cause the one or more processors to perform a method, technique, or process, or a portion thereof, described in or related to any one of Embodiments 1 to 42.

[0171] Embodiment 48 may include a signal as described or related to any one of Embodiments 1 to 42, or a portion or component thereof.

[0172] Embodiment 49 may include a datagram, packet, frame, segment, protocol data unit (PDU) or message, or a portion or component thereof, as described or associated with any of Embodiments 1 to 42, or otherwise described in the present disclosure.

[0173] Embodiment 50 may include a signal encoded with data as described or associated with any one of Embodiments 1 to 42, or a portion or component thereof, or otherwise described in this disclosure.

[0174] Embodiment 51 may include a signal encoded with a datagram, packet, frame, segment, PDU or message as described or associated with any of Embodiments 1 to 42 above, or a portion or component thereof, or otherwise described in the present disclosure.

[0175] Embodiment 52 may include an electromagnetic signal carrying computer-readable instructions, wherein execution of the computer-readable instructions by one or more processors will cause the one or more processors to perform a method, technique, or process, or a portion thereof, described in or related to any one of Embodiments 1 to 42.

[0176] Embodiment 53 may include a computer program comprising instructions, wherein execution of the program by a processing element causes the processing element to perform a method, technique, or process, or a portion thereof, as described or related to any one of Embodiments 1 to 42.

[0177] Embodiment 54 may include signals in a wireless network as shown and described herein.

[0178] Embodiment 55 may include a method of communicating in a wireless network as shown and described herein.

[0179] Embodiment 56 may include a system for providing wireless communications as shown and described herein.

[0180] Embodiment 57 may include an apparatus for providing wireless communications as shown and described herein.

[0181] Unless explicitly stated otherwise, any of the above examples may be combined with any other example (or combination of examples). The foregoing description of one or more specific implementations provides illustration and description, but is not intended to be exhaustive or to limit the scope of the embodiments to the precise form disclosed. Modifications and variations are possible in view of the above teachings or may be obtained from the practice of the various embodiments.

Claims

1. A method for wireless communication, comprising: Obtaining visualization data and supplemental enhancement information (SEI) messages based on a received real-time transport protocol (RTP) stream, the RTP stream including the visualization data, the visualization data being associated with a plurality of images of a video conference between two or more devices; generating a mapping of the visualization data to a field of view based on the visualization data and the SEI message; as well as Data related to the mapping of the visualization data to the field of view is output to a display device. 2 . The method of claim 1 , wherein obtaining the visualization data and the SEI message comprises obtaining the visualization data and the SEI message by one or more baseband processors. 3 . The method of claim 1 , wherein the SEI message is an equirectangular projection SEI message, a cubic projection SEI message, a sphere rotation SEI message, or a partition packing SEI message. 4 . The method of claim 1 , further comprising transmitting an indication of a desired viewing direction of the visualization data in an RTP control protocol (RTCP) feedback message. The method of claim 4 , wherein the visualization data is based on the indication of the desired viewing direction.

6. The method according to claim 1, wherein: The RTP stream is a first RTP stream including first visualization data associated with a first image of the plurality of images; and Obtaining the visualization data and the SEI message includes obtaining the visualization data and the SEI message based on the first RTP stream and a second RTP stream including second visualization data associated with a second image of the plurality of images. 7 . The method of claim 1 , wherein the RTP stream includes visualization data associated with at least a first image and a second image of the plurality of images. 8 . The method of claim 1 , wherein the visualization data is a High Efficiency Video Coding (HEVC) bitstream including the SEI message.

9. The method of claim 1, wherein the plurality of images are taken at a location substantially simultaneously.

10. The method of claim 1, wherein the visualization data is based on an Omnidirectional Media Format (OMAF) video profile.

11. An apparatus for wireless communication, the apparatus comprising one or more processors, the one or more processors configured to perform operations comprising: Obtaining visualization data and supplemental enhancement information (SEI) messages based on a received real-time transport protocol (RTP) stream, the RTP stream including the visualization data, the visualization data being associated with a plurality of images of a video conference between two or more devices; At least some of the visualization data mapped to the field of view is output based at least on the SEI message. 12 . The apparatus of claim 11 , the operations further comprising mapping the visualization data to the field of view based at least on the visualization data and the SEI message.

13. The apparatus of claim 11, wherein the one or more processors comprise one or more baseband processors.

14. The apparatus of claim 11, wherein the operations comprise: One or more two-dimensional images captured by a camera connected to the device are sent to a system generating the RTP stream, wherein the one or more two-dimensional images are included in the plurality of images of the video conference.

15. The apparatus of claim 11, wherein at least some of the plurality of images are captured by an omnidirectional camera used for the video conference.

16. The apparatus of claim 11, wherein the operations comprise: A 360 degree video is generated based at least on the texture-to-sphere mapping function and the visualization data, wherein outputting at least some of the visualization data comprises outputting at least some of the 360 ​​degree video mapped to the field of view.

17. An apparatus according to claim 11, wherein at least some of the visualization data mapped to the field of view is output based on at least one of: data of a sensor position, display characteristics of a display device that is to output at least some of the visualization data, or initial viewport data.

18. The apparatus of claim 11, wherein the operations comprise: An indication of a desired field of view for the visualization data is transmitted in an RTP control protocol RTCP feedback message, wherein obtaining the visualization data comprises obtaining the visualization data based at least in part on the desired field of view.

19. The device according to claim 11, wherein: The RTP stream is a first RTP stream including first visualization data associated with a first image of the plurality of images; and Obtaining the visualization data and the SEI message includes obtaining the visualization data and the SEI message based on: a) the first RTP stream, and b) a second RTP stream including second visualization data associated with a second image of the plurality of images.

20. An apparatus for wireless communication, comprising one or more processors configured to perform operations comprising: Obtaining visualization data and supplemental enhancement information (SEI) messages based on a received real-time transport protocol (RTP) stream, wherein the RTP stream includes the visualization data, and the visualization data is associated with a plurality of images of a video conference; generating output data based on the visualization data and the SEI message by mapping the visualization data to a field of view; as well as The output data is output.

21. A computer program product comprising a computer program which, when executed by a processor, causes the processor to perform the method according to any one of claims 1 to 10.

22. A computer-readable storage medium storing instructions, which, when executed by one or more processors, cause the one or more processors to perform the method according to any one of claims 1 to 10.

Images