Method, architecture, apparatus and system for data driven prediction of augmented reality (XR) device user inputs

Abstract

Programs, methods, architectures, apparatuses, systems, devices, and computer program products implemented by a wireless transmit / receive unit (WTRU) include obtaining first user input data at a first time; transmitting first information including neural network data generated from a first distributed deep neural network (DNN) of the WTRU; receiving second information including first predicted user input data generated from a second DNN of the edge device at a second time; obtaining second user input data at a second time; determining a first prediction error based on the first predicted user input data at the second time and second user input data obtained at the second time; and sending third information comprising the first prediction error and / or an indication to trigger an error correction algorithm for predicting the user input under the condition that the first prediction error is above the threshold.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of European Patent Application No. 22191237.1 filed on August 19, 2022, which is incorporated herein by reference. Background Art

[0003] The present disclosure relates generally to the fields of communications, software, and coding, including, for example, methods, architectures, apparatuses, and systems related to data-driven prediction of user input for XR devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] A more detailed understanding can be obtained by referring to the detailed description given below in conjunction with the accompanying drawings. Like the detailed description, the figures in the drawings are examples. Likewise, the drawings (figures) and detailed description should not be considered limiting, and other equally effective examples are possible and may occur. In addition, the same reference numerals ("reference numerals") in the figures represent the same elements, wherein:

[0005] Figure 1A is a system diagram illustrating an example communication system;

[0006] Figure 1B It is a diagram that can be Figure 1A A system diagram of an example wireless transmit / receive unit (WTRU) for use within the illustrated communication system;

[0007] Figure 1C It is a diagram that can be Figure 1A a system diagram of an exemplary radio access network (RAN) and an exemplary core network (CN) used within the illustrated communication system;

[0008] Figure 1D It is a diagram that can be Figure 1A a system diagram of another example RAN and another example CN used within the illustrated communication system;

[0009] Figure 2 The diagram shows the relationship between network latency and response time;

[0010] Figure 3 Figure 1 illustrates the relationship between user action prediction and low response time.

[0011] Figure 4 An example architecture running on a WTRU is illustrated;

[0012] Figure 5 Illustration of an example architecture running on an edge device.

[0013] Figure 6is a system diagram illustrating an example of 5G-XR functionality integrated in a 5G system;

[0014] Figure 7 is a system diagram illustrating an example of expansion of functionality of a 5G system;

[0015] Figure 8 yes Figure 7 A more detailed view of the system diagram;

[0016] Figure 9 is a message sequence chart for first exploring the space of alternative solutions and then triggering online retraining of a model for a deep neural network (DNN);

[0017] Figure 10 is a flow chart illustrating a method implemented at a WTRU according to one or more embodiments;

[0018] Figure 11 is a further flow chart illustrating a method implemented at an edge device according to one or more embodiments; and

[0019] Figure 12 is a flow chart of another method implemented at a WTRU according to one or more embodiments. DETAILED DESCRIPTION

[0020] In the following detailed description, many specific details are set forth to provide a comprehensive understanding of the embodiments and / or examples disclosed herein. However, it will be understood that such embodiments and examples can be implemented without some or all of the specific details set forth herein. In other cases, well-known methods, processes, components, and circuits have not yet been described in detail to avoid blurring the following description. In addition, the embodiments and examples not specifically described herein may be implemented in place of or in combination with the embodiments and other examples explicitly, implicitly, and / or inherently described, disclosed, or otherwise provided (collectively, “provided”) herein. Although various embodiments are described and / or claimed herein, in which devices, systems, equipment, etc. and / or any elements thereof perform operations, processes, algorithms, functions, etc. and / or any part thereof, it should be understood that any embodiment described and / or claimed herein assumes that any device, system, equipment, etc. and / or any element thereof is configured to perform any operation, process, algorithm, function, etc. and / or any part thereof.

[0021] Example Communication System

[0022] The methods, apparatus, and systems provided herein are well-suited for communications involving both wired and wireless networks. Figures 1A-1DAn overview is provided of various types of wireless devices and infrastructure in which various elements of a network may utilize, perform, be arranged according to and / or adapted for the methods, apparatus, and systems provided herein and / or be configured for the methods, apparatus, and systems provided herein.

[0023] Figure 1A 1 is a system diagram illustrating an example communication system 100 in which one or more disclosed embodiments may be implemented. The communication system 100 may be a multiple access system that provides content, such as voice, data, video, messaging, broadcast, etc., to multiple wireless users. The communication system 100 may enable multiple wireless users to access such content by sharing system resources, including wireless bandwidth. For example, the communication system 100 may employ one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single carrier FDMA (SC-FDMA), zero tail (ZT) unique word (UW) discrete Fourier transform (DFT) spread OFDM (ZT UW DTS-s OFDM), unique word OFDM (UW-OFDM), resource block filtered OFDM, filter bank multi-carrier (FBMC), etc.

[0024] like Figure 1AAs shown, the communication system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, a radio access network (RAN) 104 / 113, a core network (CN) 106 / 115, a public switched telephone network (PSTN) 108, the Internet 110, and other networks 112. However, it will be appreciated that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Each of the WTRUs 102a, 102b, 102c, 102d may be any type of device configured to operate and / or communicate in a wireless environment. By way of example, the WTRUs 102a, 102b, 102c, 102d (any of which may be referred to as a “station” and / or “STA”) may be configured to transmit and / or receive wireless signals and may include (or be) a user equipment (UE), a mobile station, a fixed or mobile subscriber unit, a subscription-based unit, a pager, a cellular phone, a personal digital assistant (PDA), a smartphone, a laptop, a netbook, a personal computer, a wireless sensor, a hotspot or Mi-Fi device, an Internet of Things (IoT) device, a watch or other wearable device, a head-mounted display (HMD), a vehicle, a drone, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in industrial and / or automated process chain environments), consumer electronic devices, devices operating on commercial and / or industrial wireless networks, etc. Any of the WTRUs 102a, 102b, 102c, and 102d may be interchangeably referred to as a UE.

[0025] The communication system 100 may also include a base station 114a and / or a base station 114b. Each of the base stations 114a, 114b may be any type of device configured to wirelessly interface with at least one of the WTRUs 102a, 102b, 102c, 102d, e.g., to facilitate access to one or more communication networks, such as the CN 106 / 115, the Internet 110, and / or the network 112. By way of example, the base stations 114a, 114b may be any one of a base transceiver station (BTS), a Node B (NB), an eNode-B (eNB), a Home Node B (HNB), a Home eNode-B (HeNB), a gNode-B (gNB), a NR Node B (NRNB), a site controller, an access point (AP), a wireless router, and the like. While the base stations 114a, 114b are depicted as a single element, it will be appreciated that the base stations 114a, 114b may include any number of interconnected base stations and / or network elements.

[0026] Base station 114a may be part of RAN 104 / 113, which may also include other base stations and / or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), relay nodes, etc. Base station 114a and / or base station 114b may be configured to transmit and / or receive wireless signals on one or more carrier frequencies, which may be referred to as cells (not shown). These frequencies may be in licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide wireless service coverage to a specific geographic area, which may be relatively fixed or may change over time. A cell may also be divided into cell sectors. For example, the cell associated with base station 114a may be divided into three sectors. Thus, in one embodiment, base station 114a may include three transceivers, one for each sector of the cell. In one embodiment, base station 114a may employ multiple-input multiple-output (MIMO) technology and may utilize multiple transceivers for each or any sector of the cell. For example, beamforming may be used to transmit and / or receive signals in a desired spatial direction.

[0027] The base stations 114a, 114b may communicate with one or more of the WTRUs 102a, 102b, 102c, 102d over an air interface 116, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 116 may be established using any suitable radio access technology (RAT).

[0028] More specifically, as described above, the communication system 100 may be a multiple-access system and may employ one or more channel access schemes such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, and the like. For example, the base station 114a in the RAN 104 / 113 and the WTRUs 102a, 102b, 102c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may use Wideband CDMA (WCDMA) to establish the air interface 116. WCDMA may include communication protocols such as High Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA may include High Speed ​​Downlink Packet Access (HSDPA) and / or High Speed ​​Uplink Packet Access (HSUPA).

[0029] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface 116 using Long Term Evolution (LTE) and / or LTE-Advanced (LTE-A) and / or LTE-Advanced Pro (LTE-A Pro).

[0030] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as NR wireless access, which may establish the air interface 116 using New Radio (NR).

[0031] In an embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement multiple radio access technologies. For example, the base station 114a and the WTRUs 102a, 102b, 102c may jointly implement LTE radio access and NR radio access, e.g., using dual connectivity (DC) principles. Thus, the air interface utilized by the WTRUs 102a, 102b, 102c may be characterized by multiple types of radio access technologies and / or transmissions to / from multiple types of base stations (e.g., eNBs and gNBs).

[0032] In one embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as IEEE 802.11 (i.e., Wireless Fidelity (Wi-Fi)), IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), GSM EDGE (GERAN), etc.

[0033] For example, Figure 1AThe base station 114b in the embodiment may be a wireless router, a Home Node-B, a Home eNode-B, or an access point, and may utilize any suitable RAT to facilitate wireless connectivity in a local area, such as a business, a home, a vehicle, a campus, an industrial facility, an air corridor (e.g., for use by drones), a road, and the like. In one embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.11 to establish a wireless local area network (WLAN). In one embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.15 to establish a wireless personal area network (WPAN). In one embodiment, the base station 114b and the WTRUs 102c, 102d may utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.) to establish any of a small cell, a pico cell, or a femto cell. As Figure 1A As shown, base station 114b may have a direct connection to the Internet 110. Thus, base station 114b may not be required to access the Internet 110 via CN 106 / 115.

[0034] The RAN 104 / 113 may be in communication with the CN 106 / 115, which may be any type of network configured to provide voice, data, applications, and / or Voice over Internet Protocol (VoIP) services to one or more of the WTRUs 102a, 102b, 102c, 102d. The data may have different quality of service (QoS) requirements, such as different throughput requirements, latency requirements, error tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, etc. The CN 106 / 115 may provide call control, billing services, mobile location-based services, prepaid calling, Internet connectivity, video distribution, etc., and / or perform advanced security functions, such as user authentication. Although in Figure 1A Although not shown, it will be appreciated that the RAN 104 / 113 and / or the CN 106 / 115 may be in direct or indirect communication with other RANs that employ the same RAT as the RAN 104 / 113 or a different RAT. For example, in addition to being connected to the RAN 104 / 113, which may utilize NR radio technology, the CN 106 / 115 may also be in communication with another RAN (not shown) that employs any of GSM, UMTS, CDMA 2000, WiMAX, E-UTRA, or Wi-Fi radio technologies.

[0035] The CN 106 / 115 may also serve as a gateway for the WTRUs 102a, 102b, 102c, 102d to access the PSTN 108, the Internet 110, and / or other networks 112. The PSTN 108 may include a circuit-switched telephone network that provides plain old telephone service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices that use common communication protocols, such as the Transmission Control Protocol (TCP), the User Datagram Protocol (UDP), and / or the Internet Protocol (IP) from the TCP / IP internet protocol suite. The networks 112 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, the networks 112 may include another CN connected to one or more RANs, which may employ the same RAT as the RAN 104 / 114 or a different RAT.

[0036] Some or all of the WTRUs 102a, 102b, 102c, 102d in the communication system 100 may include multi-modal capabilities (e.g., the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers for communicating with different wireless networks via different wireless links). Figure 1A The WTRU 102c shown in FIG. 1 may be configured to communicate with the base station 114a, which may employ a cellular-based radio technology, and with the base station 114b, which may employ an IEEE 802 radio technology.

[0037] Figure 1B is a system diagram illustrating an example WTRU 102. Figure 1B As shown, the WTRU 102 may include, among other things, a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a keypad 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power supply 134, a global positioning system (GPS) chipset 136, and / or other elements / peripherals 138. It will be appreciated that the WTRU 102 may include any subcombination of the foregoing elements while remaining consistent with an embodiment.

[0038] The processor 118 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, etc. The processor 118 may perform signal coding, data processing, power control, input / output processing, and / or any other functionality that enables the WTRU 102 to operate in a wireless environment. The processor 118 may be coupled to the transceiver 120, which may be coupled to the transmit / receive element 122. Although Figure 1B The processor 118 and the transceiver 120 are depicted as separate components, but it is understood that the processor 118 and the transceiver 120 may be integrated together, such as in an electronic package or chip.

[0039] The transmit / receive element 122 can be configured to transmit signals to a base station (e.g., base station 114a) or receive signals from a base station via the air interface 116. For example, in one embodiment, the transmit / receive element 122 can be an antenna configured to transmit and / or receive RF signals. In one embodiment, the transmit / receive element 122 can be, for example, an emitter / detector configured to transmit and / or receive IR, UV, or visible light signals. In one embodiment, the transmit / receive element 122 can be configured to transmit and / or receive both RF and light signals. It should be understood that the transmit / receive element 122 can be configured to transmit and / or receive any combination of wireless signals.

[0040] Although the transmit / receive element 122 is Figure 1B Although depicted as a single element in FIG1 , the WTRU 102 may include any number of transmit / receive elements 122. For example, the WTRU 102 may employ MIMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit / receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 116.

[0041] The transceiver 120 may be configured to modulate signals to be transmitted by the transmit / receive element 122 and demodulate signals received by the transmit / receive element 122. As described above, the WTRU 102 may have multi-modal capabilities. Thus, the transceiver 120 may include multiple transceivers to enable the WTRU 102 to communicate via multiple RATs, such as NR and IEEE 802.11.

[0042] The processor 118 of the WTRU 102 may be coupled to, and may receive user input data from, a speaker / microphone 124, a keypad 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) display unit or an organic light emitting diode (OLED) display unit). The processor 118 may also output user data to the speaker / microphone 124, the keypad 126, and / or the display / touchpad 128. Furthermore, the processor 118 may access information from and store data in any type of suitable memory, such as non-removable memory 130 and / or removable memory 132. The non-removable memory 130 may include random access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. In other embodiments, the processor 118 may access information from and store data in memory that is not physically located on the WTRU 102, such as on a server or a home computer (not shown).

[0043] The processor 118 may receive power from the power source 134 and may be configured to distribute and / or control power to the other components in the WTRU 102. The power source 134 may be any suitable device for powering the WTRU 102. For example, the power source 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel-metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0044] The processor 118 may also be coupled to the GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 102. In addition to or in lieu of information from the GPS chipset 136, the WTRU 102 may receive location information from a base station (e.g., base stations 114a, 114b) over the air interface 116 and / or determine its location based on the timing of signals received from two or more nearby base stations. It will be appreciated that the WTRU 102 may acquire location information by any suitable location-determination method while remaining consistent with an embodiment.

[0045] The processor 118 may also be coupled to other components / peripherals 138, which may include one or more software and / or hardware modules / units that provide additional features, functionality, and / or wired or wireless connectivity. For example, the components / peripherals 138 may include an accelerometer, an electronic compass, a satellite transceiver, a digital camera (e.g., for photos and / or videos), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands-free headset, Module, frequency modulation (FM) radio unit, digital music player, media player, electronic game player module, Internet browser, virtual reality and / or augmented reality (VR / AR) device, activity tracker, etc. Component / peripheral 138 may include one or more sensors, which may be one or more of a gyroscope, an accelerometer, a Hall effect sensor, a magnetometer, an orientation sensor, a proximity sensor, a temperature sensor, a time sensor; a geolocation sensor; an altimeter, a light sensor, a touch sensor, a magnetometer, a barometer, a gesture sensor, a biometric sensor, and / or a humidity sensor.

[0046] The WTRU 102 may include a full-duplex radio for which transmission and reception of some or all signals (e.g., associated with particular subframes for both uplink (e.g., for transmission) and downlink (e.g., for reception)) may be concurrent and / or simultaneous. The full-duplex radio may include an interference management unit to reduce and / or substantially eliminate self-interference via hardware (e.g., a choke) or via signal processing by a processor (e.g., a separate processor (not shown) or via the processor 118). In one embodiment, the WTRU 102 may include a half-duplex radio for which transmission and reception of some or all signals (e.g., associated with particular subframes for both uplink (e.g., for transmission) or downlink (e.g., for reception)) may be concurrent and / or simultaneous.

[0047] Figure 1C 1 is a system diagram illustrating the RAN 104 and the CN 106 according to an embodiment. As described above, the RAN 104 may employ an E-UTRA radio technology to communicate with the WTRUs 102a, 102b, and 102c over the air interface 116. The RAN 104 may also be in communication with the CN 106.

[0048] The RAN 104 may include eNode-Bs 160a, 160b, 160c, though it will be appreciated that the RAN 104 may include any number of eNode-Bs while remaining consistent with an embodiment. The eNode-Bs 160a, 160b, 160c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116. In one embodiment, the eNode-Bs 160a, 160b, 160c may implement MIMO technology. Thus, for example, the eNode-B 160a may use multiple antennas to transmit wireless signals to, and receive wireless signals from, the WTRU 102a.

[0049] Each of the eNode-Bs 160a, 160b, and 160c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, user scheduling in uplink (UL) and / or downlink (DL), etc. Figure 1C As shown, eNode-Bs 160a, 160b, and 160c may communicate with each other via an X2 interface.

[0050] Figure 1C The CN 106 shown in FIG may include a mobility management entity (MME) 162, a serving gateway (SGW) 164, and a packet data network (PDN) gateway (PGW) 166. While each of the foregoing elements is depicted as part of the CN 106, it should be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0051] The MME 162 may be connected to each of the eNode-Bs 160a, 160b, and 160c in the RAN 104 via an S1 interface and may serve as a control node. For example, the MME 162 may be responsible for authenticating users of the WTRUs 102a, 102b, and 102c, bearer activation and deactivation, and selecting a particular serving gateway during an initial attach of the WTRUs 102a, 102b, and 102c. The MME 162 may also provide a control plane function for facilitating switching between the RAN 104 and other RANs (not shown) that employ other radio technologies, such as GSM and / or WCDMA.

[0052] The SGW 164 may be connected to each of the eNode-Bs 160a, 160b, 160c in the RAN 104 via an S1 interface. The SGW 164 may generally route and forward user data packets to and from the WTRUs 102a, 102b, 102c. The SGW 164 may perform other functions, such as anchoring the user plane during inter-eNode-B handovers, triggering paging when downlink data is available for the WTRUs 102a, 102b, 102c, managing and storing the context of the WTRUs 102a, 102b, 102c, and the like.

[0053] The SGW 164 may be connected to the PGW 166, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices.

[0054] The CN 106 may facilitate communications with other networks. For example, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices. For example, the CN 106 may include, or may communicate with, an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that acts as an interface between the CN 106 and the PSTN 108. In addition, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0055] Even though the WTRU Figures 1A-1D Although described as a wireless terminal, it is contemplated that in certain representative embodiments such a terminal may employ (eg, temporarily or permanently) a wired communication interface with a communication network.

[0056] In a representative embodiment, the other network 112 may be a WLAN.

[0057] A WLAN in infrastructure basic service set (BSS) mode may have an access point (AP) for the BSS and one or more stations (STAs) associated with the AP. The AP may have access or an interface to a distributed system (DS) or another type of wired / wireless network that transmits traffic to and / or out of the BSS. Traffic originating from outside the BSS and destined for a STA may reach the AP and be delivered to the STA. Traffic from a STA to a destination outside the BSS may be sent to the AP for delivery to the corresponding destination. For example, traffic between STAs within a BSS may be sent through the AP, where the source STA may send traffic to the AP, and the AP may deliver traffic to the destination STA. Traffic between STAs within a BSS may be considered and / or referred to as peer-to-peer traffic. Peer-to-peer traffic may be sent between the source and destination STAs (e.g., directly between them) using direct link setup (DLS). In certain representative embodiments, the DLS may use 802.11e DLS or 802.11z tunnel DLS (TDLS). A WLAN using an independent BSS (IBSS) mode may not have an AP, and STAs (eg, all STAs) within or using the IBSS may communicate directly with each other. The IBSS communication mode is sometimes referred to herein as an "ad-hoc" communication mode.

[0058] When using 802.11ac infrastructure operating mode or a similar operating mode, the AP can transmit beacons on a fixed channel (such as a primary channel). The primary channel can be a fixed width (e.g., a wide bandwidth of 20 MHz) or a width dynamically set via signaling. The primary channel can be the operating channel of the BSS and can be used by STAs to establish a connection with the AP. In certain representative embodiments, carrier sense multiple access with collision avoidance (CSMA / CA) can be implemented, such as in an 802.11 system. For CSMA / CA, STAs (e.g., each STA) including the AP can sense the primary channel. If a particular STA senses / detects and / or determines that the primary channel is busy, the particular STA can back off. One STA (e.g., only one station) can transmit at any given time in a given BSS.

[0059] High throughput (HT) STAs may communicate using a 40 MHz wide channel, for example, via a combination of a primary 20 MHz channel and adjacent or non-adjacent 20 MHz channels to form the 40 MHz wide channel.

[0060] Very high throughput (VHT) STAs can support 20 MHz, 40 MHz, 80 MHz, and / or 160 MHz wide channels. 40 MHz and / or 80 MHz channels can be formed by combining consecutive 20 MHz channels. A 160 MHz channel can be formed by combining eight consecutive 20 MHz channels, or by combining two discontinuous 80 MHz channels, which can be referred to as an 80+80 configuration. For the 80+80 configuration, after channel coding, the data can pass through a segment parser that can separate the data into two streams. Each stream can be subjected to inverse fast Fourier transform (IFFT) processing and time domain processing separately. The streams can be mapped onto two 80 MHz channels, and the data can be transmitted by the transmitting STA. At the receiver of the receiving STA, the operations of the above-mentioned 80+80 configuration can be reversed, and the combined data can be sent to the medium access control (MAC) layer, entity, etc.

[0061] 802.11af and 802.11ah support sub-1GHz operating modes. The channel operating bandwidth and carrier are reduced in 802.11af and 802.11ah relative to those used in 802.11n and 802.11ac. 802.11af supports 5MHz, 10MHz, and 20MHz bandwidths in the TV White Space (TVWS) spectrum, and 802.11ah supports 1MHz, 2MHz, 4MHz, 8MHz, and 16MHz bandwidths using non-TVWS spectrum. According to a representative embodiment, 802.11ah can support metered type control / machine type communication (MTC), such as MTC devices in macro coverage areas. MTC devices may have certain capabilities, for example, limited capabilities, including support for (e.g., only support for) certain and / or limited bandwidths. MTC devices may include batteries with battery life above a threshold (e.g., in order to maintain very long battery life).

[0062] WLAN systems that can support multiple channels and channel bandwidths (such as 802.11n, 802.11ac, 802.11af, and 802.11ah) include a channel that can be designated as a primary channel. The primary channel can have a bandwidth equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel can be set and / or limited by the STA that supports the smallest bandwidth operating mode among all STAs operating in the BSS. In the example of 802.11ah, for a STA that supports (e.g., only supports) a 1 MHz mode (e.g., an MTC-type device), the primary channel can be 1 MHz wide, even if the AP and other STAs in the BSS support 2 MHz, 4 MHz, 8 MHz, 16 MHz, and / or other channel bandwidth operating modes. Carrier sensing and / or network allocation vector (NAV) settings may depend on the status of the primary channel. If the primary channel is busy transmitting to the AP, for example, due to a STA (supporting only a 1 MHz operating mode), the entire available frequency band may be considered busy, even if most of the frequency band remains idle and may be available.

[0063] In the United States, 802.11ah can be used in the available frequency band from 902MHz to 928MHz. In South Korea, the available frequency band is from 917.5MHz to 923.5MHz. In Japan, the available frequency band is from 916.5MHz to 927.5MHz. The total available bandwidth for 802.11ah ranges from 6MHz to 26MHz, depending on the country code.

[0064] Figure 1D 1 is a system diagram illustrating the RAN 113 and the CN 115 according to an embodiment. As described above, the RAN 113 may employ NR radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 116. The RAN 113 may also be in communication with the CN 115.

[0065] The RAN 113 may include gNBs 180a, 180b, and 180c, though it will be appreciated that the RAN 113 may include any number of gNBs while remaining consistent with an embodiment. The gNBs 180a, 180b, and 180c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, and 102c over the air interface 116. In one embodiment, the gNBs 180a, 180b, and 180c may implement MIMO technology. For example, the gNBs 180a and 180b may utilize beamforming to transmit signals to and / or receive signals from the WTRUs 102a, 102b, and 102c. Thus, for example, the gNB 180a may use multiple antennas to transmit wireless signals to and / or receive wireless signals from the WTRU 102a. In one embodiment, the gNBs 180a, 180b, and 180c may implement carrier aggregation technology. For example, the gNB 180a may transmit multiple component carriers (not shown) to the WTRU 102a. A subset of these component carriers may be on unlicensed spectrum, while the remaining component carriers may be on licensed spectrum. In one embodiment, the gNBs 180a, 180b, and 180c may implement coordinated multi-point (CoMP) technology. For example, the WTRU 102a may receive coordinated transmissions from gNB 180a and gNB 180b (and / or gNB 180c).

[0066] The WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using transmissions associated with scalable numerology. For example, the OFDM symbol spacing and / or OFDM subcarrier spacing may be different for different transmissions, different cells, and / or different portions of the wireless transmission spectrum. The WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using subframes or transmission time intervals (TTIs) of varying or scalable lengths (e.g., including a variable number of OFDM symbols and / or duration of a variable length of absolute time).

[0067] The gNBs 180a, 180b, 180c may be configured to communicate with the WTRUs 102a, 102b, 102c in a standalone configuration and / or a non-standalone configuration. In a standalone configuration, the WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c without accessing other RANs (e.g., such as the eNode-Bs 160a, 160b, 160c). In a standalone configuration, the WTRUs 102a, 102b, 102c may utilize one or more of the gNBs 180a, 180b, 180c as mobility anchors. In a standalone configuration, the WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using signals in an unlicensed band. In a non-standalone configuration, the WTRUs 102a, 102b, 102c may communicate / connect with the gNBs 180a, 180b, 180c while also communicating / connecting with another RAN, such as the eNode-Bs 160a, 160b, 160c. For example, the WTRUs 102a, 102b, 102c may implement DC principles to communicate with one or more gNBs 180a, 180b, 180c and one or more eNode-Bs 160a, 160b, 160c substantially simultaneously. In a non-standalone configuration, the eNode-Bs 160a, 160b, 160c may serve as mobility anchors for the WTRUs 102a, 102b, 102c, and the gNBs 180a, 180b, 180c may provide additional coverage and / or throughput for the serving WTRUs 102a, 102b, 102c.

[0068] Each of the gNBs 180a, 180b, 180c may be associated with a specific cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, user scheduling in UL and / or DL, network slicing support, dual connectivity, interworking between NR and E-UTRA, routing user plane data to a user plane function (UPF) 184a, 184b, routing control plane information to an access and mobility management function (AMF) 182a, 182b, etc. Figure 1D As shown, gNB180a, 180b, and 180c can communicate with each other through the Xn interface.

[0069] Figure 1DThe illustrated CN 115 may include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one session management function (SMF) 183a, 183b, and at least one data network (DN) 185a, 185b. While each of the aforementioned elements is depicted as part of the CN 115, it should be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0070] The AMF 182a, 182b may be connected to one or more gNBs 180a, 180b, 180c in the RAN 113 via the N2 interface and may act as a control node. For example, the AMF 182a, 182b may be responsible for authenticating users of the WTRU 102a, 102b, 102c, supporting network slicing (e.g., handling different protocol data unit (PDU) sessions with different requirements), selecting a specific SMF 183a, 183b, managing registration areas, terminating NAS signaling, mobility management, and the like. The AMF 182a, 182b may use network slicing, for example, to customize CN support for the WTRU 102a, 102b, 102c based on the type of service being used by the WTRU 102a, 102b, 102c. For example, different network slices may be established for different use cases, such as services relying on ultra-reliable low latency (URLLC) access, services relying on enhanced massive mobile broadband (eMBB) access, services for machine-to-machine communication (MTC) access, and the like. The AMF 162 may provide a control plane function for switching between the RAN 113 and other RANs (not shown) that employ other radio technologies, such as LTE, LTE-A, LTE-A Pro, and / or non-3GPP access technologies such as Wi-Fi.

[0071] SMFs 183a and 183b can connect to AMFs 182a and 182b in CN 115 via the N11 interface. SMFs 183a and 183b can also connect to UPFs 184a and 184b in CN 115 via the N4 interface. SMFs 183a and 183b can select and control UPFs 184a and 184b and configure the routing of traffic through UPFs 184a and 184b. SMFs 183a and 183b can perform other functions, such as managing and allocating UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, and providing downlink data notifications. PDU session types can be IP-based, non-IP-based, Ethernet-based, and so on.

[0072] The UPF 184a, 184b may be connected to one or more gNBs 180a, 180b, 180c in the RAN 113 via the N3 interface, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, for example, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices. The UPF 184, 184b may perform other functions, such as routing and forwarding packets, enforcing user plane policies, supporting multi-homed PDU sessions, handling user plane QoS, buffering downlink packets, providing mobility anchoring, etc.

[0073] The CN 115 may facilitate communications with other networks. For example, the CN 115 may include, or may communicate with, an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that acts as an interface between the CN 115 and the PSTN 108. Furthermore, the CN 115 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers. In one embodiment, the WTRUs 102a, 102b, 102c may be connected to a local data network (DN) 185a, 185b through the UPF 184a, 184b via the N3 interface to the UPF 184a, 184b and the N6 interface between the UPF 184a, 184b and the DN 185a, 185b.

[0074] Given that Figures 1A-1D as well as Figures 1A-1D

[0066] As described herein, one or more or all of the functionality described herein with respect to any of the following: the WTRU 102a-d, base station 114a-b, eNode-B 160a-c, MME 162, SGW 164, PGW 166, gNB 180a-c, AMF 182a-b, UPF 184a-b, SMF 183a-b, DN 185a-b, and / or any other element / device(s) described herein may be performed by one or more emulation elements / devices (not shown). An emulation device may be one or more devices configured to emulate one or more or all of the functionality described herein. For example, an emulation device may be used to test other devices and / or simulate network and / or WTRU functionality.

[0075] Emulated devices can be designed to implement one or more tests of other devices in a laboratory environment and / or in a carrier network environment. For example, one or more emulated devices can perform one or more or all functions while being fully or partially implemented and / or deployed as part of a wired and / or wireless communication network in order to test other devices within the communication network. One or more emulated devices can perform one or more or all functions while being temporarily implemented / deployed as part of a wired and / or wireless communication network. For testing purposes, the emulated device can be directly coupled to another device and / or can use over-the-air wireless communication to perform the tests.

[0076] One or more emulated devices can perform one or more functions, including all functions, without being implemented / deployed as part of a wired and / or wireless communication network. For example, the emulated device can be used in a test lab and / or a test scenario in a non-deployed (e.g., testing) wired and / or wireless communication network to enable testing of one or more components. The one or more emulated devices can be test devices. The emulated device can transmit and / or receive data using direct RF coupling and / or wireless communication via RF circuitry (e.g., which can include one or more antennas).

[0077] introduce

[0078] Use cases for XR applications

[0079] 3GPP document TR26.928 proposes a method to support Figure 6 The architecture of Extended Reality (XR) is shown in the document. In this document, XR is defined as follows: "Extended Reality (XR) refers to all real and virtual environments generated by computer technology and wearable devices, as well as the associated human-computer interactions. It includes representative forms such as augmented reality (AR), mixed reality (MR), virtual reality (VR), and areas interspersed between them."

[0080] In the present disclosure, the definition of XR corresponds to the above description.

[0081] The Internet Engineering Task Force (IETF) Media Operations Practices (MOPS) working group document has proposed the following use cases for XR applications:

[0082] Consider a group of tourists being taken on a tour around the historic site of the Tower of London. As they move around the site and within the historic buildings, they can watch and listen to 3D historical scenes generated by an AR app and then superimposed on their real-world view through their AR headsets. The headsets then continuously update their view as they move around.

[0083] The AR application first processes the scene viewed by the walking tourists in real time and identifies the objects that will be used as targets for high-resolution video overlays. It then generates high-resolution 3D images of the historical scenes in real time, relevant to the tourists' perspective. These generated video images are then superimposed on the real-world view as seen by the tourists.

[0084] Components of AR applications perform computationally intensive tasks such as real-time generation and processing of high-quality video content. As a result, on AR devices (such as AR glasses), the chipsets involved in the computation generate excessive heat. In addition, when running such applications, the batteries on such devices discharge rapidly. A solution to the heat dissipation and battery consumption issues is to offload the processing and video generation tasks to a remote cloud. However, running such tasks on the cloud is not feasible because the end-to-end latency must be on the order of a few milliseconds. In addition, such applications require high bandwidth and low jitter to provide users with a high quality of experience (QoE). To achieve such strict time constraints, computationally intensive tasks can be offloaded to edge devices.

[0085] This use case may require (e.g., require) computationally intensive algorithms to process scenes and generate videos in real time. Computationally intensive tasks can be offloaded to the edge. The following describes the response time and associated issues of such offloading.

[0086] Current XR applications are organized around a processing loop that begins with a WTRU (e.g., UE) sending a user command and ends with the XR platform running at the edge returning the resulting video frame. Assuming frames are generated at a 30fps rate, the time it takes to return the resulting frame is 33ms (approximately 1 / 30) + the network latency between the WTRU (e.g., UE) and the XR system running at the edge. This can be (essentially) the response time it takes for the XR system to render a frame from the time the user makes a certain movement (say, turning their head to look around) until the corresponding frame is rendered on the user's device. This response time can be at or below the desired 100ms to give a realistic display only when the time it takes to generate the frame is at or below 33ms and the network latency is at or below 67ms. The response time, without the overhead of network latency, can be referred to as the frame time. When we assume a 30fps frame rate, the frame time might be 33ms. The next section shows how predicting user input can reduce response time.

[0087] Predicting user input in XR applications

[0088] Figure 2The figure illustrates how the response time of XR applications such as cloud gaming depends on latency when the user input to the WTRU (e.g., UE) is not predicted, but simply processed and sent back by the cloud. Time is discretized, for example, at 33ms per clock tick (which is the frame time as defined above). At clock tick t5, the user's input i5 is sent to the server on the cloud. Assuming an RTT of 4 clock ticks (4×33=132ms), the user input arrives at the server at t7 after 2 clock ticks. On the server, it takes one clock tick until t8 to generate the appropriate frame f5. As shown in the figure, this frame arrives at the client at t10 after 2 clock ticks. Therefore, the total response time is 5 clock ticks, which is 5×33=165ms milliseconds. The amount of increase in response time is equal to the RTT (4 clock ticks).

[0089] Figure 3 It illustrates how the impact of network latency can be mitigated by predicting the user's actions in advance and generating corresponding frames that are then sent to the WTRU (eg, UE). Figure 3 Each clock tick in corresponds to a discrete time of 33ms. At t0, the user's input i0 is sent to the server. Assuming an RTT of 4 clock ticks, the input i0 arrives at the server at time t2. The server then predicts the user input sequence for, say, 1 RTT in the future and sends the corresponding frame f'5 of the predicted input i'5 to the WTRU (e.g., UE). If the prediction is accurate, this predicted frame f'5 is then rendered on the WTRU (e.g., UE). Figure 3 As shown, the response time between the i5 and the f'5 is now significantly reduced.

[0090] Overview

[0091] Architecture of an XR System Running on a WTRU (e.g., UE)

[0092] Architecture of components on a WTRU (e.g., UE)

[0093] Figure 4 Illustrated is the architecture of the proposed system that can run on a user equipment (UE) to deliver low latency to XR applications.

[0094] like Figure 4 The illustrated architecture may include (e.g., consist of) the following components, the inputs and outputs of which are discussed below:

[0095] AR / VR Logic 401: The user's "user action" can be interpreted by the "AR / VR Logic" module of the WTRU (e.g., UE) as a change in the user's view. Figure 4As shown, these changes can be carried to the "Data Gathering" or "MPC Controller" modules as a data structure marked "User View Changes". Figure 4 The data structure marked with "Current User View Change" is carried to the "Decision Module".

[0096] GPU Rendering 402: One input to this component can be something like Figure 4 The "current user view change" is shown carried in a data structure. This data may be forwarded by the "decision module" and may originally come from the "AR / VR logic" module on the WTRU (e.g., UE). Another input may be the "predicted user view change" carried in a data structure and forwarded by the "decision module." Figure 4 It is shown that this second input can come from the "MPC Controller" component on the WTRU (e.g., UE) or edge device. The "GPU Rendering" component can convert these data structures into appropriate video frames as output.

[0097] Video Decoder 403: This module can receive the "encoded video" stream from the network and can decompress it into a video frame format suitable for rendering the scene on the WTRU (e.g., UE), such as Figure 4 shown.

[0098] Server interaction module 404: This module can provide services including but not limited to marshaling arguments into byte arrays and unmarshaling the byte arrays back into a suitable data format, thereby providing authentication services, naming and directory services, programming abstractions, etc.

[0099] Decision Module 405: This module may receive as input the “Current User View Change” carried in a data structure from the “AR / VR Logic” component on the WTRU (e.g., UE), such as Figure 4 In addition, Figure 4 As shown, it can receive "predicted user view change" data structures from the "MPC controller" component of the WTRU (e.g., UE) or the "MPC controller" of the edge device. For example, based on the decision algorithm, both the "current user view change" data structure and the "predicted user view change" data structure can be output to the "GPU rendering" module and / or a call can be made to the program of the "server interaction module."

[0100] In one embodiment, the decision algorithm on the decision module may select any one of the following selected combinations: (a) low-quality video rendering using the GPU rendering module of the WTRU (e.g., UE) + low-quality prediction using the DNN of the WTRU (e.g., UE); (b) low-quality video rendering using the GPU rendering module of the WTRU (e.g., UE) + high-quality prediction using the DNN of the edge device; (c) low-quality prediction using the DNN of the WTRU (e.g., UE) + high-quality video rendering using the GPU rendering module of the edge device; (d) high-quality video rendering using the GPU rendering module of the edge device + high-quality prediction using the DNN of the edge device.

[0101] This selection may be made in coordination with a decision module on the edge device based on the exchange of telemetry data. In one embodiment, the telemetry data may be the round trip time (RTT) between the WTRU (eg, UE) and the edge device.

[0102] MPC Controller 406: This module runs the Model-Predictive-Controller (MPC) algorithm, which can be a (e.g., classical) control strategy. Figure 4 The predictor trained by the deep neural network (DNN) labeled "Trained Neural Network" in the

[15] gets predictions labeled "UE Neural Network Data" (in one embodiment, this prediction can be a point estimate, while in another embodiment, it can be a probability distribution). These predictions can be used as input to optimize the objective QoE function. In one embodiment, QoE can be a function of round trip time (RTT). The MPC controller can get updates on the user's movement (e.g., as the current "user view change") from the "AR / VR Logic" module. The final input can be updated to the value from Figure 4 Numerical parameters of the control algorithm of the MPC controller labeled "neural network update" in the edge device. The output of the MPC controller can be a data structure carrying the "predicted user view change", which can be sent to the "decision module".

[0103] Data Aggregation 407: This module may collect “User View Changes” carried in a data structure from the “AR / VR Logic” module of the WTRU (e.g., UE) and may use this data to train Figure 4 The DNN shown is labeled "Trained Neural Network." In one embodiment, this training can be performed once a day.

[0104] Trained Neural Network 408: This component may be a (e.g., small) DNN that can run on a UE. Due to its (e.g., small) size, this DNN may produce approximate predictions based (e.g., only) on inputs from the "data aggregation" module of the WTRU (e.g., UE) discussed above. Its output may be a prediction of the future state of the inputs of the users in the environment. In one embodiment, the prediction may be in the form of a point estimate. In another embodiment, the prediction may be in the form of a probability distribution. These are Figure 4 The approximate predictions labeled "UE Neural Network Data" can be used by the MPC controller for the WTRU (e.g., UE) itself. It can be sent to the edge device using the "Server Interaction Module" program.

[0105] Architecture of components running on the edge

[0106] Figure 5 Components running on an edge device 500 are shown. This architecture may include (e.g., consist of) the following components, the inputs and outputs of which are discussed below:

[0107] GPU Rendering 501: Its input can be a data structure carrying “changes in user view” or a data structure carrying “predicted user view changes” from a “decision module” at the edge and originally coming from a WTRU (e.g., UE) (in Figure 5 The GPU Rendering module can convert the data structure into appropriate video frames to be fed as output to the Video Encoder component at the edge.

[0108] Client Interaction Module 502: This component can get its inputs from the UE. These inputs can be divided into three categories: "Current User View Change", "Predicted User View Change" and "UE Neural Network Data". This "Client Interaction Module" can then forward these inputs to the "Decision Module". "Current User View Change" can also be forwarded to the "MPC Controller" at the edge. The "Client Interaction Module" can provide services, including but not limited to marshaling of arguments into byte arrays and unmarshaling of byte arrays back into the appropriate data format, authentication services, naming and directory services, programming abstractions, etc.

[0109] MPC Controller 503: This module may run, for example, a Model-Predictive-Controller (MPC) algorithm, which may be a classical control strategy. It may receive data from the WTRU (e.g., UE) ( Figure 4) on a DNN trained predictor “trained neural network” (which may be a low quality prediction) Figure 5 The predictions are marked as “DNN data from UE”. It can be obtained from the DNN trained predictor “trained neural network” on the edge device. Figure 5 As shown in the prediction, the DNN-trained predictor "Trained Neural Network" on the edge device combines the low-quality prediction from the WTRU (e.g., UE) "Trained Neural Network" with the edge's own "Trained Neural Network." The decision to choose the prediction input from these two options can be made by the "Decision Module" on the edge device, discussed below. In one embodiment, this prediction can be a point estimate, while in another embodiment, it can be a probability distribution that serves as an input to optimize the target QoE function. In one embodiment, QoE can be a function of RTT.

[0110] like Figure 5 As shown, the MPC controller can obtain the update of the user view carried in the data structure as "current user view change" from the "AR / VR logic" module on the WTRU (e.g., UE), and send it from the "client interaction module" of the WTRU (e.g., UE) to the "server interaction module" of the edge device.

[0111] One output of the MPC controller can be a data structure carrying the "predicted user view change", such as Figure 5 As shown in FIG, it can be sent to the “GPU rendering” component on the edge device. The second output can be updated as a numerical parameter of the control algorithm from the “MPC controller” of the edge device to the “MPC controller” of the WTRU (e.g., UE). The final output can be a data structure carrying the “predicted user view change”, which can be transmitted via Figure 5 The procedures of the "client interaction module" at the edge are shown to be sent to the WTRU (eg, UE).

[0112] Video Encoder 504: Its input may be appropriate video frames to be compressed from the "GPU Rendering" module at the edge. Its output may be a set of compressed video frames using appropriate encoding.

[0113] Video Streaming Module 505: The input of this component may be a set of compressed video frames using appropriate encoding. Its output may be a video stream suitable for transmission over a network.

[0114] Decision module 506: This module may accept the following inputs from the “client interaction module” of the edge device: the first input may be the “current user view change” carried in a data structure from the WTRU (e.g., UE), the second input may be the “predicted user view change” carried in a data structure, and the third input that may be accepted may be the “WTRU (e.g., UE) neural network data”, which may be a point estimate in one embodiment and a probability distribution in another embodiment.

[0115] The outputs can be as follows: the module can forward the "current user view change" as a data structure carrying state updates to the "GPU rendering" module or the "data aggregation" module; the module can also forward the "predicted user view change" carried in the data structure to the "GPU rendering" module on the edge device. Another output can be "DNN data from UE" (which can be a point estimate in one embodiment and a probability distribution in another embodiment) to Figure 5 The same “DNN data from UE” can also be forwarded to the “MPC controller” component on the edge device, such as Figure 5 shown.

[0116] In one embodiment, the decision algorithm on the decision module selects any one of the following combinations: (a) low-quality video rendering using the GPU rendering module of the WTRU (e.g., UE) + low-quality prediction using the DNN of the WTRU (e.g., UE); (b) low-quality video rendering using the GPU rendering module of the WTRU (e.g., UE) + high-quality prediction using the DNN of the edge device; (c) low-quality prediction using the DNN of the WTRU (e.g., UE) + high-quality video rendering using the GPU rendering module of the edge device; (d) high-quality video rendering using the GPU rendering module of the edge device + high-quality prediction using the DNN of the edge device. This selection can be made in coordination with the decision module on the WTRU (e.g., UE) based on the exchange of telemetry data. In one embodiment, the telemetry data can be the round-trip time (RTT) between the WTRU (e.g., UE) and the edge device.

[0117] Data Aggregator 507: This module can collect Figure 5 The current user view changes are marked as "data for training" and carried in the data structure of the "decision module" from the edge device. It can use this data to train Figure 5 In one embodiment, this training can be performed once a day.

[0118] Trained Neural Network 508: This can be a DNN with the following inputs: The first input can be the data from the "Data Aggregator" 507 as discussed above for training the DNN. In one embodiment, this training can be done once a day. The second input can be Figure 5 "DNN data from UE" is shown forwarded by the "Decision Module". This second input, originally from the WTRU's (e.g., UE's) (e.g., smaller) DNN, may be an approximate point estimate in one embodiment and an approximate probability distribution in another embodiment. The output of this DNN may be a prediction, which may be an exact point estimate in one embodiment and an exact probability distribution in another embodiment.

[0119] Mapping of WTRU (e.g., UE) and edge components to the 3GPP architecture

[0120] 3GPP specifications (e.g. TR 26.928) may propose Figure 6 The architecture shown supports extended reality (XR).

[0121] AR / VR low latency requirements are discussed in this document.

[0122] This section proposes an architecture for extending the 3GPP proposal.

[0123] exist Figure 6 In the context of 5G-XR, a “5G-XR” client can be the recipient of 5G-XR session data that can be accessed by 5G-XR aware applications through well-defined interfaces / APIs. This client accesses the 5G-XRAF (e.g. Figure 6 This 5G-XRAF can be an application function dedicated to 5G-XR services similar to that defined in the 3GPP specification (e.g. TS23.501, clause 6.2.10). In addition, the 5G-XR client accesses the 5G-XRAS (e.g. Figure 6 ). 5G-XRAS can be an application server dedicated to 5G-XR services.

[0124] Figure 7 It shows how the described components can be used to extend the functionality of the architecture proposed by 3GPP specifications (e.g. TR 26.928). A "5G-XR client" can be implemented by including Figure 4 It is extended by the "Server Interaction Module", "GPU Rendering" and "Video Decoder" components.

[0125] According to 3GPP specifications (e.g. TR26.928), AI components can be considered dedicated to a specific XR application and can be created for that specific XR application. Figure 7 As shown, "MPC controller," "data aggregation," and "trained neural network" can be part of the 5G-XR application in this embodiment. "AR / VR logic" and "decision module" can also be part of the application.

[0126] Trusted DN (Data Network) module can act as a "client interaction module", such as Figure 7 In addition, in this embodiment, “GPU rendering”, “video encoder” and “video streaming” can be part of the trusted DN.

[0127] The “Decision Module”, “MPC Controller”, “Data Aggregation” and “Trained Neural Network” components can extend the functionality of the “External Data Network (DN)” for 5G-XR application providers, such as Figure 7 shown.

[0128] Figure 8 yes Figure 7 A more detailed view of the system diagram. The "XR Engine" in the 3GPP architecture can fulfill the roles of "GPU rendering" and "video decoder." The "XR Session Handler" can fulfill the role of "Server Interaction Module." Finally, the "5GXR Perception Application" can embody the components "AR / VR Logic," "Decision Module," "MPC Controller," "Data Aggregation," and "Trained Neural Network."

[0129] In such Figure 8 On the server side shown, the "5G XRAS" module can embody the components of "GPU rendering" and "video encoder" as well as "video streaming"; the "5G XR AF" module can embody the "client interaction module", and finally the "5GXR application provider" module can embody the components of "decision module", "MPC controller", "data aggregation" and "trained neural network".

[0130] although Figure 6-8 The embodiments in focus on 5G networks, but the ideas presented in this disclosure can be similarly applied to beyond 5G and 6G cellular networks.

[0131] Unique AI model in WTRU (e.g., UE) / edge node

[0132] Using a unique AI model in the WTRU (eg, UE) / edge node may not provide better KPIs due to the heavy-tailed WTRU (eg, UE) input distribution.

[0133] Due to the heavy-tailed nature of parameters such as network bandwidth and buffer occupancy, predictions using offline models for XR applications may be inaccurate. This may be because the law of large numbers (LLN) may be too slow to be of any practical use for such heavy-tailed distributions, or the LLN may not converge because the distribution of the derived data may not have a well-defined mean (or variance).

[0134] The LLN says that as the sample size gets larger, the sample mean gets closer to the population mean. This is mathematically expressed as where samples X1, X2,…, X N can be drawn from a distribution with finite mean μ (population mean) and variance (population / true variance). Then as the number of samples N becomes larger, the sample mean μ s becomes the population mean μ.

[0135] LLN relies on the assumption that the source distribution has finite mean and variance, which may not be the case for real-world data that follows a heavy-tailed distribution, such as the user input data distribution discussed in this disclosure.

[0136] A problem with such a distribution (i.e., a heavy-tailed distribution) is that the sample mean and the underlying true mean, or probability distribution of the data, may not be equal, resulting in the sample mean and variance being inappropriate for use as input to, or dependent upon, the calculation of a forecast. This can be mathematically captured by an LLN. Furthermore, such a distribution can suffer from the "expectation paradox," where the duration of a future occurrence of an event may be proportional to the time elapsed (elapsed duration) of a past occurrence of the same event.

[0137] This problem can be solved by observing the entire dataset to make accurate predictions. For the use cases discussed in this disclosure, this may not be achievable in practice. Therefore, any unseen data (e.g., outliers) that appear in real time may distort the predictions.

[0138] Trigger(s) for online (re)training of models running on WTRUs (e.g., UEs) and DNNs in the edge running XR applications

[0139] Current systems retrain their models once a day, running on DNNs. However, because user input to XR devices can be heavy-tailed as users move around, models running on DNNs can often become outdated because past user inputs cannot be used to predict future user inputs.

[0140] The environment in which XR devices can operate may be subject to failures of edge devices and communication links, variations in communication characteristics (such as bandwidth), and the creation and destruction of logical communication relationships between the WTRU (e.g., UE) and software components running on the edge. This high volatility in the environment can be another reason for prediction failures, as the rate at which DNN data arrives at both the WTRU (e.g., UE) and the edge can be heavy-tailed in nature and may not accurately match current user actions.

[0141] In other words, the user's real-time data is significantly different from the data used for DNN training, which may be adopted offline. In this case, the DNN may not be able to accurately predict the user's future movements. Therefore, retraining of the DNN weights based on the observed real-world data set (which may be adopted online) may be used (e.g., required), where the DNN weights may be updated. Since the retraining / recalibration of the DNN weights may occur on a pre-trained DNN, the overhead involved in online retraining / recalibration of the DNN will be significantly reduced when compared to online training of the DNN when it may be in a blank slate state.

[0142] Frequency of training

[0143] In current systems, the frequency with which models running on DNNs need to be updated (e.g., once a day) can be independent of the user's behavior and the dynamic nature of the environment. However, due to the heavy-tailed nature of user input to XR devices, these models quickly become outdated. This leads to incorrect predictions of user input, which need to be mitigated by frequent updates. However, current architectures may not be able to handle the need to frequently update models running on DNNs. A mechanism may be needed so that some prediction errors can be corrected without retraining the model, while more severe errors require retraining.

[0144] Updating or retraining of a DNN can be employed in any of the following ways: a periodic mechanism, a semi-persistent mechanism, and an aperiodic mechanism, all of which require significant overhead and should be invoked as a last resort.

[0145] Representative Procedures and Examples

[0146] The next section describes one or more embodiments that address a problem where unique AI models running on the WTRU (e.g., UE) and at the edge that are used to predict future user actions become outdated, leading to prediction errors. This may be due to the fact that user input at the WTRU (e.g., UE) follows a heavy-tailed distribution as discussed in the previous section.

[0147] The next section describes one or more embodiments that address trigger condition(s) for online (re)training of machine learning in a WTRU (e.g., UE) running an XR application.

[0148] According to one embodiment, this may involve a gradual upgrade of mechanisms used to improve the accuracy of the prediction, ultimately leading to the WTRU (e.g., UE) and the edge reaching an agreement on retraining their DNNs at, for example, a mutually acceptable time. That is, if the predicted user input deviates too much from the actual user input, this embodiment may use the following algorithm:

[0149] i. First, the UE-Edge distributed system can use a mutually agreed standard algorithm for error correction.

[0150] ii. If error correction fails, it can use a shorter time horizon to make predictions using the existing model.

[0151] ii. In cases where (eg only) using shorter time horizons does not yield accurate predictions, the proposed mechanism may decide to trigger re-learning.

[0152] According to this embodiment, a mechanism is provided that can (e.g., first) explore the alternative solution space and / or trigger online retraining of the model for the DNN.

[0153] Figure 9 The mechanism is described in the form of a message sequence diagram.

[0154] 1. (S901) The WTRU 102 (e.g., UE) may send pre-processed data from its DNN to the edge 910. This may be early exit data, which may be used by the edge's DNN for further processing and (e.g., then) forwarded to the MPC controller for prediction on the edge device. This prediction may be a point estimate or a probability distribution.

[0155] 2. (S902) The edge device 910 may use the edge's DNN and / or MPC controller to predict the user's view change, for example, by passing them early exit data sent by the UE. The edge may send the prediction of the user's view change to the WTRU 102 (e.g., UE).

[0156] 3. (S903) The WTRU 102 (eg, UE) may check whether the prediction error P e >τ1, i.e., whether the error is above a threshold. If the error is above a threshold, the WTRU (e.g., UE) may send (signal) a message to the edge to propose an error correction algorithm - examples of these algorithms include but are not limited to Kalman filtering, partial cube rendering, depth map rendering.

[0157] 4. (S904) The edge device 910 may calculate / determine whether the error is still above a threshold after running one of the error correction algorithms proposed by the WTRU 102 (e.g., UE). If so, the edge device 910 may send a message to the WTRU 102 (e.g., UE) suggesting a shorter prediction horizon for more accurate prediction.

[0158] 5. (S905) The WTRU 102 (e.g., UE) may observe / monitor the data exchange to determine if the shorter timeframe proposed by the edge device 910 may result in an error within the user's QoS requirements. If not, the WTRU 102 (e.g., UE) may configure a time for retraining the DNN on both the WTRU 102 (e.g., UE) and the edge device 910.

[0159] 6. (S906) The edge device 910 may agree (send ACK) in the message to the time configured from the WTRU 102 (eg, UE).

[0160] The described mechanism can ensure that relearning of the model running on the DNN on the WTRU 102 (e.g., UE) and edge 910 can be triggered (e.g., only) when other mitigation mechanisms such as error correction algorithms and using shorter time horizons for predictions do not improve the accuracy of the prediction. In the proposed embodiment, the triggering of relearning can depend on the dynamics of the environment, rather than being performed once a day. In other words, the triggering of relearning of the DNN adapts to the time-varying conditions of the environment.

[0161] The next section describes one or more embodiments that address the frequency of training.

[0162] Discussed in the previous section Figure 9 The mechanism in may support retraining of the model running on the DNN at a non-periodic frequency. More specifically, the WTRU (e.g., UE) may trigger retraining of the DNN model weights, for example, based on the variability of the conditions observed in the environment.

[0163] In the current architecture, retraining of the DNN model weights can be employed periodically, regardless of the variability of the conditions in the observed environment. This may impose unnecessary overhead (sending training data) if the conditions in the environment are already suitable for the UE-edge pair.

[0164] In the current architecture, retraining of DNN model weights may be employed in a semi-persistent manner, where the WTRU (e.g., UE) may activate retraining of the DNN aperiodically. In the case where the WTRU (e.g., UE) has already activated retraining, as described above, DNN retraining may occur periodically. The WTRU (e.g., UE) may choose to deactivate the DNN model based on conditions in the environment. More specifically, in a semi-persistent retraining design, the WTRU (e.g., UE) may activate / deactivate retraining aperiodically, while the retraining itself occurs periodically. Thus, semi-persistence may be a combination of aperiodic activation and periodic training. Figure 9 It is shown that the proposed embodiments can be used to implement scenarios where the WTRU (e.g., UE) and the edge can (e.g., need to) dynamically agree on retraining of a model running on a DNN, for example in the face of changing radio conditions and fluctuations in user movement resulting in heavy-tail input to the XR device.

[0165] The initial approach of running the error correction algorithm on the edge ensures that if the error is within a threshold, it can safely avoid retraining to get accurate predictions. This can be important because errors in predictions are caused by heavy-tailed user inputs, which can result in a large number of small errors and a small number of very large errors. For a large number of small errors, running the error correction algorithm may be sufficient.

[0166] However, for certain numbers of errors, error correction algorithms may not be sufficient. In these cases, forecasting over a shorter time horizon may be sufficient to mitigate the effects of these errors.

[0167] The model encapsulating the user behavior running on the WTRU (e.g., UE) and the DNN on the edge may become completely unsuitable for predictions due to the mismatch between the probability distribution used in the model and the probability distribution of the user's behavior. This can be achieved using (e.g., necessary) retraining of the model as a last resort, for example, as in Figure 9 is implemented in step S906.

[0168] Figure 9 The proposed mechanism supports retraining with aperiodic frequencies to cope with the heavy-tailed nature of user input.

[0169] Extensions to the Session Description Protocol (SDP)

[0170] Figure 9 The fields and data exchanged in the messages may be mapped to, for example, capability negotiation parameters of an SDP "Offer" procedure contained in the Update method of the Session Initiation Protocol (SIP) protocol.

[0171] In order to add new capabilities to SDP, a new attribute type "a=" may be defined at the session level. Session-level attributes may be listed before the first media line in SDP.

[0172] The next section shows the proposed set of capability attributes for a new SDP provider:

[0173] The novel message for step S901 may be sent as the following new SDP provider capability attribute:

[0174] a=dnn_data:parameter1 parameter 2……parameterN

[0175] Here "a=dnn_data:" is a new attribute representing DNN data, which has multiple parameters separated by spaces.

[0176] The novel message for step S902 may be sent as the following new SDP provider capability attribute:

[0177] a=predict:point_est:value prob_dist:parameter1 parameter 2……parameterN

[0178] Here, "a=predict:" is a new attribute representing a prediction, which has multiple arguments separated by spaces. In this proposal, the first argument "point_est:" represents the label for the point estimate prediction and has a parameter called "value". The next argument "prob_dist:" represents the label for the probability distribution prediction and can have multiple arguments separated by spaces.

[0179] The novel message for step S903 may be sent as the following new SDP provider capability attribute:

[0180] a=predict_error:parameter1 parameter 2……parameter

[0181] Here "a=predict_error:" is a new attribute representing the prediction error, which has multiple parameters separated by spaces.

[0182] The novel message for step S904 may be sent as the following new SDP provider capability attribute:

[0183] a=predict_time_horizon:parameter1 parameter 2……parameter

[0184] Here, "a=predict_time_horizon:" is a new attribute representing the time horizon proposed for prediction, and it has multiple parameters separated by spaces.

[0185] Finally, the novel message for step S905 may be sent as the following new SDP provider capability attribute:

[0186] a=retrain_time:parameter1 parameter 2……parameter

[0187] Here "a=retrain_time:" is a new attribute indicating the time the DNN needs to be retrained and has multiple parameters separated by spaces.

[0188] Example Implementation(s)

[0189] It is contemplated that the methods previously described herein may be combined in various ways. For example, Figure 10 The wireless communication method 1000 implemented by the WTRU 102 begins at a first step 1010, where the WTRU sends first information including neural network data to the edge device 910, where the neural network data is generated using a first DNN and user input data obtained at a first time. From step 1010, processing may proceed to a second step 1020.

[0190] At step 1020, the WTRU 102 may receive second information including first predicted user input data at a second time instant from the edge device 910, wherein the first predicted user input data is generated using the second DNN and the transmitted neural network data, and wherein the second time instant is associated with a first prediction time period starting from the first time instant. Processing may proceed from step 1020 to a third step 1030.

[0191] At step 1030 , the WTRU 102 may determine a first prediction error based on the first predicted user input at the second time instant and the obtained user input at the second time instant. Processing may proceed from step 1030 to a fourth step 1040 .

[0192] At step 1040, the WTRU 102 may send third information to the edge device 910, if the first prediction error is above a threshold, the third information including the first prediction error and / or an indication to trigger error correction of the algorithm calculated for predicting the user input.

[0193] In certain representative embodiments, the WTRU 102 may receive fourth information including a second prediction time period from the edge device 910, wherein the second prediction time period is shorter than the first prediction time period; and may determine a second prediction error based on the second predicted user input predicted at a third moment in time and additional user input obtained at the third moment in time, wherein the third moment in time is associated with the second prediction time period.

[0194] In certain representative embodiments, the WTRU 102 may send fourth information to the edge device 910 under the condition that the second prediction error is higher than a threshold, the fourth information including a fourth time for retraining the first DNN and the second DNN; may receive a confirmation message from the edge device 910 to retrain the first DNN and / or the second DNN at the fourth time; and may retrain the first DNN at the fourth time.

[0195] In certain representative embodiments, any one of the first information, the second information, the third information, and the fourth information uses SDP.

[0196] In certain representative embodiments, any one of the first information, the second information, the third information, and the fourth information is mapped to an SDP provider capability negotiation parameter.

[0197] In certain representative embodiments, any of the first information, the second information, the third information, and the fourth information of the WTRU 102 utilizes a SIP update function.

[0198] Figure 11 is a flow chart illustrating an exemplary procedure 1100 implemented by the edge device 910 .

[0199] refer to Figure 11 A representative method may include, at block 1110, receiving first information from the WTRU 102 including neural network data, wherein the neural network data is generated using a first distributed DNN and user input data obtained at a first time instant. At block 1120, the edge device 910 may send second information to the WTRU 102 including first predicted user input data at a second time instant, wherein the first predicted user input data is generated using a second DNN and the sent neural network data, and wherein the second time instant is associated with a first prediction time period starting at the first time instant. At block 1130, the edge device 910 may receive third information from the WTRU 102 including an indication to trigger error correction of an algorithmic calculation for predicting user input if a first prediction error is above a threshold, wherein the first prediction error is based on the first predicted user input at the second time instant and the user input obtained at the second time instant. At block 1140, the edge device 910 may apply the error correction to the algorithmic calculation for predicting user input.

[0200] In certain representative embodiments, the third information includes the first prediction error, and / or applying an error correction to an algorithmic calculation used to predict the user input is based on the first prediction error.

[0201] In certain representative embodiments, the edge device 910 may send fourth information to the WTRU 102 including a second predicted time period, where the second predicted time period is shorter than the first predicted time period.

[0202] In certain representative embodiments, the edge device 910 may receive fourth information from the WTRU 102 under the condition that the second prediction error is above a threshold, the fourth information including a fourth time instant for retraining the first DNN and the second DNN, e.g., where the second prediction error is determined based on a second predicted user input predicted at a third time instant and additional user input obtained at the third time instant, e.g., where the third time instant is associated with the second prediction time period; the edge device 910 may send a confirmation message to the WTRU 102 to retrain the first DNN and / or the second DNN at the fourth time instant; and the edge device 910 may retrain the second DNN at the fourth time instant.

[0203] In certain representative embodiments, any one of the first information, the second information, the third information, and the fourth information uses SDP.

[0204] In certain representative embodiments, any one of the first information, the second information, the third information, and the fourth information is mapped to an SDP provider capability negotiation parameter.

[0205] In certain representative embodiments, any one of the first information, the second information, the third information, and the fourth information uses a SIP update function.

[0206] Figure 12 is a flow chart illustrating an example procedure 1200 performed by the WTRU 102.

[0207] According to an embodiment, the WTRU 102 may be configured to obtain first user input data at a first time (1210).

[0208] According to an embodiment, the WTRU 102 may be configured to send first information to an edge device, the first information comprising neural network data generated from a first distributed deep neural network (DNN) of the WTRU, wherein the neural network data is generated based on first user input data (1220).

[0209] According to an embodiment, the WTRU 102 may be configured to receive second information from an edge device, the second information comprising first predicted user input data generated from a second DNN of the edge device at a second time, wherein the first predicted user input data is generated based on neural network data, and wherein the second time is associated with a first prediction time period starting from the first time (1230).

[0210] According to an embodiment, the WTRU 102 may be configured to obtain second user input data at a second time (1240).

[0211] According to an embodiment, the WTRU 102 may be configured to determine a first prediction error based on the first predicted user input data at the second time and the second user input data obtained at the second time (1250).

[0212] According to an embodiment, the WTRU 102 may be configured to send third information to the edge device, if the first prediction error is above a threshold, the third information including the first prediction error and / or an indication to trigger an error correction algorithm for predicting user input (1260).

[0213] According to an embodiment, the WTRU 102 may be configured to determine a second prediction error based on second predicted user input data predicted at a third time and a third user input obtained at the third time; and receive fourth information including a second prediction time period from the edge device, wherein the second prediction time period is shorter than the first prediction time period, and wherein the fourth information is received under the condition that the second prediction error is higher than a threshold.

[0214] According to an embodiment, the WTRU 102 may be configured to determine a third prediction error based on third predicted user input data predicted at a fourth time and a fourth user input obtained at the fourth time; send fifth information including a fifth time for retraining the first DNN and / or the second DNN to the edge device under the condition that the third prediction error is higher than a threshold; receive a confirmation message from the edge device for retraining the first DNN and / or the second DNN at the fifth time; and retrain the first DNN and / or the second DNN at the fifth time.

[0215] According to an embodiment, any one of the first information, the second information, the third information, and the fourth information uses a Session Description Protocol (SDP).

[0216] According to an embodiment, any one of the first information, the second information, the third information, and the fourth information is mapped to an SDP provider capability negotiation parameter.

[0217] According to an embodiment, any one of the first information, the second information, the third information, and the fourth information uses a session initiation protocol update function.

[0218] According to an embodiment, the error correction algorithm includes any one of the following: (1) Kalman filtering, (2) partial cube rendering, (3) depth map rendering.

[0219] in conclusion

[0220] Although features and elements are provided above in specific combinations, it will be understood by those skilled in the art that each feature or element can be used alone or in combination with other features and elements. The present disclosure is not limited to the specific embodiments described in this application, which are intended to illustrate various aspects. Many modifications and variations may be made without departing from the spirit and scope of the present invention, as will be apparent to those skilled in the art. Any element, action, or instruction used in the description of this application should not be interpreted as being critical or essential to the present invention unless explicitly provided as such. Based on the foregoing description, in addition to the methods and devices listed here, functionally equivalent methods and devices within the scope of this disclosure will be apparent to those skilled in the art. Such modifications and variations are intended to fall within the scope of the appended claims. The present disclosure is limited only by the terms of the appended claims and the full scope of equivalents to which these claims are assigned. It should be understood that the present disclosure is not limited to a specific method or system.

[0221] For simplicity, the foregoing embodiments are discussed with respect to the terminology and structure of devices having infrared capabilities (i.e., infrared transmitters and receivers). However, the embodiments discussed are not limited to these systems, but may be applied to other systems that use other forms of electromagnetic waves or non-electromagnetic waves (such as sound waves).

[0222] It should also be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, the term "video" or the term "image" may refer to any of a snapshot, a single image, and / or a plurality of images displayed on a time basis. As another example, when referred to herein, the term "user equipment" and its abbreviation "UE", the term "remote" and / or the term "head mounted display" and its abbreviation "HMD" may mean or include (i) a wireless transmit and / or receive unit (WTRU); (ii) any of a number of embodiments of a WTRU; (iii) a device having wireless capabilities and / or wired capabilities (e.g., tetherable) that is configured with some or all of the structure and functionality of a WTRU; (iii) a device having wireless capabilities and / or wired capabilities that is configured with less than all of the structure and functionality of a WTRU; or (iv) the like. Reference herein Figures 1A-1DDetails of an example WTRU are provided, which may represent any WTRU described herein. As another example, various embodiments disclosed above and below are described as utilizing a head-mounted display. Those skilled in the art will recognize that devices other than head-mounted displays may be utilized and that some or all of the present disclosure and various disclosed embodiments may be modified accordingly without undue experimentation. Examples of such other devices may include drones or other devices configured to stream information to provide an adapted reality experience.

[0223] In addition, the methods provided herein can be implemented in a computer program, software, or firmware, which is contained in a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted via a wired or wireless connection) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital versatile disks (DVDs). A processor associated with the software can be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.

[0224] Variations of the methods, devices, and systems provided above are possible without departing from the scope of the present invention. In view of the various embodiments that may be applied, it should be understood that the illustrated embodiments are merely examples and should not be considered as limiting the scope of the appended claims. For example, the embodiments provided herein include handheld devices that may include or be used with any suitable voltage source, such as a battery, that provides any suitable voltage.

[0225] In addition, in the embodiments provided above, processing platforms, computing systems, controllers and other devices including processors are mentioned. These devices may include at least one central processing unit ("CPU") and memory. According to the practice of those skilled in the art of computer programming, reference to the symbolic representation of actions and operations or instructions can be performed by various CPUs and memories. Such actions and operations or instructions can be referred to as being "executed," "computer-executed," or "CPU-executed."

[0226] Those skilled in the art will appreciate that the actions and symbolic representations of operations or instructions comprise the manipulation of electrical signals by the CPU. The electrical system represents data bits, which may result in the ultimate conversion or reduction of the electrical signal and the maintenance of the data bits in memory locations within the memory system, thereby reconfiguring or otherwise changing the operation of the CPU and other processing of the signal. The memory locations where the data bits are maintained are physical locations having specific electrical, magnetic, optical, or organic properties corresponding to or representing the data bits. It should be understood that the embodiments are not limited to the aforementioned platforms or CPUs, and that other platforms and CPUs may support the provided methods.

[0227] The data bits may also be maintained on computer-readable media, including magnetic disks, optical disks, and any other volatile (e.g., random access memory (RAM)) or non-volatile (e.g., read-only memory (ROM)) mass storage systems readable by the CPU. The computer-readable media may include cooperating or interconnected computer-readable media that reside exclusively on a processing system or distributed across multiple interconnected processing systems, which may be local or remote to the processing system. It should be understood that the embodiments are not limited to the aforementioned memories, and that other platforms and memories may support the provided methods.

[0228] In an illustrative embodiment, any operations, processes, etc. described herein may be implemented as computer-readable instructions stored on a computer-readable medium. The computer-readable instructions may be executed by a processor of a mobile unit, a network element, and / or any other computing device.

[0229] There is little distinction between hardware and software implementations of various aspects of the system. The use of hardware or software is typically (but not always, as in some cases the choice between hardware and software may become important) a design choice that represents a cost-efficiency trade-off. There may be various means to implement the processes and / or systems and / or other technologies described herein (e.g., hardware, software, and / or firmware), and the preferred means may vary depending on the environment in which the processes and / or systems and / or other technologies are deployed. For example, if an implementer determines that speed and accuracy are most important, the implementer may choose a tool that is primarily hardware and / or firmware. If flexibility is most important, the implementer may choose a primarily software implementation. Alternatively, the implementer may choose some combination of hardware, software, and / or firmware.

[0230] The foregoing detailed description has described various embodiments of the device and / or process using block diagrams, flow charts, and / or examples. To the extent that such block diagrams, flow charts, and / or examples include one or more functions and / or operations, it will be understood by those skilled in the art that each function and / or operation in such block diagrams, flow charts, or examples can be implemented individually and / or collectively by various hardware, software, firmware, or any combination thereof. In one embodiment, several portions of the subject matter described herein can be implemented by application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), and / or other integrated formats. However, those skilled in the art will recognize that some aspects of the embodiments disclosed herein can be implemented in whole or in part equivalently in an integrated circuit, as one or more computer programs running on one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more processors (e.g., as one or more programs running on one or more microprocessors), as firmware, or as a combination thereof, and that designing circuits and / or writing code for software and / or firmware will be well within the skill of those skilled in the art based on the present disclosure. Furthermore, those skilled in the art will appreciate that the mechanisms of the subject matter described herein may be distributed as a program product in a variety of forms, and that the illustrative embodiments of the subject matter described herein apply regardless of the particular type of signal-bearing medium used to actually perform the distribution. Examples of signal-bearing media include, but are not limited to, the following: recordable-type media such as floppy disks, hard drives, CDs, DVDs, digital tapes, computer memories, and the like, and transmission-type media such as digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, and the like).

[0231] Those skilled in the art will recognize that it is common in the art to describe devices and / or processes in the manner set forth herein and then use engineering practices to integrate such described devices and / or processes into data processing systems. That is, at least a portion of the devices and / or processes described herein can be integrated into a data processing system through a reasonable amount of experimentation. Those skilled in the art will recognize that a typical data processing system can typically include one or more of the following: a system unit housing, a video display device, memory such as volatile and non-volatile memory, a processor such as a microprocessor and a digital signal processor, a computing entity such as an operating system, a driver, a graphical user interface and an application, one or more interactive devices such as a touchpad or screen, and / or a control system including feedback loops and control motors (e.g., feedback for sensing position and / or velocity, control motors for moving and / or adjusting components and / or quantities). A typical data processing system can be implemented using any suitable commercially available components, such as those typically found in data computing / communication and / or network computing / communication systems.

[0232] The subject matter described herein sometimes illustrates different components that are included in or connected to different other components. It should be understood that the architectures described in this way are merely examples, and in fact many other architectures that realize the same function can be implemented. In a conceptual sense, any arrangement of components that realize the same function is effectively "associated" so that the desired function can be realized. Therefore, any two components that are combined to realize a specific function can be considered to be "associated" with each other so that the desired function is realized, without considering architecture or intermediate components. Similarly, any two components that are so associated can also be considered to be "operably connected" or "operably coupled" to each other to realize the desired function, and any two components that can be so associated can also be considered to be "operably coupled" to each other to realize the desired function. Specific examples of operable coupling include, but are not limited to, physically compatible and / or physically interactive components and / or wirelessly interactive and / or wirelessly interactive components and / or logically interactive and / or logically interactive components.

[0233] With respect to the use of substantially any plural and / or singular terms herein, those skilled in the art can translate from the plural to the singular and / or from the singular to the plural as appropriate, depending on the context and / or application. For clarity, various singular / plural arrangements may be expressly set forth herein.

[0234] Those skilled in the art will understand that, in general, the terms used herein, especially in the appended claims (e.g., the bodies of the appended claims), are generally intended as "open" terms (e.g., the term "including" should be interpreted as "including but not limited to," the term "having" should be interpreted as "having at least," the term "comprising" should be interpreted as "including but not limited to," etc.). Those skilled in the art will further understand that if a specific number of a claim recitation is intended, such intent will be expressly recited in the claim, and if no such recitation is made, no such intent is present. For example, where only one item is intended, the term "single" or similar language may be used. To aid understanding, the claims appended below and / or the description herein may include the use of the introductory phrases "at least one" and "one or more" to introduce claim recitations. However, the use of such phrases should not be interpreted as implying that the introduction of a claim recitation by the indefinite article "a" or "an" limits any particular claim including such introduced claim recitation to including only one embodiment of such recitation, even when the same claim includes the introductory phrases "one or more" or "at least one" and an indefinite article such as "a" or "an" (e.g., "a" and / or "an" should be interpreted as meaning "at least one" or "one or more"). The same applies to the use of definite articles to introduce claim recitations. Furthermore, even if a specific number of an introduced claim recitation is explicitly recited, those skilled in the art will recognize that such recitation should be interpreted to mean at least the recited number (e.g., the simple recitation of "two recitations" without other modifiers means at least two recitations, or two or more recitations). In addition, in those cases where there is a convention similar to “at least one of A, B, and C, etc.”, generally speaking, such construction is intended to allow those skilled in the art to understand the convention (e.g., “a system having at least one of A, B, and C” would include but is not limited to systems having A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together, etc.). In those cases where there is a convention similar to “at least one of A, B, or C, etc.”, generally speaking, such construction is intended to allow those skilled in the art to understand the convention (e.g., “a system having at least one of A, B, or C” would include but is not limited to systems having A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together, etc.). Those skilled in the art will further understand that any disjunctive words and / or phrases that actually represent two or more optional terms, whether in the specification, claims, or drawings, should be understood to include the possibility of one, either, or both of the terms. For example, the phrase “A or B” would be understood to include the possibility of “A” or “B” or “A and B.”Furthermore, as used herein, the term "any" followed by a listing of multiple items and / or multiple categories of items is intended to include "any one," "any combination," "any plurality," and / or "any combination of a plurality" of the items and / or categories of items, either individually or in combination with other items and / or other categories of items. Furthermore, as used herein, the term "set" is intended to include any number of items, including zero. Furthermore, as used herein, the term "number" is intended to include any number, including zero. And the term "plurality," as used herein, is intended to be synonymous with "multiple."

[0235] In addition, where features or aspects of the disclosure are described in terms of Markush groups, those skilled in the art will recognize that the disclosure is also thereby described in terms of any individual member or subgroup of members of the Markush group.

[0236] As will be understood by those skilled in the art, for any and all purposes, such as for providing a written description, all ranges disclosed herein also include any and all possible subranges and combinations of subranges thereof. Any listed range can be readily considered to fully describe and be capable of dividing the same range into at least equal halves, thirds, quarters, fifths, tenths, etc. As non-limiting examples, each range discussed herein can be readily decomposed into a lower third, middle third, and upper third, etc. Those skilled in the art will also understand that all language, such as "up to," "at least," "greater than," "less than," etc., includes the recited numbers and refers to ranges that can subsequently be decomposed into subranges as described above. Finally, as will be understood by those skilled in the art, a range includes each individual member. Thus, for example, a group having 1-3 cells refers to a group having 1, 2, or 3 cells. Similarly, a group having 1-5 cells refers to a group having 1, 2, 3, 4, or 5 cells, and so on.

[0237] Furthermore, the claims should not be read as limited to the order or elements provided unless so stated. Furthermore, the use of the term "means for..." in any claim is intended to refer to or means-plus-function claim format, and any claim without the term “means for…” is not so.

Claims

1. A method implemented by a wireless transmit / receive unit (WTRU), the method comprising: Obtaining first user input data at the first time; sending first information to an edge device, the first information comprising neural network data generated from a first distributed deep neural network (DNN) of the WTRU, wherein the neural network data is generated based on the first user input data; receiving second information from the edge device, the second information comprising first predicted user input data generated from a second DNN of the edge device at a second time, wherein the first predicted user input data is generated based on the neural network data, and wherein the second time is associated with a first prediction time period starting from the first time; obtaining second user input data at the second time; determining a first prediction error based on the first predicted user input data at the second time and second user input data obtained at the second time; and Under the condition that the first prediction error is higher than a threshold, third information is sent to the edge device, where the third information includes the first prediction error and / or an indication for triggering an error correction algorithm for predicting user input.

2. The method according to claim 1, comprising: determining a second prediction error based on second predicted user input data predicted at a third time and a third user input obtained at the third time; as well as Fourth information is received from the edge device, the fourth information comprising a second prediction time period, wherein the second prediction time period is shorter than the first prediction time period, and wherein the fourth information is received under the condition that the second prediction error is higher than the threshold.

3. The method according to any one of claims 1 to 2, further comprising: determining a third prediction error based on third predicted user input data predicted at a fourth time and a fourth user input obtained at the fourth time; Under the condition that the third prediction error is higher than the threshold, sending fifth information to the edge device, wherein the fifth information includes a fifth time for retraining the first DNN and / or the second DNN; receiving a confirmation message from the edge device for retraining the first DNN and / or the second DNN at the fifth time; as well as Retrain the first DNN and / or the second DNN at the fifth time.

4. The method according to claim 3, wherein: Any one of the first information, the second information, the third information, and the fourth information uses a Session Description Protocol (SDP).

5. The method according to claim 4, wherein: Any one of the first information, the second information, the third information, and the fourth information is mapped to an SDP provider capability negotiation parameter.

6. The method according to claim 3, wherein: Any one of the first information, the second information, the third information, and the fourth information uses a session initiation protocol update function.

7. The method according to any one of claims 1 to 6, wherein: The error correction algorithm includes any one of the following: (1) Kalman filtering, (2) partial cube rendering, (3) depth map rendering.

8. A wireless transmit / receive unit (WTRU), comprising a processor, a transmitter, a receiver, and a memory, the WTRU being configured to: Obtaining first user input data at the first time; sending first information to an edge device, the first information comprising neural network data generated from a first distributed deep neural network (DNN) of the WTRU, wherein: The neural network data is generated based on the first user input data; receiving second information from the edge device, the second information comprising first predicted user input data generated from a second DNN of the edge device at a second time, wherein the first predicted user input data is generated based on the neural network data, and wherein the second time is associated with a first prediction time period starting from the first time; obtaining second user input data at the second time; determining a first prediction error based on the first predicted user input data at the second time and second user input data obtained at the second time; and Under the condition that the first prediction error is higher than a threshold, third information is sent to the edge device, where the third information includes the first prediction error and / or an indication for triggering an error correction algorithm for predicting user input.

9. The WTRU of claim 8, configured to: determining a second prediction error based on second predicted user input data predicted at a third time and a third user input obtained at the third time; and receiving fourth information from the edge device, the fourth information including a second prediction time period, wherein: The second prediction period is shorter than the first prediction period, and wherein the fourth information is received under the condition that the second prediction error is higher than the threshold.

10. The WTRU according to any one of claims 8-9, configured to: determining a third prediction error based on third predicted user input data predicted at a fourth time and a fourth user input obtained at the fourth time; Under the condition that the third prediction error is higher than the threshold, sending fifth information to the edge device, wherein the fifth information includes a fifth time for retraining the first DNN and / or the second DNN; receiving a confirmation message from the edge device for retraining the first DNN and / or the second DNN at the fifth time; as well as Retrain the first DNN and / or the second DNN at the fifth time.

11. The WTRU of claim 10 wherein: Any one of the first information, the second information, the third information, and the fourth information uses a Session Description Protocol (SDP).

12. The WTRU of claim 11 wherein: Any one of the first information, the second information, the third information, and the fourth information is mapped to an SDP provider capability negotiation parameter.

13. The WTRU of claim 10 wherein: Any one of the first information, the second information, the third information, and the fourth information uses a session initiation protocol update function.

14. The WTRU of any one of claims 9 to 13, wherein: The error correction algorithm includes any one of the following: (1) Kalman filtering, (2) partial cube rendering, (3) depth map rendering.

Images