Methods, architectures, apparatuses, and systems relating to cross-user AR processing and resource sharing for enabling dynamic and customized AR experiences

Abstract

Processes, methods, architectures, apparatuses, systems, devices, and computer program products are described relating to cross-user augmented reality (AR) processing and resource sharing for enabling dynamic and customized AR experiences. A first wireless transmit / receive unit (WTRU) may be configured to send an AR resource request to a network element, the AR resource request indicating: (1) a requested type of AR resource, and (2) positioning information associated with the first WTRU. The first WTRU may be further configured to receive an AR resource response from the network element, the AR resource response including first information for communicating with a second WTRU capable of providing the AR resource based on the requested type of AR resource and the positioning information. The first WTRU may be further configured to transmit an AR resource sharing request to the second WTRU, the AR resource sharing request indicating at least one of the requested types of AR resources.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. patent application No. 63 / 444,782, filed February 10, 2023, which is incorporated herein by reference in its entirety. Technical Field

[0003] The present disclosure relates generally to the fields of communications, software, and coding, including, for example, methods, architectures, apparatuses, and systems for cross-user augmented reality (AR) processing and resource sharing for enabling dynamic and customized augmented reality (AR) experiences. Background Art

[0004] Augmented reality (AR) can be viewed as integrating the virtual world with the physical world by applying (e.g., overlaying) virtual information (e.g., assets, scenes) to the physical world. In AR applications, the real world and virtual objects can coexist in the same viewport of the user, thereby enhancing the user's real sensory experience. The embodiments described herein have been designed with the foregoing in mind. Summary of the Invention

[0005] Briefly, according to one embodiment, a first wireless transmit / receive unit (WTRU) is described herein that includes a processor and a transceiver operably coupled to the processor. In an example, the processor and transceiver may be configured to: send an AR resource request to a network element, the AR resource request indicating: (1) an AR resource of the requested type, and (2) positioning information associated with the first WTRU (e.g., and a physical environment). In an example, the processor and transceiver may be further configured to receive an AR resource response from the network element, the AR resource response including first information for communicating with a second WTRU that is capable of providing (e.g., based on) one or more AR resources of the requested type (e.g., and positioning information). In an example, the processor and transceiver may be further configured to send an AR resource sharing request to the second WTRU, the AR resource sharing request indicating at least one of the AR resource of the requested type and the positioning information. In an example, the processor and transceiver may be further configured to receive an AR resource sharing response from the second WTRU indicating an agreement to provide the AR resource of the requested type.

[0006] According to one embodiment, a second WTRU is described herein that includes a processor and a transceiver operably coupled to the processor. In an example, the processor and the transceiver may be configured to send AR capability information to a network element, the AR capability information indicating: (1) at least one type of AR resource that the second WTRU may be able to provide, and (2) positioning information associated with the second WTRU (e.g., in a physical environment). In an example, the processor and the transceiver may be configured to receive an AR resource sharing request from a first WTRU indicating at least one type of AR resource that the second WTRU may be able to provide. In an example, the processor and the transceiver may be configured to determine to provide one or more AR resources of at least one type of AR resource (e.g., associated with a physical environment) based on the availability of processing resources in the second WTRU. In an example, the processor and the transceiver may be configured to send an AR resource sharing response to the first WTRU indicating an agreement to provide at least one type of AR resource. According to one embodiment, a first method and a second method are disclosed herein, wherein the first method and the second method may include steps performed by the first WTRU and the second WTRU, respectively, as described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] A more detailed understanding can be obtained from the following detailed description given by way of example in conjunction with the accompanying drawings. As with the detailed description, the figures in such drawings are examples. Therefore, the drawings (figures) and detailed description should not be considered limiting, and other equally effective examples are possible and feasible. Moreover, like reference numerals ("ref.") in the figures indicate similar elements, and wherein:

[0008] Figure 1A is a system diagram illustrating an example communication system;

[0009] Figure 1B It is a diagram that can be Figure 1A A system diagram of an example wireless transmit / receive unit (WTRU) for use within a communications system is shown in FIG.

[0010] Figure 1C It is a diagram that can be Figure 1A A system diagram of an example radio access network (RAN) and an example core network (CN) used within the communication system illustrated in FIG.

[0011] Figure 1D It is a diagram that can be Figure 1A A system diagram of a further example RAN and a further example CN for use within the communication system illustrated in FIG.

[0012] Figure 2 is a diagram illustrating an example of a dynamic and customized AR experience for exploring new city streets;

[0013] Figure 3 is a diagram illustrating an example of cross-user collaborative AR resource sharing;

[0014] Figure 4 is a diagram illustrating an example of an AR device architecture for enabling cross-user collaborative AR processing and resource sharing;

[0015] Figure 5 is a diagram illustrating four examples of AR device functional architecture of an AR resource consumer (ARRC);

[0016] Figure 6 is a diagram illustrating three examples of AR device functional architecture of an AR resource provider (ARRP);

[0017] Figure 7A is a diagram illustrating an example process for enabling establishment of cross-user collaborative relationships;

[0018] Figure 7B is a diagram illustrating another example process for enabling establishment of cross-user collaborative relationships;

[0019] Figure 8 is a diagram illustrating an example process for enabling sharing of raw video frames;

[0020] Figure 9 is a diagram illustrating an example process for enabling physical world knowledge sharing for physical object detection and recognition;

[0021] Figure 10 is a diagram illustrating an example process for enabling physical world knowledge accuracy and freshness validation;

[0022] Figure 11 is a diagram illustrating an example process for enabling pre-notification associated with a mobile PO appearance;

[0023] Figure 12 is a diagram illustrating an example 3GPP SA4 implementation of an ARD architecture for supporting cross-user AR processing and resource sharing;

[0024] Figure 13 is a diagram illustrating an example 3GPP SA2 implementation for cross-user collaborative relationship establishment;

[0025] Figure 14 is a diagram illustrating an example 3GPP SA6 implementation;

[0026] Figure 15 is a diagram illustrating an example ETSI Augmented Reality Framework (ARF) implementation;

[0027] Figure 16 is a diagram illustrating an example method for enabling establishment of cross-user collaborative relationships; and

[0028] Figure 17 is a diagram illustrating another example method for enabling cross-user collaborative relationship establishment. DETAILED DESCRIPTION

[0029] In the following detailed description, many specific details are set forth to provide a thorough understanding of the embodiments and / or examples disclosed herein. However, it will be understood that such embodiments and examples can be put into practice without some or all of the specific details set forth herein. In other cases, well-known methods, processes, components, and circuits have not yet been described in detail to avoid blurring the following description. In addition, the embodiments and examples not specifically described herein can replace the embodiments and other examples that are explicitly, implicitly, and / or inherently described, disclosed, or otherwise provided (collectively referred to as "providing") herein to practice, or to practice in combination with the embodiments and other examples. Although various embodiments are described and / or claimed herein, wherein devices, systems, equipment, etc. and / or any of its elements perform operations, processes, algorithms, functions, etc. and / or any part thereof, it will be understood that any embodiment described and / or claimed herein assumes that any device, system, equipment, etc. and / or any of its elements are configured to perform any operations, processes, algorithms, functions, etc. and / or any part thereof.

[0030] Example Communication System

[0031] The methods, apparatus, and systems provided herein are well-suited for communications involving both wired and wireless networks. Figures 1A-1D An overview of various types of wireless devices and infrastructure is provided, wherein various elements of the network can utilize, perform, be arranged according to, and / or be adapted and / or configured for the methods, apparatus, and systems provided herein.

[0032] Figure 1Ais a system diagram illustrating an example communication system 100 in which one or more disclosed embodiments may be implemented. The communication system 100 may be a multiple access system that provides content, such as voice, data, video, messaging, broadcast, etc., to multiple wireless users. The communication system 100 may enable multiple wireless users to access such content through the sharing of system resources, including wireless bandwidth. For example, the communication system 100 may employ one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single carrier FDMA (SC-FDMA), zero tail (ZT) unique word (UW) discrete Fourier transform (DFT) spread OFDM (ZT UW DTS-sOFDM), unique word OFDM (UW-OFDM), resource block filtered OFDM, filter bank multi-carrier (FBMC), and the like.

[0033] like Figure 1A As shown in FIG, the communication system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, a radio access network (RAN) 104 / 113, a core network (CN) 106 / 115, a public switched telephone network (PSTN) 108, the Internet 110, and other networks 112. However, it will be understood that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Each of the WTRUs 102a, 102b, 102c, 102d may be any type of device configured to operate and / or communicate in a wireless environment. By way of example, the WTRUs 102a, 102b, 102c, 102d (any of which may be referred to as a "station" and / or "STA") may be configured to transmit and / or receive wireless signals and may include (or be) a user equipment (UE), a mobile station, a fixed or mobile subscriber unit, a subscription-based unit, a pager, a cellular phone, a personal digital assistant (PDA), a smartphone, a laptop, a netbook, a personal computer, a wireless sensor, a hotspot or Mi-Fi device, an Internet of Things (IoT) device, a watch or other wearable device, a head-mounted display (HMD), a vehicle, a drone, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in the context of an industrial and / or automated process chain), a consumer electronic device, a device operating on a commercial and / or industrial wireless network, and the like. Any of the WTRUs 102a, 102b, 102c, and 102d may be interchangeably referred to as a UE.

[0034] The communication system 100 may also include a base station 114a and / or a base station 114b. Each of the base stations 114a, 114b may be any type of device configured to wirelessly interface with at least one of the WTRUs 102a, 102b, 102c, 102d to, for example, facilitate access to one or more communication networks, such as the CN 106 / 115, the Internet 110, and / or the network 112. By way of example, the base stations 114a, 114b may be any of the following: a base transceiver station (BTS), a Node-B (NB), an eNode-B (eNB), a Home Node-B (HNB), a Home eNode-B (HeNB), a gNode-B (gNB), an NR Node-B (NR NB), a site controller, an access point (AP), a wireless router, and the like. While the base stations 114a, 114b are each depicted as a single element, it will be appreciated that the base stations 114a, 114b may include any number of interconnected base stations and / or network elements.

[0035] Base station 114a may be part of RAN 104 / 113, which may also include other base stations and / or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), relay nodes, etc. Base station 114a and / or base station 114b may be configured to transmit and / or receive wireless signals on one or more carrier frequencies, which may be referred to as cells (not shown). These frequencies may be in licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum. A cell may provide wireless service coverage to a specific geographic area, which may be relatively fixed or may change over time. A cell may be further divided into cell sectors. For example, the cell associated with base station 114a may be divided into three sectors. Thus, in an embodiment, base station 114a may include three transceivers, one for each sector of the cell. In an embodiment, base station 114a may employ multiple-input multiple-output (MIMO) technology and may utilize multiple transceivers for each or any sector of the cell. For example, beamforming may be used to transmit and / or receive signals in a desired spatial direction.

[0036] The base stations 114a, 114b may communicate with one or more of the WTRUs 102a, 102b, 102c, 102d over an air interface 116, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 116 may be established using any suitable radio access technology (RAT).

[0037] More specifically, as noted above, the communication system 100 may be a multiple access system and may employ one or more channel access schemes such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, and the like. For example, the base station 114a in the RAN 104 / 113 and the WTRUs 102a, 102b, 102c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may use Wideband CDMA (WCDMA) to establish the air interface 116. WCDMA may include communication protocols such as High Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA may include High Speed ​​Downlink Packet Access (HSDPA) and / or High Speed ​​Uplink Packet Access (HSUPA).

[0038] In an embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface 116 using Long Term Evolution (LTE) and / or LTE-Advanced (LTE-A) and / or LTE-Advanced Pro (LTE-A Pro).

[0039] In an embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology, such as NR radio access, that may establish the air interface 116 using New Radio (NR).

[0040] In an embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement multiple radio access technologies. For example, the base station 114a and the WTRUs 102a, 102b, 102c may implement both LTE radio access and NR radio access, for example, using dual connectivity (DC) principles. Thus, the air interface utilized by the WTRUs 102a, 102b, 102c may be characterized by multiple types of radio access technologies and / or transmissions sent to / from multiple types of base stations (e.g., eNBs and gNBs).

[0041] In an embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as IEEE 802.11 (i.e., Wireless Fidelity (Wi-Fi)), IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), GSM EDGE (GERAN), and the like.

[0042] Figure 1A The base station 114b in the may be, for example, a wireless router, a home Node-B, a home eNode-B, or an access point, and may utilize any suitable RAT to facilitate wireless connectivity in a local area, such as a business, a home, a vehicle, a campus, an industrial facility, an air corridor (e.g., for use by drones), a road, and the like. In an embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.11 to establish a wireless local area network (WLAN). In an embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.15 to establish a wireless personal area network (WPAN). In an embodiment, the base station 114b and the WTRUs 102c, 102d may utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR, etc.) to establish any of a small cell, a pico cell, or a femto cell. As Figure 1A As shown in FIG, base station 114b may have a direct connection to the Internet 110. Thus, base station 114b may not need to access the Internet 110 via CN 106 / 115.

[0043] The RAN 104 / 113 may be in communication with the CN 106 / 115, which may be any type of network configured to provide voice, data, applications, and / or Voice over Internet Protocol (VoIP) services to one or more of the WTRUs 102a, 102b, 102c, 102d. The data may have different quality of service (QoS) requirements, such as different throughput requirements, delay requirements, error tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, and the like. The CN 106 / 115 may provide call control, billing services, mobile location-based services, prepaid calling, Internet connectivity, video distribution, etc., and / or perform advanced security functions, such as user authentication. Although not described in Figure 1A Although not shown in the figures, it will be appreciated that the RAN 104 / 113 and / or the CN 106 / 115 may be in direct or indirect communication with other RANs that employ the same RAT as the RAN 104 / 113 or a different RAT. For example, in addition to being connected to the RAN 104 / 113, which may utilize NR radio technology, the CN 106 / 115 may also be in communication with another RAN (not shown) that employs any of GSM, UMTS, CDMA 2000, WiMAX, E-UTRA, or Wi-Fi radio technologies.

[0044] The CN 106 / 115 may also serve as a gateway for the WTRUs 102a, 102b, 102c, 102d to access the PSTN 108, the Internet 110, and / or other networks 112. The PSTN 108 may include a circuit-switched telephone network that provides plain old telephone service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices that use common communication protocols, such as the Transmission Control Protocol (TCP), the User Datagram Protocol (UDP), and / or the Internet Protocol (IP) from the TCP / IP internet protocol suite. The networks 112 may include wired and / or wireless communication networks owned and / or operated by other service providers. For example, the networks 112 may include another CN connected to one or more RANs, which may employ the same RAT as the RAN 104 / 114 or a different RAT.

[0045] Some or all of the WTRUs 102a, 102b, 102c, 102d in the communication system 100 may include multi-mode capabilities (e.g., the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers for communicating with different wireless networks via different wireless links). Figure 1A The WTRU 102c shown in FIG. 1 may be configured to communicate with the base station 114a, which may employ a cellular-based radio technology, and with the base station 114b, which may employ an IEEE 802 radio technology.

[0046] Figure 1B is a system diagram illustrating an example WTRU 102. Figure 1BAs shown in FIG, the WTRU 102 may include, among other things, a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a keypad 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power supply 134, a global positioning system (GPS) chipset 136, and / or other elements / peripherals 138. It will be appreciated that the WTRU 102 may include any subcombination of the foregoing elements while remaining consistent with an embodiment.

[0047] The processor 118 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, and the like. The processor 118 may perform signal coding, data processing, power control, input / output processing, and / or any other functionality that enables the WTRU 102 to operate in a wireless environment. The processor 118 may be coupled to the transceiver 120, which may be coupled to the transmit / receive element 122. Although Figure 1B The processor 118 and the transceiver 120 are depicted as separate components, but it will be understood that the processor 118 and the transceiver 120 may be integrated together, for example, in an electronic package or chip.

[0048] The transmit / receive element 122 may be configured to transmit signals to or receive signals from a base station (e.g., base station 114a) over the air interface 116. For example, in an embodiment, the transmit / receive element 122 may be an antenna configured to transmit and / or receive RF signals. In an embodiment, the transmit / receive element 122 may be an emitter / detector configured to transmit and / or receive, for example, IR, UV, or visible light signals. In an embodiment, the transmit / receive element 122 may be configured to transmit and / or receive both RF and optical signals. It will be understood that the transmit / receive element 122 may be configured to transmit and / or receive any combination of wireless signals.

[0049] Although the transmit / receive element 122 Figure 1B Although depicted as a single element in FIG. 1 , the WTRU 102 may include any number of transmit / receive elements 122. For example, the WTRU 102 may employ MIMO technology. Thus, in an embodiment, the WTRU 102 may include two or more transmit / receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 116.

[0050] The transceiver 120 may be configured to modulate signals to be transmitted by the transmit / receive element 122 and demodulate signals received by the transmit / receive element 122. As noted above, the WTRU 102 may have multi-mode capabilities. Thus, the transceiver 120 may include multiple transceivers to, for example, enable the WTRU 102 to communicate via multiple RATs, such as NR and IEEE 802.11.

[0051] The processor 118 of the WTRU 102 may be coupled to a speaker / microphone 124, a keypad 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) display unit or an organic light emitting diode (OLED) display unit) and may receive user input data from the speaker / microphone 124, the keypad 126, and / or the display / touchpad 128 (e.g., a liquid crystal display (LCD) display unit or an organic light emitting diode (OLED) display unit). The processor 118 may also output user data to the speaker / microphone 124, the keypad 126, and / or the display / touchpad 128. Furthermore, the processor 118 may access information from and store data in any type of suitable memory, such as non-removable memory 130 and / or removable memory 132. The non-removable memory 130 may include random access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. In other embodiments, the processor 118 may access information from, and store data in, memory that is not physically located on the WTRU 102, such as on a server or a home computer (not shown).

[0052] The processor 118 may receive power from the power source 134 and may be configured to distribute and / or control power to the other components in the WTRU 102. The power source 134 may be any suitable device for powering the WTRU 102. For example, the power source 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel-metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, and the like.

[0053] The processor 118 may also be coupled to the GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 102. In addition to or in lieu of the information from the GPS chipset 136, the WTRU 102 may receive location information from a base station (e.g., base stations 114a, 114b) over the air interface 116 and / or determine its location based on the timing of signals received from two or more nearby base stations. It will be appreciated that the WTRU 102 may acquire location information by any suitable location-determination method while remaining consistent with an embodiment.

[0054] The processor 118 may be further coupled to other components / peripherals 138, which may include one or more software and / or hardware modules / units that provide additional features, functionality, and / or wired or wireless connectivity. For example, the components / peripherals 138 may include an accelerometer, an electronic compass, a satellite transceiver, a digital camera (e.g., for photos and / or videos), a universal serial bus (USB) port, a vibration device, a television transceiver, a hands-free headset, Modules, frequency modulation (FM) radio units, digital music players, media players, video game player modules, internet browsers, virtual reality and / or augmented reality (VR / AR) devices, activity trackers, and the like. Components / peripherals 138 may include one or more sensors. The sensor may be one or more of: a gyroscope, an accelerometer, a Hall effect sensor, a magnetometer, an orientation sensor, a proximity sensor, a temperature sensor, a time sensor; a geolocation sensor; an altimeter, a light sensor, a touch sensor, a magnetometer, a barometer, a gesture sensor, a biometric sensor, and / or a humidity sensor.

[0055] The WTRU 102 may include a full-duplex radio for which transmission and reception of some or all signals (e.g., associated with specific subframes for both uplink (e.g., for transmission) and downlink (e.g., for reception)) may be concurrent and / or simultaneous. The full-duplex radio may include an interference management unit for reducing and / or substantially eliminating self-interference via hardware (e.g., a choke) or via signal processing performed by a processor (e.g., a separate processor (not shown) or via the processor 118). In an embodiment, the WTRU 102 may include a half-duplex radio for which transmission and reception of some or all signals (e.g., associated with specific subframes for both uplink (e.g., for transmission) or downlink (e.g., for reception)) may be concurrent and / or simultaneous.

[0056] Figure 1C1 is a system diagram illustrating the RAN 104 and the CN 106 in accordance with an embodiment. As noted above, the RAN 104 may employ an E-UTRA radio technology to communicate with the WTRUs 102a, 102b, and 102c over the air interface 116. The RAN 104 may also be in communication with the CN 106.

[0057] The RAN 104 may include eNode-Bs 160a, 160b, 160c, though it will be appreciated that the RAN 104 may include any number of eNode-Bs while remaining consistent with an embodiment. The eNode-Bs 160a, 160b, 160c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116. In an embodiment, the eNode-Bs 160a, 160b, 160c may implement MIMO technology. Thus, the eNode-B 160a, for example, may use multiple antennas to transmit wireless signals to, and receive wireless signals from, the WTRU 102a.

[0058] Each of the eNode-Bs 160a, 160b, and 160c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the uplink (UL) and / or downlink (DL), and the like. Figure 1C As shown in FIG, eNode-Bs 160a, 160b, 160c may communicate with each other via an X2 interface.

[0059] Figure 1C The CN 106 shown in FIG may include a mobility management entity (MME) 162, a serving gateway (SGW) 164, and a packet data network (PDN) gateway (PGW) 166. While each of the foregoing elements is depicted as part of the CN 106, it will be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0060] The MME 162 may be connected to each of the eNode-Bs 160a, 160b, and 160c in the RAN 104 via an S1 interface and may serve as a control node. For example, the MME 162 may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, bearer activation / deactivation, selecting a particular serving gateway during an initial attach of the WTRUs 102a, 102b, 102c, and the like. The MME 162 may also provide a control plane function for switching between the RAN 104 and other RANs (not shown) that employ other radio technologies, such as GSM and / or WCDMA.

[0061] The SGW 164 may be connected to each of the eNode-Bs 160a, 160b, 160c in the RAN 104 via an S1 interface. The SGW 164 may generally route and forward user data packets to and from the WTRUs 102a, 102b, 102c. The SGW 164 may perform other functions such as anchoring the user plane during inter-eNode-B handovers, triggering paging when downlink data is available for the WTRUs 102a, 102b, 102c, managing and storing the context of the WTRUs 102a, 102b, 102c, and the like.

[0062] The SGW 164 may be connected to the PGW 166, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices.

[0063] The CN 106 may facilitate communications with other networks. For example, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices. For example, the CN 106 may include, or may be in communication with, an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that serves as an interface between the CN 106 and the PSTN 108. In addition, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers.

[0064] Even though the WTRU Figures 1A-1D Although described as a wireless terminal, it is contemplated that in certain representative embodiments, such a terminal may (eg, temporarily or permanently) employ a wired communication interface with a communication network.

[0065] In a representative embodiment, the other network 112 may be a WLAN.

[0066] A WLAN in infrastructure basic service set (BSS) mode may have an access point (AP) for the BSS and one or more stations (STAs) associated with the AP. The AP may have access to or an interface with a distribution system (DS) or another type of wired / wireless network that carries traffic into and / or out of the BSS. Traffic originating from outside the BSS and destined for a STA may reach the AP and be delivered to the STA. Traffic originating from a STA destined for a destination outside the BSS may be sent to the AP for delivery to the destination. Traffic between STAs within a BSS may be sent through the AP, for example, where a source STA may send traffic to the AP, and the AP may deliver the traffic to the destination STA. Traffic between STAs within a BSS may be considered and / or referred to as point-to-point traffic. Point-to-point traffic may be sent between a source and destination STA (e.g., directly between them) using direct link setup (DLS). In certain representative embodiments, the DLS may use 802.11e DLS or 802.11z tunneled DLS (TDLS). A WLAN using an independent BSS (IBSS) mode may not have an AP, and STAs (eg, all STAs) within or using the IBSS may communicate directly with each other. The IBSS communication mode may sometimes be referred to herein as an "ad-hoc" communication mode.

[0067] When using 802.11ac infrastructure operation mode or a similar operation mode, the AP can transmit beacons on a fixed channel (such as a primary channel). The primary channel can be a fixed width (e.g., a 20 MHz wide bandwidth) or a width dynamically set via signaling. The primary channel can be the operating channel of the BSS and can be used by STAs to establish a connection with the AP. In certain representative embodiments, for example, carrier sense multiple access with collision avoidance (CSMA / CA) can be implemented in an 802.11 system. For CSMA / CA, STAs (e.g., each STA) including the AP can sense the primary channel. If the primary channel is sensed / detected and / or determined to be busy by a specific STA, the specific STA can back off. One STA (e.g., only one station) can transmit at any given time in a given BSS.

[0068] High throughput (HT) STAs may communicate using a 40 MHz wide channel, for example, via a combination of a primary 20 MHz channel and adjacent or non-adjacent 20 MHz channels to form a 40 MHz wide channel.

[0069] Very high throughput (VHT) STAs can support 20 MHz, 40 MHz, 80 MHz, and / or 160 MHz wide channels. 40 MHz and / or 80 MHz channels can be formed by combining consecutive 20 MHz channels. A 160 MHz channel can be formed by combining eight consecutive 20 MHz channels, or by combining two non-contiguous 80 MHz channels (which can be referred to as an 80+80 configuration). For the 80+80 configuration, after channel coding, the data can pass through a segment parser that can separate the data into two streams. Each stream can be separably subjected to inverse fast Fourier transform (IFFT) processing and time domain processing. The streams can be mapped onto two 80 MHz channels, and the data can be transmitted by the transmitting STA. At the receiver of the receiving STA, the above operations for the 80+80 configuration can be reversed, and the combined data can be sent to the media access control (MAC) layer, entity, etc.

[0070] 802.11af and 802.11ah support operating modes below 1 GHz. The channel operating bandwidth and carrier are reduced in 802.11af and 802.11ah relative to those used in 802.11n and 802.11ac. 802.11af supports 5 MHz, 10 MHz, and 20 MHz bandwidths in the TV White Space (TVWS) spectrum, and 802.11ah supports 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz bandwidths using non-TVWS spectrum. According to a representative embodiment, 802.11ah can support meter type control / machine type communication (MTC), such as MTC devices in macro coverage areas. MTC devices may have certain capabilities, such as limited capabilities, including support for (e.g., only support for) certain bandwidths and / or limited bandwidths. MTC devices may include batteries with battery life above a threshold (e.g., for maintaining very long battery life).

[0071] WLAN systems that can support multiple channels and channel bandwidths such as 802.11n, 802.11ac, 802.11af, and 802.11ah include channels that can be designated as primary channels. The primary channel can have a bandwidth equal to the maximum common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel can be set and / or limited by a STA (that supports the minimum bandwidth operating mode) from among all STAs operating in the BSS. In the example of 802.11ah, for a STA (e.g., an MTC-type device) that supports (e.g., only supports) 1 MHz mode, the primary channel can be 1 MHz wide, even if the AP and other STAs in the BSS support 2 MHz, 4 MHz, 8 MHz, 16 MHz, and / or other channel bandwidth operating modes. Carrier sensing and / or network allocation vector (NAV) settings can depend on the status of the primary channel. If the primary channel is busy, for example, because a STA (that only supports 1 MHz operating mode) is transmitting to the AP, the entire available frequency band can be considered busy even if most of the frequency band remains idle and may be available.

[0072] In the United States, the available frequency band for 802.11ah is from 902 MHz to 928 MHz. In South Korea, the available frequency band is from 917.5 MHz to 923.5 MHz. In Japan, the available frequency band is from 916.5 MHz to 927.5 MHz. Depending on the country code, the total bandwidth available for 802.11ah ranges from 6 MHz to 26 MHz.

[0073] Figure 1D 1 is a system diagram illustrating the RAN 113 and the CN 115 according to an embodiment. As noted above, the RAN 113 may employ NR radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 116. The RAN 113 may also be in communication with the CN 115.

[0074] The RAN 113 may include gNBs 180a, 180b, and 180c, though it will be appreciated that the RAN 113 may include any number of gNBs while remaining consistent with an embodiment. The gNBs 180a, 180b, and 180c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, and 102c over the air interface 116. In an embodiment, the gNBs 180a, 180b, and 180c may implement MIMO technology. For example, the gNBs 180a and 180b may utilize beamforming to transmit signals to and / or receive signals from the WTRUs 102a, 102b, and 102c. Thus, the gNB 180a may, for example, use multiple antennas to transmit wireless signals to and / or receive wireless signals from the WTRU 102a. In an embodiment, the gNBs 180a, 180b, and 180c may implement carrier aggregation technology. For example, the gNB 180a may transmit multiple component carriers (not shown) to the WTRU 102a. A subset of these component carriers may be on unlicensed spectrum, while the remaining component carriers may be on licensed spectrum. In an embodiment, the gNBs 180a, 180b, and 180c may implement coordinated multi-point (CoMP) technology. For example, the WTRU 102a may receive coordinated transmissions from gNB 180a and gNB 180b (and / or gNB 180c).

[0075] The WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using transmissions associated with scalable numerologies. For example, OFDM symbol spacing and / or OFDM subcarrier spacing may vary for different transmissions, different cells, and / or different portions of the wireless transmission spectrum. The WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using subframes or transmission time intervals (TTIs) of varying or scalable lengths (e.g., including varying numbers of OFDM symbols and / or continuously varying absolute time lengths).

[0076] The gNBs 180a, 180b, 180c may be configured to communicate with the WTRUs 102a, 102b, 102c in a standalone configuration and / or a non-standalone configuration. In a standalone configuration, the WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c while not accessing other RANs (e.g., such as the eNode-Bs 160a, 160b, 160c). In a standalone configuration, the WTRUs 102a, 102b, 102c may use one or more of the gNBs 180a, 180b, 180c as mobility anchor points. In a standalone configuration, the WTRUs 102a, 102b, 102c may communicate with the gNBs 180a, 180b, 180c using signals in an unlicensed band. In a non-standalone configuration, the WTRUs 102a, 102b, 102c may communicate / connect to the gNBs 180a, 180b, 180c while also communicating / connecting to another RAN, such as the eNode-Bs 160a, 160b, 160c. For example, the WTRUs 102a, 102b, 102c may implement DC principles to communicate with one or more gNBs 180a, 180b, 180c and one or more eNode-Bs 160a, 160b, 160c substantially simultaneously. In a non-standalone configuration, the eNode-Bs 160a, 160b, 160c may serve as mobility anchors for the WTRUs 102a, 102b, 102c and the gNBs 180a, 180b, 180c may provide additional coverage and / or throughput for serving the WTRUs 102a, 102b, 102c.

[0077] Each of the gNBs 180a, 180b, 180c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in UL and / or DL, support of network slicing, dual connectivity, interworking between NR and E-UTRA, routing of user plane data towards a user plane function (UPF) 184a, 184b, routing of control plane information towards an access and mobility management function (AMF) 182a, 182b, and the like. Figure 1D As shown in , gNBs 180a, 180b, and 180c can communicate with each other via the Xn interface.

[0078] Figure 1DThe CN 115 shown in FIG may include at least one AMF 182 a, 182 b, at least one UPF 184 a, 184 b, at least one session management function (SMF) 183 a, 183 b, and at least one data network (DN) 185 a, 185 b. While each of the aforementioned elements is depicted as part of the CN 115, it will be understood that any of these elements may be owned and / or operated by an entity other than the CN operator.

[0079] The AMF 182a, 182b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 113 via the N2 interface and may serve as a control node. For example, the AMF 182a, 182b may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, supporting network slicing (e.g., handling of different protocol data unit (PDU) sessions with different requirements), selecting a specific SMF 183a, 183b, managing registration areas, terminating NAS signaling, mobility management, and the like. Network slicing may be used by the AMF 182a, 182b to customize CN support for the WTRUs 102a, 102b, 102c, for example, based on the type of services utilized by the WTRUs 102a, 102b, 102c. For example, different network slices may be established for different use cases, such as services relying on ultra-reliable low latency (URLLC) access, services relying on enhanced massive mobile broadband (eMBB) access, services for MTC access, and / or the like. The AMFs 182a and 182b may provide a control plane function for switching between the RAN 113 and other RANs (not shown) employing other radio technologies, such as LTE, LTE-A, LTE-A Pro, and / or non-3GPP access technologies, such as Wi-Fi.

[0080] The SMF 183a, 183b may connect to the AMF 182a, 182b in the CN 115 via the N11 interface. The SMF 183a, 183b may also connect to the UPF 184a, 184b in the CN 115 via the N4 interface. The SMF 183a, 183b may select and control the UPF 184a, 184b and configure the routing of traffic through the UPF 184a, 184b. The SMF 183a, 183b may perform other functions such as managing and allocating UE IP addresses, managing PDU sessions, controlling policy enforcement and QoS, providing downlink data notification, and the like. The PDU session type may be IP-based, non-IP-based, Ethernet-based, and the like.

[0081] The UPFs 184a, 184b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 113 via an N3 interface, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices, for example. The UPFs 184a, 184b may perform other functions, such as routing and forwarding packets, enforcing user plane policies, supporting multi-homed PDU sessions, handling user plane QoS, buffering downlink packets, providing mobility anchoring, and the like.

[0082] The CN 115 may facilitate communications with other networks. For example, the CN 115 may include, or may communicate with, an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that serves as an interface between the CN 115 and the PSTN 108. Additionally, the CN 115 may provide the WTRUs 102a, 102b, 102c with access to other networks 112, which may include other wired and / or wireless networks owned and / or operated by other service providers. In an embodiment, the WTRUs 102a, 102b, 102c may be connected to a local data network (DN) 185a, 185b through the UPFs 184a, 184b via the N3 interface to the UPFs 184a, 184b and the N6 interface between the UPFs 184a, 184b and the DNs 185a, 185b.

[0083] Given that Figures 1A-1D as well as Figures 1A-1D

[0066] As described herein, one or more or all of the functions described herein with reference to any of the following may be performed by one or more emulated elements / devices (not shown): the WTRUs 102a-d, base stations 114a-b, eNode-Bs 160a-c, MMEs 162, SGWs 164, PGWs 166, gNBs 180a-c, AMFs 182a-b, UPFs 184a-b, SMFs 183a-b, DNs 185a-b, and / or any other element / device(s) described herein. An emulated device may be one or more devices configured to emulate one or more or all of the functions described herein. For example, an emulated device may be used to test other devices and / or simulate network and / or WTRU functions.

[0084] The emulation device can be designed to implement one or more tests of other devices in a laboratory environment and / or in an operator network environment. For example, one or more emulation devices can perform one or more or all functions while being fully or partially implemented and / or deployed as part of a wired and / or wireless communication network in order to test other devices within the communication network. One or more emulation devices can perform one or more functions or all functions while being temporarily implemented / deployed as part of a wired and / or wireless communication network. The emulation device can be directly coupled to another device for testing and / or can use over-the-air wireless communication to perform testing purposes.

[0085] One or more simulation devices can perform one or more (including all) functions without being implemented / deployed as part of a wired and / or wireless communication network. For example, the simulation device can be used in a test lab and / or in a test scenario in a non-deployed (e.g., testing) wired and / or wireless communication network to enable testing of one or more components. The one or more simulation devices can be test equipment. Data can be transmitted and / or received by the simulation device using direct RF coupling and / or wireless communication via an RF circuit system (e.g., which can include one or more antennas).

[0086] Throughout the embodiments described herein, the terms "serving base station," "base station," "gNB," and "network," collectively referred to as "gNB," may be used interchangeably to refer to any network element, such as, for example, a network element acting as a serving base station. The embodiments described herein are not limited to gNBs and are applicable to any other type of serving base station.

[0087] For clarity, conditions are satisfied, conditions are not satisfied, and "configured (one or more) condition parameters" are described throughout the embodiments described herein as being relative to a threshold (e.g., greater than or less than a (e.g., threshold) value, configured (e.g., threshold) value, etc.). For example, a condition is satisfied as being above a (e.g., threshold) value, and a condition (e.g., performance criterion) is not satisfied as being below a (e.g., threshold) value. The embodiments described herein are not limited to threshold-based conditions. Any variety of other conditions and (one or more) parameters (such as, for example, falling within or not falling within a range of values) may be applicable to the embodiments described herein.

[0088] Throughout the embodiments described herein, (e.g., configuration) information may be described as being received by the WTRU from the network, for example, through system information or via any type of protocol message. Although not explicitly mentioned throughout the embodiments described herein, the same (e.g., configuration) information may be pre-configured in the WTRU (e.g., via any type of pre-configuration method, such as, for example, via factory settings) so that the (e.g., configuration) information can be used by the WTRU without being received from the network.

[0089] Throughout the embodiments described herein, the term “sending a request (or a response)” may be used interchangeably with “sending information indicating a request (or a response)”.

[0090] Throughout the embodiments described herein, the terms “augmented reality device (ARD)” and “WTRU” may be used interchangeably to refer to any wireless device that includes AR capabilities. Throughout the embodiments described herein, the terms “first ARD,” “ARD-1,” “first WTRU,” and “AR resource consumer (ARRC)” may be used interchangeably to refer to any AR-capable WTRU that operates as an AR resource consumer. Throughout the embodiments described herein, the terms “second ARD,” “ARD-2,” “second WTRU,” and “AR resource provider (ARRP)” may be used interchangeably to refer to any AR-capable WTRU that operates as an AR resource provider, e.g., an ARRC.

[0091] Throughout the embodiments described herein, the term "position A" may be used interchangeably with "first position," the term "position B" may be used interchangeably with "second position," and the term "position C" may be used interchangeably with "third position."

[0092] Throughout the embodiments described herein, the terms "physical world knowledge" and "physical world knowledge information" may be used interchangeably to refer to any piece of information, including any kind of knowledge about a physical (eg, real) environment.

[0093] Throughout the embodiments described herein, the terms “requested type of AR resource” and “requested type of AR resource” may be used interchangeably.

[0094] Overview

[0095] Existing AR applications are rudimentary (e.g., providing basic AR effects, pre-designed AR designs, pre-selected (e.g., pre-configured) virtual objects, etc.). Future AR applications may be more advanced, more dynamic, and more customized. In an example, an AR device (ARD) may dynamically complete an AR scene designed locally on site (e.g., when the ARD enters an unknown or new physical environment). Compared to existing centralized or pre-designed AR scene designs, the embodiments described herein may enable personalized (e.g., customized) AR experiences. For example, advanced AR processing may include two stages (e.g., steps). Advanced AR processing may include: a first stage of dynamic (e.g., and customized) AR scene design for a new environment, which may further involve a first task of physical target object (PTO) determination for the physical environment; and a second task of virtual object (VO) selection (e.g., and preparation).

[0096] A PTO can be a physical object (PO) in a physical environment that can be enhanced with AR effects. The AR effect can be considered as the user's perceived AR experience. For example, enhancing a PO with an AR effect can be achieved by superimposing one or more virtual assets (e.g., (one or more) objects) onto a real-world rendered physical object.

[0097] For a PTO, it may be determined what type of VO may be overlaid on the PTO, and the VO may be prepared (e.g., if not stored locally, the VO may be downloaded, which may have a large size, and the behavior (and / or appearance) of the VO may be configured in a customized manner based on, for example, user preferences).

[0098] Advanced AR processing may include a second stage of AR scene deployment for execution and rendering, which may include any of deploying the designed AR scene to an AR processor (ARP) for execution, performing VO registration (e.g., overlaying the VO onto the PTO), and AR rendering (the rendered content is sent to a display for presentation to the user when accessing the physical environment).

[0099] Existing AR solutions are unable to provide dynamic and customized AR experiences as described in this article. In an example, existing solutions may focus on how AR devices can utilize the computing resources of edge infrastructure, for example, to increase the processing speed of AR scene execution and rendering, but they do not allow for dynamic and customized AR scene design as described in this article. In addition, existing solutions do not allow for "cross-user collaborative AR processing" (e.g., device-to-device collaboration). For example, after device-to-device collaboration can be enabled, other potential types of resource sharing (in addition to computing resources) for supporting advanced AR processing can be achieved.

[0100] The embodiments described herein may allow cross-user AR processing and resource sharing to be enabled to support dynamic and customized AR experiences. The embodiments described herein may allow an ARD to obtain relevant (e.g., useful) AR-related resources (shared by other ARDs), such as, for example, any of video frames and physical world knowledge about the physical environment (e.g., extracted from video frames), earlier than resources that may be provided by existing non-device collaboration solutions. The embodiments described herein may allow an ARD to, for example, become familiar with a physical environment (e.g., receive information associated with the physical environment) before accessing (e.g., arriving into) the physical environment. The ARD may have more time to perform dynamic AR design and processing for AR scene deployment and rendering, thereby creating a richer AR experience.

[0101] This paper describes the functional architecture of an ARD that can play the logical role of an AR Resource Provider (ARRP).

[0102] This paper describes the functional architecture of an ARD that can play the logical role of an AR Resource Consumer (ARRC).

[0103] In the case where the ARRC expects to obtain AR resources (eg, either original video frames or physical world knowledge extracted from video frames) from the ARRP, a collaborative (eg, resource sharing) relationship may be established. A process for enabling cross-user collaborative relationship establishment is described herein.

[0104] Two types of AR resource sharing between ARRP and ARRC are described herein, which may be referred to herein as Type-1 and Type-2.

[0105] In the first type (e.g., Type-1) of AR resource sharing, which may be referred to herein as raw video frame sharing, the ARRC may send a request to the ARRP to share its raw video frame, and the ARRC may perform further AR processing to extract physical world knowledge. A process for enabling raw video frame sharing between the ARRP and the ARRC is described herein.

[0106] In a second type of AR resource sharing, which may be referred to herein as physical world knowledge sharing (e.g., Type-2), a portion of the AR processing may be performed directly by the ARRP on behalf of the ARRC. For example, the ARRC may send an AR resource sharing request to the ARRP, indicating a request to share physical world knowledge. For example, (e.g., only) physical world knowledge information may be returned (e.g., transmitted) to the ARRC. Depending on how the physical world knowledge may be utilized by the ARRC, different processes are described herein.

[0107] The first and second types of AR resource sharing may be used in combination, such that raw video frames and physical world knowledge may be shared between ARRC and ARRP (e.g., simultaneously).

[0108] A first process for enabling physical object detection and recognition is described herein. In this process, the ARRC may request the ARRP to obtain useful (eg, relevant) physical world knowledge information (eg, indicating PTO) about the (eg, unknown) physical environment X.

[0109] A second process for enabling knowledge accuracy and freshness verification is described herein. In this process, the ARRC may hold (e.g., possess, include) some physical world knowledge about physical environment X (which the ARRC may / may not have yet reached or entered). For example, the knowledge may be outdated. The ARRC may request (e.g., request) the ARRP to verify the accuracy and freshness of the knowledge held by the ARRC.

[0110] A third process for enabling pre-notification of the appearance of a mobile PO is described herein. In this process, the ARRP may detect a mobile PO that may / may not yet have entered the viewport of the ARRC. The ARRP may send information indicating the pre-notification to the ARRC so that the ARRC may be aware of (e.g., be notified of) the upcoming appearance of the PO in advance, so that the ARRC may have more time to prepare the AR effects to be used on such a mobile PO (e.g., before it appears in the viewport of the ARRC).

[0111] Augmented Reality (AR) Overview

[0112] Augmented reality (AR) can be viewed as integrating the virtual world with the physical world by applying (e.g., overlaying) virtual information (e.g., assets, scenes) to the physical world. In AR applications, the real world and virtual objects can coexist in the same viewport of the user, thereby enhancing the user's real sensory experience. AR applications can have three (e.g., main) features (e.g., characteristics). The first feature can be a combination of the virtual world and the real world. The second feature can be real-time interaction. AR users can use any of their gestures, voice, touch, etc. to interact with the AR system in real time. The third feature can be virtual object registration, which can refer to the tracking and positioning of virtual assets (e.g., objects) in the real world so that virtual information (e.g., assets) can be superimposed on the real world. The main enabling technologies for AR can involve any of multimedia, 3D modeling, sensors, computer vision (such as, for example, any of object recognition and detection, real-time object tracking, and virtual asset registration), and video rendering.

[0113] An example of an AR processing pipeline may be as follows. First, a camera of an AR device may capture a video frame from the real world. The AR device may then perform (e.g., execute) a feature extraction operation from the video frame. Based on the extracted feature information, the AR device may perform (e.g., execute) object recognition to identify one or more physical objects (POs). For example, based on business logic, the AR device may determine whether some of the POs may be physical target objects (PTOs), which may be POs to be annotated (e.g., enhanced) with AR effects. For example, either a marker-based technique or a markerless-based technique may be used to determine where to place the VOs in the physical environment. The AR effect may be the user's perceived AR experience. Enhancing the POs with AR effects may be achieved by superimposing one or more virtual assets / objects onto real-world physical objects. Throughout the embodiments described herein, virtual assets and virtual objects (VOs) may be used interchangeably.

[0114] Simultaneous localization and mapping (SLAM) techniques may also be performed (e.g., used, executed) to build a world map and locate the AR device within the map. These operations may enable the AR device to extract relevant (e.g., useful) knowledge about the real (e.g., physical) world. Depending on, for example, business logic, the AR device may select an appropriate VO to overlay on the PTO. For example, the VO may be a static logo, a dynamic (e.g., moving) cartoon character, a decoration, etc. During this process, the VO may maintain precise alignment with the PTO to be annotated in the physical environment. This may be achieved through 3D tracking and registration techniques. The process of positioning the VO in a precise (e.g., perfect) position overlaid on the real world is referred to herein as VO registration. The performance of VO registration may depend on the accuracy of the physical world knowledge held by the AR device. For example, a 3D map or model of the physical world generated via SLAM may be used to accurately estimate the pose (position + orientation) of the AR device's camera. The AR device may track (and react to) various real-time changes (such as, for example, the user's current position, head angle, motion, etc.) to re-establish coordinate system alignment based on the user's current viewport and maintain the VO in the appropriate position. For example, to avoid repeated PTO detections in (e.g., every) video frame, the AR device may take an initial video frame as input and may perform object tracking across multiple frames. For example, rendering may be performed to create a final AR effect that may be delivered and displayed on a display for presentation to the user.

[0115] In the example, for a physical environment X, an example of AR scene design to be applied to create an AR experience in physical environment X and deliver the AR experience to the user may involve two main tasks:

[0116] The first task of AR scene design may include determining the PTO for the physical environment. PTO can be considered as a point of view (PO) in the real world to be enhanced with AR effects. For example, an AR device can overlay virtual information (e.g., VO) onto the PTO so that from the user's viewpoint, the physical world can be integrated or enhanced with virtual assets (e.g., information, content). Determining which POs can be PTOs may depend on the application business logic. Detecting and identifying PTOs may involve AR processing technologies, such as object detection and recognition technologies in the field of computer vision. In existing AR applications, PTOs can be detected based on, for example, a marker-based VO registration method. The AR device can capture, identify, and locate visual markers (e.g., fiducial markers) in the camera's video frame. VO can be annotated to the marker. Another example may be a markerless VO registration method. The TikTok app can use natural features to detect faces, so that different facial effects (such as adding hats, beards, and other virtual objects) can be added in real time. Detecting (e.g., locating) faces in the camera view can be performed in real time.

[0117] The second task of AR scene design may include VO selection and preparation. For example, for a PTO, it may be determined which VO can be superimposed on the PTO. For example, a VO may be a 3D virtual asset that may be represented as a file including a data structure (e.g., information) about the VO, such as any of its shape, geometry, color, and texture (determining the appearance of the VO). For example, the behavior of the VO may (e.g., further) be configured. The appearance of the VO may refer to what the VO may look like (e.g., a Hello Kitty cat, a 3D hat, a cartoon Superman character, etc.), including any of its shape, color, texture, etc. For example, a VO designer may use an AR authoring tool to design a 3D virtual asset (e.g., a VO), which may be published in a 3D asset store for use in one or more AR applications. The VO may be associated with, for example, a (e.g., planned) behavior. The VO may provide a configuration API so that the behavior of the VO can be configured when it is used in an AR scene. In existing (e.g., basic) AR applications, preparing the VO may be basic. For example, when TikTok users can add facial effects, VO (such as hats and beards) can be stored locally and at a reduced size (as props).

[0118] In an example, in AR scene design, VO selection and preparation can be performed first, and PTO can be determined after VO selection and preparation have been performed. For example, AR design can first decide what VO can be used, and then based on the behavior of the VO, PTO will be selected.

[0119] Overview of 3GPP activities on AR

[0120] 3GPP SA4 has a series of work focused on AR, virtual reality (VR) and extended reality (XR). For example, 3GPP TR 26.918 "Virtual Reality (VR) media services over 3GPP", v.17.0.0, 2022-04-07, includes VR audio and video content production processes, VR business use cases, and VR audio and video quality assessment. 3GPP TS 26.118, "Virtual Reality (VR) profiles for streaming applications," v.17.0.0, June 26, 2021, defines operating points for VR streaming media services and a VR client reference architecture for VR streaming media applications. 3GPP TR 26.928, "Extended Reality (XR) in 5G," v.17.0.0, April 7, 2022, introduces XR-related content, including various XR use cases, basic XR technology background (XR delivery, XR rendering, XR visual formats, etc.), and how XR delivery can be mapped to the 5G system architecture. 3GPP TR 26.998, "Support of 5G Glass-type Augmented Reality / Mixed Reality (AR / MR) devices," v.17.1.0, September 23, 2022, focuses on the design of AR glasses and defines three different ARWTRU architectures. For example, type-1 AR devices can perform independent AR processing. For example, type-2 and type-3 AR devices can utilize computing resources on the associated WTRU or on the edge (e.g., cloud). In addition, subsequent 3GPP standardization activities are still ongoing, such as 3GPP TS26.119 "Media Capabilities for Augmented Reality", v.0.1.0, 2022-05-02, on media capabilities for AR, and 3GPP TR26.806 "Study on Tethering AR Glasses–Architectures, QoS and Media", v.0.3.0, 2022-09-23, which focuses on smart tethering of AR glasses.

[0121] In addition, the Metaverse-related SA1 study item (3GPP TR.22.856 “Study on Localized Mobile Metaverse Services”, 0.2.0, 2022-09-21) focuses on the feasibility study of how to provide a shared and interactive user experience of local AR content and services between nearby users.

[0122] Overview of the ETSIARF ISG group

[0123] The ETSI Augmented Reality Framework Industry Specification Group (ISG ARF) aims to define a framework for the interoperability of AR components, systems, and services, as described in ETSI GSS ARF 003 V1.1.1 (2020-03), "Augmented Reality Framework (ARF); AR framework architecture". By using this framework, components from different vendors can interoperate through standard interfaces to provide a holistic AR solution. For example, ETSI ARF can leverage existing 5G or 6G communication infrastructure to improve AR design, deployment, and operation. ARF can provide support for the provision of portable AR components, allowing AR solutions to run on different software or hardware platforms.

[0124] According to ETSI ARF, the AR system can be divided into three layers, the upper layer can be the hardware layer, the middle layer can be the software layer, and the lower layer can be the data layer. The hardware layer may include any one of one or more sensors, a computing processing unit, a presentation interface, and a user interaction interface. The software layer may include a visual engine (which can combine virtual content with the real world, detect (e.g., any one of identifying, understanding, and modeling) the physical world) and a rendering engine (for generating the final visualization to present to the user). The data layer may include physical world knowledge (including any one of semantic understanding, 3D models, and 3D maps of the physical world), etc.

[0125] The functional architecture defined by ETSI ARF may involve three functions: 1) physical world knowledge management, including any one of physical world capture, physical world analysis, semantic understanding, and physical world knowledge storage; 2) AR scene management, including AR scene design (e.g., using an AR authoring tool for designing AR scenes) and 3D virtual asset preparation. Regarding the scene data format, a scene graph or scene description may describe virtual assets, which may be a tree-based data structure (glTF 2.0 format may be considered as a popular scene data format adopted by MPEG); 3) AR rendering and display. The designed scene may be rendered by a 3D rendering engine and presented to the user through various display devices (such as, for example, an optical see-through display).

[0126] Use case example of street walking with dynamic and customized AR experience

[0127] Wearable AR devices such as AR glasses may be the next big trend in our lives. Like smartphones, AR glasses can provide similar functions to those available on smartphones today, such as, for example, making calls, accessing the internet, etc. Compared to smartphones (which people can hold when interacting with them), wearable AR devices such as AR glasses can free people's hands by supporting human-computer interfaces based on body gestures. Considering that people may wear their AR glasses all the time, those devices can capture real-time images or videos using built-in forward-facing high-definition (HD) cameras without any human intervention. In the case where the user is walking in a shopping mall, an indoor exhibition event, etc., the captured image (e.g., video) may include either an outdoor street view or an indoor view. AR glasses can overlay one or more virtual assets onto the physical world in the user's viewport to enable a vivid AR experience.

[0128] Today's AR experiences can be basic and rudimentary, and can be implemented on heavy-client AR headsets (such as Microsoft HoloLens) or lighter-weight AR glasses via computational offloading to the edge. For example, existing solutions can enable AR experiences through pre-designed AR scenes, e.g., pre-defined PTOs to be annotated, pre-selected or pre-configured VOs, etc. These solutions may or may not be able to support future AR applications, which may be 1) more complex and 2) more dynamic (e.g., online) and customized, enabling AR devices to be based on more complex AR processing mechanisms. For example, AR devices may dynamically complete AR scene design for unknown (e.g., new) physical environments. For example, AR devices may enable user-customized AR experiences. Compared to detecting a face in a selfie camera, the AR processing involved, such as PTO detection and recognition (associated with physical world understanding), may require significantly more AR processing power. Similarly, with respect to VO selection, the VOs anticipated by future AR applications may not be locally available, and some of the VOs may be so large in size that they may not be downloadable from the VO repository in a reasonable amount of time. For example, a typical 3D human character published today by the Unity Asset Store may be 300-500MB in size. This can grow to over a GB, and AR scenes may be based on many objects. For example, future VO preparation may be computationally more complex. For example, different users may expect VOs to have different customized behaviors, involving intensive AR scene design processing.

[0129] Existing predefined AR scene solutions may become inefficient for more advanced AR experiences and may / may not provide the dynamic and customized AR scene design expected by users.

[0130] In a first example, predefined AR scene designs may or may not be feasible for more advanced AR experiences. For example, an AR device may enter a new physical environment X for the first time and may not utilize prior knowledge to pre-complete scene designs for the unknown (e.g., new) physical environment X. For example, it may not know what potential PTOs may exist in the physical world, their locations / places / appearances, etc.

[0131] In a second example, the same predefined AR scene may or may not be suitable for different AR users due to different preferences. For example, for the same VO, some users may want to annotate the VO on a vertical plane (such as a window or wall of a street-level building), while other AR users may want to annotate the same VO on the roof of a building. In other words, what PTOs can be annotated may vary from user to user. This can dynamically add another dimension to the AR scene. Based on user preferences, PTO detection and recognition can be more dynamic.

[0132] In a third example, while different AR users may wish to annotate the same PO, different users may want to employ different VO(s). For example, some users may wish to use a cartoon character (e.g., Superman), while other users may wish to use a different character, such as Hello Kitty.

[0133] In a fourth example, where different AR users intend to annotate the same VO to the same PO, different users may want the VO appearing in their respective viewports to have different actions and behaviors.

[0134] Figure 2is a diagram illustrating an example of a dynamic and customized AR experience for exploring a new city street. In this example, a user 20, referred to herein as Lisa, may wear AR glasses 21 (referred to herein as any of AR device-1, ARD-1, or the first ARD) and visit a new shopping street (e.g., which may have just been built in the downtown area of ​​a tourist city). As Lisa 20 may be walking along the street, ARD-121 may capture video frames from the physical world. For example, ARD-121 may analyze the captured video frames in real time using image recognition. For example, a customized approach may be utilized to construct (e.g., determine) a real-time or dynamic AR scene design based on, for example, Lisa's preferences. For example, to design this customized AR scene, and to understand the physical world, ARD-121 may determine which points of view (POs) may be PTOs that can be enhanced with AR effects (e.g., based on Lisa's preferences). After the PTOs may have been determined, ARD-121 may determine what types of voices (VOs) may be selected and what types of actions or behaviors the VOs may have. Designing dynamic and customized AR experiences for the Lisa 20 (walking in a new physical environment) may involve more efficient and timely AR processing than provided by existing AR solutions.

[0135] For example, VO selection and preparation may be performed (e.g., executed) first, and a 3D virtual character 22 (e.g., a virtual Superman character) may be selected based on Lisa's preferences. For example, the 3D virtual character 22 (Superman) may have customized behaviors, such as walking with Lisa 20 and introducing popular local pizza styles served by a nearby pizza shop. For example, Lisa may prefer Superman 22 to walk, run, or jump around her for a more vivid AR experience. For example, ARD-1 may prepare a high-fidelity VO (e.g., a 3D Superman asset 22) with detailed behavioral design. For example, any of Superman 22's body color, brightness, and texture may be configured to match the current daylight intensity of the street, for example. PTO determination may be performed in an efficient manner to detect and determine one or more PTOs (in the new physical environment), such as, for example, Superman 22's jumping point. For example, Superman's 22 actions may involve identifying two PTOs 211 , 212 : Superman jumps from a street truck 211 (which may be referred to herein as PTO- 1 ) to the roof of a pizza shop 212 (which may be referred to herein as PTO- 2 ) to guide Lisa 20 to the shop, etc.

[0136] Depend on Figure 2 The illustrated examples are examples of advanced AR experiences with dynamic and customized AR scene creation in unknown environments, which can be enabled by the embodiments described herein.

[0137] To achieve such advanced AR experiences, AR processing operations can be based on efficiently performing (1) dynamic AR scene design and (2) AR scene deployment for rendering.

[0138] Performing dynamic AR scene design (e.g., for a new environment) may include PTO determination for the physical environment, such as, for example, detecting and identifying POs in the physical environment, determining which POs may be PTOs to be annotated with AR effects. Performing dynamic AR scene design (e.g., for a new environment) may further include VO selection and preparation, such as, for example, determining (e.g., selecting, downloading) a VO and configuring any of customized behaviors of the VO.

[0139] Performing AR scene deployment for rendering may include any of deploying the designed AR scene to an AR processor for execution, performing (e.g., executing) VO registration (e.g., overlaying the VO to the selected PTO), and AR rendering (the rendered content may be sent to a display for presentation to a user).

[0140] The dynamic and customized AR experiences described herein may involve higher implementation complexity and more efficient AR processing than is available in existing AR solutions. For example, collaborative (e.g., cross-user) AR processing (e.g., and / or resource sharing) between different ARs that may / may not be available in existing AR solutions may allow for improved AR experiences while reducing implementation complexity.

[0141] In some examples, in some AR solutions, AR devices may operate independently with limited collaboration between other AR devices. For example, collaboration may be limited to VO sharing between different ARs, where (e.g., only) VO (e.g., visual assets) may be shared between the ARs, and AR processing tasks of the different ARs may be performed (e.g., executed) individually rather than in a collaborative manner.

[0142] To enable advanced dynamic AR experiences, for example, in an efficient manner (in terms of computational load, power, timeliness, etc.), AR devices can share resources, insights, and AR assets across one or more AR processing operations. Cross-user collaborative AR processing and resource sharing are described herein, where one or more AR processing tasks of an ARD can be assisted (e.g., assisted) by one or more other ARDs.

[0143] Figure 33 is a diagram illustrating an example of cross-user collaborative AR resource sharing that enables a shared AR experience. A second user (who may be referred to herein as Mike) may walk from location B 302 to location C 301. A first user (who may be referred to herein as Lisa) may be at three hundred meters 300, which may thereafter be referred to herein as location A 301. For example, Lisa may walk toward location B 302 and toward location C 303. In this example, video frames captured by a second ARD 32 (e.g., Mike's AR device, which may be referred to herein as ARD-2) may be considered another type of AR resource (e.g., for sharing), which may be utilized by a first ARD 31 (e.g., Lisa's AR device, which may be referred to herein as ARD-1) to enable (e.g., facilitate) its customized AR scene design and processing. For example, by utilizing the video frames of the second ARD 32, the first ARD 31 may be enabled to know (e.g., learn) a new (e.g., unknown) physical environment X (e.g., the street between location B 302 and location C 303, even when the first ARD 31 is at location A 301 and has not yet reached location B 302). The physical world knowledge information shared by the second ARD 32 may include any of the following: (i) a physical map of the unknown physical environment X; and (ii) a semantic perception of the unknown physical environment X (such as, for example, PO detected by the second ARD 32 via object recognition). The physical world knowledge information may allow the first ARD 31 more time to proactively and dynamically design a customized AR experience for Lisa, as she may later walk between location B 302 and location C 303. Physical world knowledge information may include (e.g., detailed) characteristics of candidate PTOs (such as any of 3D position, shape, orientation, color, etc.). This can allow first ARD 31 to accelerate (e.g., facilitate, improve) its own PTO identification when processing video frames later captured from its own camera (e.g., because Lisa can walk between position B 302 and position C 303). For example, first ARD 31 can (e.g., quickly) verify the presence of a PTO in its own video frames, which may be faster than starting from scratch for image recognition and may require less computational power. For example, by knowing the PTO information shared by second ARD 32 in advance, first ARD 31 can (e.g., proactively) determine which VO (e.g., Superman) can be used. In the event that the VO is not available locally (e.g., and represents a large amount of data), first ARD 31 may have sufficient time to download the Superman 3D VO from a remote repository. For example, first ARD 31 may have time to complete the customized behavioral design of the Superman VO.

[0144] Described herein are methods, architectures, devices, and systems that enable cross-user collaborative AR processing and resource sharing for enabling dynamic and customized AR experiences.

[0145] This paper describes an AR device functional architecture for cross-user collaborative AR resource sharing, where a first ARD plays a first logical role of AR resource consumer (ARRC) and a second ARD plays a second logical role of AR resource provider (ARRP).

[0146] Described herein is a process for enabling the establishment of cross-user collaborative relationships.

[0147] This article describes embodiments based on the following types of AR resource sharing between ARRP and ARRC: a first type of AR resource sharing (referred to as Type-1), in which raw video frames can be shared; and a second type of AR resource sharing (referred to as Type-2), in which physical world knowledge can be shared. The embodiments described herein are not limited to the described raw video frame sharing and physical world knowledge sharing. The embodiments described herein can be applied to any other type of AR resource sharing between ARRC and ARRP.

[0148] This document describes a process for enabling sharing of raw video frames between ARRP and ARRC.

[0149] Depending on how the physical world knowledge can be utilized by the ARRC, different processes for sharing physical world knowledge are described in this paper:

[0150] Described herein are processes for enabling physical object detection and recognition.

[0151] A process for enabling knowledge accuracy and freshness verification is described herein.

[0152] Described herein is a process for enabling advance notification regarding mobile PO appearance.

[0153] Architecture for cross-user AR resource sharing

[0154] A method for enabling cross-user AR resource sharing to support dynamic and customized AR experiences is described herein. In an example, in addition to computing resources, other types of AR-related resources can be shared between different AR devices, which may include resources such as, for example, raw video frames, physical world knowledge extracted from raw video frames (such as, for example, any of a map of the (e.g., unknown) physical environment and semantic perception of the physical environment (e.g., POs detected in the physical environment and their characteristics)) and other AR resources. For example, physical world knowledge can be shared between different ARDs based on a centralized knowledge repository. In another example, physical world knowledge can be shared in a decentralized manner, for example, directly among multiple ARDs. Sharing physical world knowledge in a decentralized manner can be used in the following example (using Figure 3 The use case of is described as an illustrative example, where the street between location B 302 and location C 303 can be considered to provide benefits in a new (e.g., unknown) physical environment for the first ARD device 31 (e.g., Lisa's AR device).

[0155] In a first example, a single world knowledge repository that may be available in the cloud and that may include (eg, all) physical world information may represent a single point of failure and may / may not be able to provide services.

[0156] In a second example, the street segment between location B 302 and location C 303 may be a new or unpopular area, and there may be no knowledge available about the area in the centralized world knowledge repository.

[0157] In a third example, the physical world knowledge about the street between location B 302 and location C 303 may be of large size. For example, due to either expensive communication costs or low-bandwidth wireless connections, Lisa's AR device 31 (e.g., ARD-1) may or may not be able to download the knowledge data from a centralized repository. For example, the AR application may expect detailed images of various POs (such as the windows or signs of pizza shops) to provide expected characteristic information about the POs, such as any of their 3D positions, orientations, colors, shapes, textures, etc.

[0158] In a fourth example, the physical world knowledge about the street between location B 302 and location C 303 may be outdated (e.g., may have been captured one or two months ago). For example, the physical world knowledge may / may not be accurate and may need to be revalidated or calibrated. For example, changes may sometimes occur, such as, for example, movable street parking vehicles, the brightness / color of street buildings (e.g., in the case of neon lighting or signs, the street may have different appearances during daytime and nighttime), etc.

[0159] by Figure 3 Taking the use case described in as an example, after resource sharing may have been enabled between ARDs, cross-user and collaborative AR processing can be performed, which can be reflected in the following aspects.

[0160] In a first aspect, ARD-2 (Mike's ARD) can help (e.g., assist) ARD-1 (Lisa's ARD) to more quickly and efficiently acquire physical-world knowledge about an unknown (e.g., new) physical environment (e.g., the portion of a street between locations B and C) before ARD-1 can even reach that location. For example, physical-world knowledge can include maps or other geo-related information. For example, ARD-2 can share SLAM-related map information with ARD-1, enabling ARD-1 to understand the physical-world layout (e.g., roads, buildings) or 3D model of the street between locations B and C before ARD-1 can even reach that location. For example, physical-world knowledge can (e.g., also) include semantic perception of the physical environment. For example, ARD-2 can perform computer vision-related operations to detect and identify one or more points of interest (POIs) between locations B and C, and can share this knowledge with ARD-1. For example, by utilizing this information, ARD-1 can understand which POIs can be annotated with AR effects, as ARD-1 (e.g., Lisa) can walk (e.g., move) between locations B and C.

[0161] In a second aspect, ARD-2 can assist (e.g., aid) ARD-1 with dynamic and customized AR scene design in VO preparation and behavior design. Based on the physical world knowledge about the (e.g., unknown) physical environment shared by ARD-2, ARD-1 can understand what types of PTOs (personal transport units) may exist in the (e.g., unknown) physical environment. For example, ARD-1 can (e.g., proactively) design customized AR effects for PTO(s) based on either business logic associated with ARD-1 (e.g., Lisa) or user preferences. Customization can include determining what types of VO(s) can be selected and overlaid on the PTO(s) and what behaviors the VO(s) can have. In cases where the selected VO(s) are not locally available and are large in size, ARD-1 may have time to proactively retrieve (e.g., download) the selected VO(s) from a VO repository in an on-demand manner. For example, either AR scene design or preparation actions can be performed in advance (e.g., before Lisa can walk between locations B and C).

[0162] In a third aspect, ARD-2 can allow ARD-1 to reduce the amount of computing resources required for AR processing. For example, ARD-1 can pre-process (e.g., execute) customized AR scene design and preparation, making it possible / unlikely necessary for these operations to be completed in the shortest possible time. This can further reduce the use of significant computing resources, such as from edge computing hosts. For example, the workload (e.g., processing) for dynamic AR scene design can be amortized (e.g., distributed) over a longer period of time. For example, sharing physical world knowledge with ARD-2 can allow ARD-1 to avoid understanding the physical world from scratch, as ARD-1 can travel between locations B and C. For example, ARD-1 can quickly relocate or verify the presence of PTO(s) in its own video frames based on already knowing their detailed characteristics (e.g., their position, shape, color, and physical features). These operations can consume fewer computing resources, making them affordable for ARD-1 and improving AR processing latency.

[0163] Figure 4 is a diagram illustrating an example of an AR device architecture for implementing cross-user collaborative AR processing and resource sharing. For example, an AR device can operate in either of two logical roles: an AR resource provider (ARRP) 42 and an AR resource consumer (ARRC) 41. For example, in Figure 3 In the use case described in , Mike's AR device (e.g., ARD-2) may include ARRP 42 and may share its video frame resources with Lisa's AR device (e.g., ARD-1) (which may include ARRC 41). For illustrative purposes, this use case will be used to describe the embodiments described herein. By default, throughout the embodiments described herein, a first ARD (e.g., Lisa's AR device ARD-1) may include ARRC 41, and a second ARD (e.g., Mike's AR device ARD-2) may include ARRP 42. For example, an AR device may simultaneously include two logical roles (e.g., functions) (e.g., ARRC 41 and ARRP 42). For example, an AR device may receive shared AR resources sent from another AR device, and may provide (e.g., share) its own resources with other AR devices. For example, an AR resource sharing coordinator (RSC) may be included in a network element to facilitate the establishment of a collaborative relationship between different ARDs. The RSC network element may allow matching ARRP 42 and ARRC 41. The RSC may be implemented (e.g., included, implemented) in the RAN (e.g., at a gNB), or as a network function in the CN, or as an application function (e.g., a server) in an edge data network or in the cloud.

[0164] Example of ARD as ARRP

[0165] The ARD including the ARRP 42 may include an AR Resource Sharing Service (ARRSS) 421 function and an AR Processor (ARP) function. The ARRSS may receive an AR resource sharing request sent from another ARD (which may include the ARRC 41). The AR resource sharing request may include information indicating what type of AR resource the ARRC 41 may request the ARRP 42 to share.

[0166] In a first example, a first type of requested resource (e.g., Type-1) may indicate a request to share an original resource such as, for example, a video frame. In the first type of sharing, ARD-1 (as ARRC) may send an AR resource request to ARRP indicating a request to share its original (e.g., raw) video frame captured at a location of interest, which may still be unknown to ARRC. For example, in Figure 3 In the use case described in [1], Lisa's AR device (e.g., ARD-1) may send an AR resource request to Mike's AR device (ARD-2), indicating a request to share video frames captured by ARD-2's built-in camera while ARD-2 may be traveling between locations B and C. In this example, the ARRP may or may not perform (e.g., execute) AR processing on its captured video frames, which may be delivered to ARD-1 via a communication link (such as, for example, 3GPP device-to-device (D2D) communication, 5G-NR, CPN, or a local LAN). This type of AR resource sharing may be applicable, for example, in situations where the ARRP may or may not have sufficient computational resources to assist the ARRC in processing the video frames. Given the potentially high quality of the raw video frames, this approach may generate more communication traffic than other approaches where the ARRP may perform some AR processing overhead. Type-1 sharing may or may not be limited to video frame sharing and may be used more broadly for any type of AR-related raw resource that may be beneficial or useful to the ARRC, such as, for example, any sensor data generated by any type of sensor, radar, etc.

[0167] In a second example, a second type of requested resource (e.g., type-2) may indicate a request to share physical world knowledge. In the second type of sharing, some of the AR processing may be performed by the ARRP on behalf of the ARRC. For example, ARD-1 (as an ARRC) may send an AR resource sharing request to an ARRP (e.g., an ARRSS hosted by the ARRP), indicating that the type of AR resource requested may be physical world knowledge (e.g., indicating what kind of physical world knowledge may be requested). For example, after the ARRP (e.g., an ARRSS on the ARRP) may have determined to serve the sharing request, the ARRSS may utilize the ARRP's AR processor to process the video frame. The output of the ARRP's AR processor may include physical world knowledge of interest that may be extracted from the original video frame. For example, (e.g., only) physical world knowledge may be delivered (e.g., transmitted) to the ARRC.

[0168] The embodiments described herein may be applicable to other types of AR resource sharing (e.g., in addition to raw video frames and physical world knowledge), such as, for example, AR scene design solution sharing and AR group information sharing.

[0169] In the AR scene design solution sharing example, ARD-2 (as an ARRP) can create customized and dynamic AR scene designs (e.g., for some profit) while ARD-2 is traveling in environment X. ARD-2 can share such AR scene designs with other ARDs (as ARRCs) that may / may not have yet reached environment X and may soon reach environment X.

[0170] In the AR group information sharing example, for some AR applications, multiple ARDs in physical environment X can interact with each other by forming a local AR group. For example, if ARD-2 is traveling in physical environment X, ARD-2 (as an ARRP) can collect AR group information. For example, such group information can be shared with other ARDs (as ARRCs) that may / may not have yet arrived in physical environment X and may soon be there and may be interested in joining the AR group.

[0171] An AR processor (ARP) can be considered as a module that can perform (e.g., all major) AR-related operations. The AR processor can collect one or more AR resources from built-in sensors and cameras. For example, the AR processor can obtain video frames from a built-in camera as input. For example, a series of AR operations can be performed by the AR processor, including extracting features from video frames and performing PO detection and recognition. The AR processor can also implement AR scenes by performing AR rendering operations.

[0172] Example of ARD as ARRC

[0173] The ARD including the ARRC 41 may include an AR resource sharing client (ARRSC) 411 function, an AR processor (ARP) function, and an AR application (ARA) 412 .

[0174] In an example, the ARRSC may send an AR resource sharing request to (one or more) other AR devices (in the role of an ARRP) (e.g., (one or more) ARRSSs hosted by (one or more) other AR devices). The AR resource sharing request may indicate what type of AR resource the ARRC may request the ARRP to share. Depending on the configuration, the ARRC's ARRSC may support either type-1 or type-2 AR resource sharing between the ARRP and the ARRC:

[0175] In Type-1 (e.g., raw video frame sharing), after the ARRC's ARRSC may have received video frames from the ARRP, it may forward them to the AR processor of the ARRC to perform further AR processing, such as, for example, extracting physical world knowledge.

[0176] In Type-2 (e.g., physical world knowledge sharing), some AR processing may be performed by the ARRP on behalf of the ARRC. For example, the ARRC (e.g., the ARRC's ARRSC) may send an AR resource sharing request to the ARRP (e.g., the ARRSS hosted by the ARRP) indicating a request to share physical world knowledge. For example, the ARRP may (e.g., only) return (e.g., transmit) information indicating physical world knowledge to the ARRSC.

[0177] In an example, the AR processor included in the ARRC 41 may have the following functions. In the case of type-1 sharing, the original video frames captured by the ARRP 42 may be shared with the ARRC 41. In this case, the AR processor of the ARRC 41 may, for example, extract physical world knowledge from the original video frames by itself. The AR processor of ARD-1 (as the ARRC 41) may obtain and process video frames (or AR resource input) about a physical environment captured by other ARDs (e.g., ARD-2), which may be new (e.g., unknown) to ARD-1. For example, the ARRC 41 may extract SLAM-related information, such as generating a map of a physical environment unknown to the ARRC 41. The ARRC 41 may further perform semantic perception of the unknown physical environment to detect and identify one or more POs. For example, based on the (one or more) AR resources shared by the ARRP 42 (e.g., ARD-2), the ARD-1 may dynamically transform an unknown physical environment into a known or at least familiar physical environment. For example, the physical world knowledge information can be further delivered (e.g., transmitted) to an AR application (ARA) 412 associated with the ARRC 41, and the AR application (ARA) 412 can perform dynamic and customized AR scene design for unknown physical environments based on the shared physical world knowledge. In the example of Type-2 sharing, the physical world knowledge can be extracted by the ARRP 42, so that the ARP of the ARRC 41 may / may not perform any knowledge extraction.

[0178] The AR processor of ARRC 41 can implement the designed AR scene output by ARA 412. For example, ARA 412 can deploy the designed AR scene for rendering so that the AR processor can generate AR effects for the user of ARRC 41. For example, after ARRC 41 may have arrived at physical environment X (e.g., target location (e.g., location B)), its own built-in camera may (e.g., begin) capturing video frames, which may be sent to the AR processor of ARRC 41. The AR processor of ARRC 41 can quickly reposition or verify the presence of (one or more) PTOs in its own video frames based on physical world knowledge about the physical environment (which may have been received from ARRP 42). For example, the AR processor of ARRC 41 can estimate the ARRC's camera pose (e.g., any one of position and orientation) and can perform one or more calibrations to overlay the VO onto the physical world. For example, the AR processor of ARRC 41 can perform rendering to deliver the AR effect to the physical viewport of the user of ARRC 41.

[0179] In an example, ARRC 41 may include ARA 412. The embodiments described herein focus on ARA 412 associated with ARRC, which may be a consumer of shared AR-related resources (such as physical world knowledge). AR applications may, for example, perform dynamic (e.g., online) and customized AR scene design based on application logic. For example, an AR application may receive physical world knowledge about an unknown physical environment and may generate a customized AR scene. Such an AR scene may be further deployed to the AR processor of ARRC 41 for implementation (e.g., rendering) so as to present AR effects to the user of ARRC 41 when ARRC 41 may later travel in a new (previously unknown) physical environment.

[0180] Figure 4 The basic AR device functional architecture is described for the ARRP 42 and ARRC 41. The ARRP 42 and ARRC 41 may be considered as separate WTRUs with communication capabilities to interact with other WTRUs.

[0181] Figure 5 Figures illustrate four examples of AR device functional architectures for ARRC. In the four examples of architectures 50A, 50B, 50C, and 50D, an ARD (as an ARRC) may include a first portion that may or may not have the ability to communicate (e.g., directly) with an ARRP. The first portion of the ARD (as an ARRC) may, for example, be associated with a second portion such as a WTRU (e.g., a smartphone). For example, the ARD may include AR glasses that may rely on an associated smartphone to communicate with other entities (e.g., an ARRP). For example, in the first architecture 50A, an associated WTRU 502A may provide communication support for the first portion 501A of the ARD (as an ARRC), which may include an ARP, an ARA, and an ARRSC. In the second architecture 50B, the ARP and ARA may be hosted by the first portion 501B of the ARD (as an ARRC), and the ARRSC may be hosted on the associated WTRU 502B. In the third architecture 50C, (e.g., only) the ARP may be hosted on the first portion 501C of the ARD (as an ARRC). The ARRSC and ARA may be hosted on the associated WTRU 502C. In the fourth architecture 50D, the ARA and ARRSC may be hosted on the associated WTRU 502D, and the first portion 502D of the ARD (as the ARRC) may (e.g., only) host the ARP. For example, the associated WTRU 502D may (e.g., also) host the ARP. For example, the ARP hosted on the associated WTRU 502D may be more powerful than the ARP hosted by the first portion 501D of the ARD (as the ARRC).

[0182] Figure 6 is a diagram illustrating three examples of AR device functional architectures for ARRP. In the three examples of architectures 60A, 60B, 60C, a first part of the ARD (as the ARRP) may / may not have the ability to communicate (e.g., directly) with the ARRC. For example, the first part of the ARRP may be associated with a second part such as, for example, a WTRU (e.g., a smartphone). For example, the first part of the ARD may be AR glasses that may rely on an associated smartphone to communicate with other entities (e.g., the ARRC). For example, in the first architecture 60A, the associated WTRU 602A may (e.g., only) provide communication support for the first part 601A of the ARD (as the ARRP), which may host ARRSS and ARP. In the second architecture 60B, the ARP may be hosted on the first part 601B of the ARD (as the ARRP), and the ARRSS may be hosted on the associated WTRU 602B. In the third architecture 60C, the ARRSS may be hosted on the associated WTRU 602C, and the first portion 601C of the ARD (as ARRP) may (e.g., only) host ARP. For example, the associated WTRU 602C may (e.g., also) host ARP. For example, the ARP hosted on the associated WTRU 602C may be more powerful than the ARP hosted by the first portion 601C of the ARD (as ARRP).

[0183] Example of establishing a cross-user collaborative relationship between ARRP and ARRC

[0184] To obtain AR resources (such as, for example, any one of raw video frames and extracted physical world knowledge) from the ARRP, the ARRC may (eg, first) establish a collaboration and resource sharing relationship with the ARRP.

[0185] Figure 7A FIGURE 1 is a diagram illustrating an example process for enabling cross-user collaborative relationship establishment. The process may depend on what type of (e.g., specific) AR resource(s) can be shared between the ARRP and the ARRC. For example, parameters in one or more steps may be adjusted. For example, an ARRC may establish one or more cross-user collaborative relationships with more than one ARD (as an ARRP).

[0186] ARD-171 may be an AR device operating as an ARRC. ARA-1 may be an AR application associated with ARD-1. For example, based on application logic, ARA-1 may be designed to serve users of ARD-171 (e.g., Figure 3ARD-272 can be another AR device operating as an ARRP, which can be used by different users (e.g., Figure 3 Mike) discussed in the use case illustrated in carries (e.g., is associated with).

[0187] As shown at 700 , either ARD- 171 or ARD- 272 may discover the RSC using any service discovery solution, such as, for example, based on any of the following examples.

[0188] In a first example, any one of the ARRC and the ARA may be pre-configured with RSC information, such as, for example, any one of a fully qualified domain name (FQDN) and an IP address of the RSC.

[0189] In a second example, the user may know a contact address, such as, for example, any one of the FQDN and IP address of the RSC, so that the RSC information may be configured via the user interface of the ARD.

[0190] In a third example, RSC information may also be provisioned via a mobile network operator (MNO) through either the 5GC and 6GC procedures. For example, RSC (e.g., configuration) information may be included in messages in either the protocol data unit (PDU) session establishment procedure or the WTRU registration procedure.

[0191] In step 701, ARD-272 may be at location B on a street and may be moving toward location C. For example, ARD-272 may operate as an ARRP. For example, ARD-272 may include a corresponding ARRSS that may service AR resource sharing requests received from one or more ARRCs. To advertise itself, ARD2 (e.g., its ARRSS) may send AR capability information (e.g., such as, for example, a capability registration request) to the network element hosting RSC-1 to indicate its AR-related resource sharing capabilities. The capability information may include any of an ARD identifier, an ARD role, types of AR resources that may be shared, AR hardware resource capabilities, AR software capabilities, and positioning information associated with the ARD and the physical environment.

[0192] In an example, the ARD identifier may indicate an identifier of ARD-272.

[0193] In an example, the ARD role may indicate what role(s) the ARD-272 may be able to play (e.g., operate). In this example, the ARD-272 may indicate that it can operate as an ARRP. In other examples, the ARD may have (e.g., operate) more than one role at a time, such as ARRP and ARRC as logical roles. For example, the ARD-272 may operate as an ARRP by sharing its AR resources with the ARD-171. At the same time, the ARD-272 may (e.g., also) operate as an ARRC in the event that it expects to obtain (one or more) AR resources from a third ARD device. For simplicity, embodiments are described herein in which the ARD-272 operates as an ARRP. The embodiments described herein are not limited to the ARD-272 operating (e.g., only) as an ARRP.

[0194] In an example, the type of AR resource that can be shared may indicate the type of AR resource that the ARD-272 may be able to provide (e.g., share). For example, in a Type-1 AR resource sharing type, the ARD-272 may indicate that it can share its original video frames with one or more ARRCs. For example, the ARD-272 may share (e.g., provide) different types of AR resources with different ARRCs. For example, the ARD-272 may perform Type-1 sharing with the ARD-171 (as described herein). At the same time, the ARD-272 may (e.g., also) perform Type-2 sharing with a third ARD. For example, the AR resource sharing capabilities of the ARD-272 may depend on any one of its hardware capabilities, software capabilities, MNO provider, application provider, policy, and user preferences.

[0195] In an example, AR hardware capabilities may indicate what kinds of AR-related hardware may be associated with the ARD-272, such as CPU capacity, memory (e.g., storage device) capacity, sensors mounted on the ARD-272 (such as inertial sensors (e.g., including any of an accelerometer and a gyroscope)), one or more cameras (including any of an RGB image capture camera, a depth camera, and a light detection and ranging (LiDAR) camera), etc.

[0196] In an example, AR software capabilities may indicate what kinds of AR software may be associated with (e.g., run on) the ARD-272, such as, for example, any of the following: (i) semantic perception capabilities, e.g., using one or more artificial intelligence (AI) capabilities, such as object detection and recognition, to understand the physical world, (ii) simultaneous localization and mapping (SLAM) capabilities, which may allow for the creation of a map of an unknown environment and provide positioning of the device within the environment, and (iii) AR rendering. For example, the AR software capabilities described herein may run (e.g., be executed) on the ARP.

[0197] In an example, the positioning information associated with the ARD and the physical environment may indicate any one of the ARD's current position (e.g., within the physical environment), the ARD's (e.g., planned travel) path (e.g., toward the physical environment), and the ARD's (e.g., average) movement speed. The ARD's (e.g., planned travel) path may, for example, indicate a direction. If the (e.g., planned travel) path changes, the ARD may send (e.g., notify) RSC-1 of the updated planned travel path. Any of the current position, path, and speed may allow RSC-1 to determine in which physical environment the ARD may be located, towards which physical environment the ARD may move, and when the ARD may exit / enter the physical environment.

[0198] In an example, in step 702, RSC-1 may record the AR resource sharing capabilities registered (e.g., indicated) by ARD-272 for future use and may send a confirmation to ARD-272. For example, if ARD-272 changes one or more of its capabilities, it may send another (e.g., updated) capability information to RSC-1, for example, based on steps 701 and 702.

[0199] In an example, ARA-1 may be an AR application for serving users of ARD-1 and for designing customized AR scenes / experiences for the users.

[0200] In an example, in step 703, based on business logic, ARA-1 may determine that ARD-171 may be about to travel through an unknown (e.g., new) environment X (such as a street between location B and location C), and ARA-1 may determine to provide an AR experience to the user traveling through environment X. The customized AR scene designed to implement the AR experience may be based on physical world knowledge about the unknown environment X, which may / may not yet be available to ARA-1.

[0201] In an example, in step 704 , in order to obtain the latest and accurate physical world knowledge about the physical environment X, ARA- 1 may determine to request (eg, some latest) AR-related resources describing the physical environment X.

[0202] In an example, in step 705, ARA-1 may send an AR resource sharing request to the ARRSC of ARD-1. The ARRSC may be considered a module of ARD1 for identifying one or more AR-related resources. In the case where ARA-1 and the ARRSC are hosted by different devices, ARA-1 and the ARRSC may maintain some information between them (e.g., to establish a connection). For example, ARA-1 may be hosted by an associated smartphone, and the ARRSC may be hosted by the ARD. For example, ARA-1 may utilize the ARRSC of one or more ARDs by maintaining the contact addresses of different ARRSCs. For example, the same ARRSC may (e.g., also) serve one or more ARAs by maintaining the contact addresses of different ARAs. In the case where ARA-1 and the ARRSC run on (e.g., a single) device, they may be implemented as a single module and may / may not exchange the AR resource sharing request message described in step 705. For example, ARD-1 may (e.g., directly) send an ARRP / ARRC match request 707 to RSC-1.

[0203] In an example, either an AR resource sharing request or an ARRP / ARRC matching request may indicate one or more types of AR resources that may be requested, information associated with the requested AR resources, and positioning information associated with the ARRC. The information associated with the requested AR resources may depend on the type of resource that may be requested and will be described in more detail below.

[0204] In an example, in the context of Type-1 sharing, the type of AR resource requested may indicate that raw video frames captured from environment X may be requested. Type-1 sharing may not be limited to video frame sharing, and in a broader sense may apply to any kind of raw data that can be used as AR-related resources by AR-1, such as sensory data generated by any type of sensor, radar, etc. In the context of Type-2 sharing, the type of AR resource requested may indicate that physical world knowledge extracted from video frames, for example, may be requested.

[0205] In an example, the information associated with the requested AR resource may indicate any of characteristics, properties, and expectations associated with the requested AR resource.

[0206] In an example, the positioning information associated with an ARD (e.g., ARD-171) may indicate any one of the current location of ARD-171, the (e.g., planned travel) path of ARD-171, and the (e.g., average) speed of ARD-171. For example, considering that ARA-1 may intend to design an AR scenario to be applied to a street between location B and location C, ARA-1 may intend to find ARRPs that may have overlapping travel paths and may provide valuable knowledge (e.g., AR resources) to ARA-1 to facilitate the AR scenario design performed by ARA-1. The average movement speed of ARD-1 (e.g., or an associated ARA-1) and the current location of ARD-1 may allow (e.g., to indicate to the ARRP) the level of urgency with which the ARRC (e.g., ARD-1) obtains the requested AR resources.

[0207] In an example, in step 706, the ARRSC may receive an AR resource sharing request from ARA-1. The ARRSC may identify an ARRP that can service the AR resource sharing request. For example, the ARRSC on ARD-1 may have (e.g., already) discovered RSC-1 and may know that RSC-1 may be able to coordinate AR resource sharing between one or more ARRPs and one or more ARRCs.

[0208] In this example, ARD-1 71 (e.g., an ARRSC running thereon) may send information 707 (which may be referred to herein as either an ARRP / ARRC matching request or an AR resource request) to, for example, a network element running an RSC (which may be referred to herein as RSC-1). As described with respect to step 705, the AR resource request 707 may include information indicating an identifier of ARD-1, a role of ARD-1, and any of the parameters indicated by the AR resource sharing request. The role may indicate either an ARRP role or an ARRC role. In this example, the role of ARD-1 may indicate that ARD-1 can operate as an ARRP.

[0209] In an example, in step 708, RSC-1 may receive an ARRP / ARRC matching request 707 from ARD-171 (e.g., ARRSC on ARD-171). For example, RSC-1 may check for registered ARRPs. For example, RSC-1 may determine that ARD-272 is an ARRP capable of servicing the request received from ARD-171 (e.g., ARRSC on ARD-171) based on, for example, determining that ARD-272 may be (e.g., currently) in the physical environment of interest and, for example, moving from location B to location C. For example, RSC-1 may (e.g., further) send information to the ARRP (e.g., ARD-272) indicating that an upcoming request may be received from the ARRC (e.g., ARD-171). For example, RSC-1 may send information to the ARRP (e.g., ARD-272) indicating that an upcoming request may be received from the ARRC (e.g., ARD-171). For example, RSC-1 may send information indicating an identifier of the ARRC and any one of characteristics (e.g., attributes) associated with the AR resource to be requested by the ARRC to the ARRP (e.g., ARD-272).

[0210] In an example, in step 709, ARD-1 (e.g., ARRSC on ARD-1) may receive an AR resource response from RSC-1, including, for example, first information for communicating with ARD-272 (e.g., available ARRSS hosted by ARD-272). For example, the first information may indicate how ARD-272 may be contacted to establish (e.g., establish) a connection. The first information for communicating with ARD-272 may indicate how ARD-272 may be contacted according to either of the following two examples (e.g., by ARD-171).

[0211] In a first example, the first information may (e.g., directly) indicate the contact point address of the ARRP (such as, for example, the IP address of the ARRP (e.g., the ARRSS of the ARRP)), so that the ARRC may be able to (e.g., directly) contact the ARRP (e.g., send information to the ARRP). Any other means for indicating the contact point (e.g., any kind of address, identifier, etc.) may be applicable to the embodiments described herein. In the first example, Figure 7A This is illustrated in step 709 of FIG.

[0212] In a second example, a monitoring / announcement technique may be used. For example, by utilizing a 3GPP D2D direct discovery mechanism (such as, for example, Model B direct discovery mode), ARD-1 as an ARRC may announce (e.g., transmit announcement information) on a (e.g., PC5) channel as a discovered party, and ARD-2 as an ARRP may monitor the announcements of the ARRC as a discoverer, so that ARD-1 and ARD-2 may retrieve (e.g., find) each other and may establish a connection. In the second example, Figure 13Transmitting and monitoring notification information on any kind of channel (eg, not limited to 3GPP PC5 channel) may be applicable to the embodiments described herein.

[0213] In the example, in step 710, ARD-171 may (e.g., begin) establishing a communication link with ARD-272. For example, ARD-171 may establish a device-to-device (D2D) communication link with ARD-272 based on, for example, information indicated by RSC-1. Any other techniques for establishing a communication link between ARD-171 and ARD-272 are applicable to the embodiments described herein. In the case where ARD-171 does not have cellular communication capabilities and is associated with another WTRU that has cellular communication capabilities (e.g., Lisa may carry a smartphone and AR glasses and Lisa's AR glasses may communicate with the other AR glasses via her smartphone, e.g., the AR glasses may be tethered ARDs), ARD-171 may rely on the associated smartphone to establish a communication link with ARD-272.

[0214] In an example, after communication may have been established, ARD-171 (e.g., ARRSC on ARD-171) may send an AR resource sharing request 711 to ARD-272 (e.g., ARRSS on ARD-272). For example, the AR resource sharing request 711 may include any of the parameters included in (e.g., indicated by) the AR resource request 707.

[0215] In an example, in step 712, the ARD-272 (e.g., ARRSS on the ARD-272) may evaluate the AR resource sharing request 711 and may determine whether to serve the request. For example, the ARD-272 (e.g., ARRSS on the ARD-272) may consider any of the AR hardware and software capabilities of the ARD-272, positioning information associated with the ARD-272 (e.g., any of the current location of the ARD-272 and the planned travel path of the ARD-272), the (e.g., current and expected) resource load on the ARD-272, MNO (or application) policies, and user preferences (e.g., based on which to determine whether to serve the request).

[0216] In the example, in step 713, if it is determined that ARD-272 (e.g., ARRSS on ARD-272) services the AR resource sharing request 711, an AR resource provisioning task may be created and assigned to ARP-2 on ARD-272. For example, ARP-2 may be instructed with detailed information, such as when and how to generate (e.g., collect, generate) the AR resources to be shared with the ARRC. For example, ARRSS may instruct (e.g., warn) the user of ARD-272 (e.g., via a user interface) that ARD-272 may capture video frames between position B and position C for sharing with another ARD so that the user can collaborate accordingly. The camera pose (e.g., position and / or orientation) may be determined by the movement of the user of ARD-272. In the event that the user is willing to share the AR resources, the user may prefer to stick to his / her currently planned path.

[0217] In an example, in step 714, ARD-272 (e.g., ARRSS on ARD-272) may determine (e.g., agree) to provide and share one or more AR resources with ARD-171 (e.g., ARRSC on ARD-171). For example, ARD-171 may transmit an AR shared resource response indicating any of the following information:

[0218] The AR shared resource response may indicate one or more types of AR resources to be shared by the ARD-272.

[0219] The AR shared resource response may provide any information details about how the AR resources may be shared by the ARD-272, such as, for example, QoS related information, resource availability scheduling information, any constraints, and the like.

[0220] The AR shared resource response may indicate any policies or rules that the ARD-272 may request (eg, expect) the ARD-171 to comply with.

[0221] For AR resource delivery (e.g., transmission), ARD-272 may operate according to any of the following examples. In a first example, the ARRP may deliver (e.g., transmit) the requested AR resources to the ARRC (e.g., directly and autonomously). In a second example, the ARD-171 may subscribe to the ARRP (e.g., send information indicating a subscription to the ARRP). In the event that the ARRP has video frames requested (e.g., subscribed) by the ARRC, the ARRP may send one or more notifications to the ARRC indicating that the ARRC may further obtain those video frames from the ARRP.

[0222] In an example, in step 715 , ARD- 171 (eg, ARRSC on ARD- 171 ) may indicate to ARA- 1 that the AR resource sharing request may have been accepted.

[0223] The process described herein can be considered an on-demand method for an ARRC (e.g., ARD-1) to locate an ARRP via an RSC. In another example, the RSC can (e.g., proactively) push notifications (e.g., information) to the ARRC indicating the availability of any ARRP(s). For example, ARD-1 and the RSC can communicate (e.g., via a connection) such that ARD-1 can (e.g., periodically and repeatedly) transmit information to the RSC indicating any positioning information updates (e.g., current location and planned travel route). For example, the RSC can be implemented by or co-located with an ARA server. Based on the positioning information updates, the RSC can (e.g., proactively) push (e.g., transmit) notifications (e.g., information) to the ARD-1 indicating the availability of (e.g., one or more) ARDs (which can also operate as ARRPs) in the physical environment X. The notifications (e.g., information) delivered to the ARRC can include sample data describing some of the ARRPs in detail. For example, a video frame captured from a candidate ARRP can be included in the notification and sent to the ARRC (e.g., ARD-1). For example, a user of ARD-1 may evaluate the sampled data to determine (eg, select) an ARRP among candidate ARRPs with which to cooperate.

[0224] In this article based on Figure 7A In the example described, ARD-2 can register as ARRP with RSC-1, and ARD-1 can register as ARRC with RSC-1. ARRP and ARRC are logical roles, and one (eg, physical) ARD can operate (eg, both) roles simultaneously.

[0225] Figure 7B is a diagram illustrating another example process for enabling cross-user collaborative relationship establishment. Figure 7B The example shown in Figure 7AThe difference from the example illustrated in is that (e.g., each) ARD can send information indicating its AR business logic to RSC-1, and the business logic can indicate at which time which kinds (e.g., (which type or types) of AR resources the ARD can request (if not locally available), and at which time which kinds (e.g., (which type or types) of AR resources the ARD can supply (e.g., for serving other ARDs). For example, RSC-1 can learn and analyze the business logic associated with the ARDs, and can dynamically assign roles (e.g., either ARRP or ARRC) to different ARDs, so that the paired ARRP and ARRC can (e.g., start) to build a collaborative relationship with each other.

[0226] In the example, in step 720, ARD-1 and ARD-2 may have discovered RSC-1. The physical ARD may operate as either (e.g., both) of ARRP and ARRC. The physical ARD may include (e.g., run) ARRSC and ARRSS. For example, when ARD-1 operates as ARRC, ARD-1 may use its ARRSC to interact with other network elements (e.g., ARDs), and when ARD-1 operates as ARRP, ARD-1 may use its ARRSS to provide one or more AR resources to other ARDs. For example, ARA-1 and ARA-2 may be associated with ARD-1 and ARD-2, respectively. ARA-1 and ARA-2 may include AR applications for designing customized AR scenarios. The designed (one or more) AR scenarios may be further deployed to the corresponding ARP for execution. For example, for ARD-1, ARP-1 may run one or more AR scenarios deployed on ARP-1, and for ARD-2, ARP-2 may run one or more AR scenarios deployed on ARP-2. For example, ARA-1 and ARA-2 may be tasked with designing one or more new customized AR scenes (e.g., new). In this setting, the ARA (ARA-1 or ARA-2) may request one or more AR resources, such as video frame resources, physical world knowledge resources, etc., to facilitate the design of its customized AR scene. For example, an ARP (ARP-1 and ARP-2) may supply (or consume) one or more AR resources. For example, an ARP may capture video frames from one of the ARD's built-in cameras, and those video frames may be shared with other ARDs. For example, an ARP may process video frames and generate physical world knowledge, and may share this knowledge with other ARPs. For example, an ARP may (e.g., also) benefit from one or more additional AR resources to enhance its AR processing to execute (e.g., render) the deployed AR scene. For example, such AR resources may include general resources, such as computing resources in the event that the ARP currently lacks sufficient computing resources and can request assistance from other ARDs (e.g., to offload some AR computing (e.g., processing) tasks to them).

[0227] In an example, in step 721, ARD-1 (e.g., ARRSC on ARD-1) may collect business processing logic from either ARP-1 or ARA-1. For example, the business processing logic may allow any of the following information fragments to be indicated:

[0228] In a first example, the business processing logic may indicate either of the computing and storage capacity of ARD-1.

[0229] In a second example, the business processing logic may indicate positioning information associated with ARD-1, such as any of a (eg, planned travel) path, travel speed, and current location, for example.

[0230] In a third example, business processing logic may indicate one or more AR scenarios that can currently be deployed and executed by ARP-1. For example, if ARP-1 is rendering an AR scene, its camera may be capturing video frames from the real world, which may be a type of AR resource that other ARDs may be interested in. For example, by analyzing planned travel path information, the ARRSC can understand what types of AR resources can be provided by ARD-1 (e.g., both now and in the future). For example, future locations can be estimated based on the planned travel path. For example, if ARD-1 is moving toward location X, the ARRSC can determine that ARP-1 can, for example, soon provide video frames for location X. In addition to supplying AR resources to other ARDs, the ARRSC can also determine what (e.g., what types of AR) resources can be used by ARP-1 (e.g., provided to ARP-1) to facilitate its AR processing. For example, if such resources are already locally available to ARD-1, ARP-1 can use them. If such resources are not locally available, ARP-1 can obtain (e.g., request) AR resources from other ARDs. For example, ARRSC may estimate any of the current and future workloads of ARP-1 based on the computation and storage capacity of ARD-1. For example, ARRSC may estimate that more computational resources may be available from other nearby ARDs when ARD-1 may later move to location Y. For example, ARP-1 may / may not be running any AR scenes (e.g., in an idle state), may capture video frames from any of its built-in cameras, and may supply (e.g., be able to supply) AR resources to other ARDs.

[0231] In a fourth example, the business processing logic may indicate (one or more) new AR scenes to be designed by ARA-1. For example, to design (one or more) new AR scenes, ARA-1 may request one or more AR resources. For example, based on ARD-1's current location and / or planned travel path, ARRSC may estimate what kinds of AR resources may be used by ARA-1 for present and future use. For example, if such AR resources are already locally available, ARP-1 may use them, and if such AR resources are not locally available, ARP-1 may obtain (e.g., desired) AR resources from other ARDs.

[0232] In an example, in step 722 , ARD-2 (eg, ARRSC on ARD-2 ) may operate as ARD-1 (eg, ARRSC on ARD-1 ) in step 721 .

[0233] In an example, in step 723, ARD-1 (e.g., ARRSC on ARD-1) may send a report request to RSC-1 indicating AR resource availability on ARD-1 and any of one or more requested ARs. For example, the report request may include any of the fragments of information indicated by the business processing logic information described in step 721. For example, the report request may include raw business logic information, which may be analyzed by RSC-1 to determine what kinds of resources (at what times) may be available at ARD-1 and what kinds of resources may be requested (at what times, etc.) by ARD-1. In another example, such analysis may be performed by ARD-1 itself. For example, the report request may include information resulting from the analysis of the business logic information, as described herein, thereby allowing the workload of RSC-1 to be reduced.

[0234] In an example, in step 724, RSC-1 may send an acknowledgement to ARD-1 (eg, ARRSC on ARD-1).

[0235] In an example, in step 725 , ARD-2 (eg, ARRSC on ARD-2 ) may operate as ARD-1 (eg, ARRSC on ARD-1 ) in step 723 .

[0236] In an example, in step 726 , RSC- 1 may send an acknowledgement to ARD- 2 (eg, ARRSC on ARD- 2 ).

[0237] In an example, in step 727, RSC-1 may analyze the business logic information received from an ARD (e.g., ARD-1) and determine what types of resources are available at ARD-1 (at what times) and what types of resources can be requested by ARD-1 (at what times, etc.). Based on the analysis, RSC-1 may (e.g., dynamically) pair different ARDs and assign them different roles (e.g., ARRP and / or ARRC). For example, RSC-1 may determine that ARD-1 can use one or more AR resources that may or may not be locally available at ARD-1. RSC-1 may determine that one or more AR resources may be available at ARD-2. RSC-1 may pair ARD-1 with ARD-2 and determine that ARD-1 can operate as an ARRC and ARD-2 can operate as an ARRP. For example, ARD-1 and ARD-2 may assist each other, for example, performing mutual assistance (e.g., AR resource provisioning). In this example, in the paired relationship, ARD-1 and ARD-2 may operate in (e.g., two) roles (e.g., ARRP and ARRC).

[0238] In an example, in step 728, RSC-1 may send first information to ARD-2 indicating that ARD-2 may pair with ARD-1 and that ARD-2 may operate as an ARRP to serve ARD-1. According to any embodiment described herein, the first information may further indicate any additional parameters describing the AR resources that may be requested by ARD-1.

[0239] For example, in step 729 ARD-2 may send an acknowledgment to RSC-1 indicating that ARD-2 agrees to operate as the ARRP for ARD-1 and to be contacted by ARD-1.

[0240] In this example, in step 730, RSC-1 may send second information to ARD-1 indicating that ARD-1 may pair with ARD-2 and that ARD-1 may operate as an ARRC to obtain one or more AR resources from ARD-2. The second information may further indicate how ARD-1 may contact ARD-2 or establish a connection with ARD-2. For example, ARD-2 may be contacted using either of the following two techniques:

[0241] In a first example of the technique, RSC-1 may transmit information indicating an address of a contact point of ARRP, such as, for example, an IP address of ARRP (e.g., ARRP's ARRSS), to ARRC so that ARRC may (e.g., directly) contact ARRP (via Figure 7A A first example of the technique is illustrated).

[0242] In a second example of the technology, a monitoring / notification technology may be used. For example, by utilizing a 3GPP D2D direct discovery mechanism (e.g., Model B direct discovery mode), ARRC may transmit information on the PC5 channel notifying ARRC as a discovered party, and ARRP may monitor the notification of ARRC as a discovered party, so that ARRC and ARRP may find each other and establish a connection ( Figure 13 A second example of the technique is illustrated in ).

[0243] In an example, in step 731 ARD-1 may send an indication of consent to RSC-1 as confirmation of the ARRC operation and may (eg, be initiated to) contact ARD-2.

[0244] In an example, in step 732 , a communication link may be established between ARD- 1 and ARD- 2 .

[0245] In an example, ARD-1 (eg, ARRSC on ARD-1) may send an AR resource sharing request 733 to ARD-2 (eg, ARRSS on ARD-2). Figure 7B The AR resource sharing request 733 described may correspond to Figure 7A AR resource sharing request 711 is described.

[0246] In an example, in step 734, ARD-2 (eg, ARRSC on ARD-2) may evaluate the request and may determine whether to service the request. Step 734 may correspond to using Figure 7A Step 712 is described.

[0247] In an example, in step 735, ARD-2 (eg, ARRSS on ARD-2) may assign the AR resource provisioning task to ARP-2. Step 735 may correspond to Figure 7A Step 713 described.

[0248] In an example, in step 736, ARD-2 (eg, ARRSS on ARD-2) may transmit information to ARD-1 indicating agreement to share the requested AR resource with ARD-1. Step 736 may correspond to Figure 7A Step 714 is described.

[0249] Example of Type-1 raw video frame sharing

[0250] In Type-1 sharing, an ARRC may obtain raw AR resources captured in a (e.g., unknown) physical environment, which may be shared by an ARRP having knowledge of the physical environment. Embodiments are described herein with examples of video frames as (e.g., raw, original) AR resources to be shared. The embodiments described herein are not limited to video frames as (e.g., raw, original) AR resources, and may be used to share any type of (e.g., raw, original) AR resources provided by an ARRP, such as sensory data, audio, etc. generated by various sensors available on the ARRP (e.g., radar).

[0251] Figure 8 is a diagram illustrating an example process for enabling sharing of raw video frames. In the example, ARD-1 (e.g., ARRSC on ARD-1) and ARD-2 (e.g., ARRSS on ARD-2) may have used, for example Figure 7A and / or Figure 7B The process described in

[15] establishes a collaborative relationship and establishes a communication connection. For example, ARP-2 may have been assigned an AR resource provisioning task (e.g., ARRSC on ARD-1) for serving ARD-1. Figure 7A In this example, the AR resource to be shared may include one or more original video frames of ARD-2.

[0252] In an example, in step 800 , ARD-1 may be at location A, moving toward a (e.g., unknown) physical environment X, such as, for example, a street between location B and location C. ARD-2, for example, which is an ARRP, may be located in physical environment X, for example, moving between location B and location C.

[0253] Figure 7A and / or Figure 7B The process for establishing a cooperative relationship between ARRP and ARRC as illustrated in FIG can be adapted for use in this example. For example, since ARRP and ARRC can establish a cooperative relationship, the following adjustments can be made to the process included in FIG. Figure 7A The following two parameters in step 705 described in (eg, in the AR resource sharing request 707):

[0254] The parameter indicating one or more types of AR resources that may be requested (eg, by ARA-1) may indicate a Type-1 type, eg, one or more original video frames captured by ARRP may be requested as AR resources.

[0255] The information associated with the requested AR resource may indicate any one of the following parameters.

[0256] In a first example, the information associated with the requested AR resource may include any geographic information indicating (e.g., describing) the physical environment X (e.g., the street between location B and location C) from which the ARRP may be requested to capture video frames.

[0257] In a second example, information associated with the requested AR resource may indicate a frame capture frequency, which may indicate how quickly a built-in camera that may request ARRP may capture video frames from the physical environment X. For example, the camera may capture one image every three seconds.

[0258] In a third example, information associated with the requested AR resource may indicate the quality or resolution of the video frame, which may indicate what quality of video frame may be requested by the ARRC. Based on this information, the ARRP may determine what type of built-in camera may be used to capture the image.

[0259] In a fourth example, information associated with the requested AR resource may indicate a camera pose preference, which may indicate a preference for the ARRP's camera pose when capturing video frames. For example, the ARRC may expect (e.g., request) the ARRP to capture high-quality zoomed-in pictures of buildings on both sides of a street. For example, the ARRP may occasionally change its camera pose in order to capture the video frames expected (e.g., requested) by the ARRC.

[0260] In an example, in step 801, ARD-2 (e.g., ARD-2's ARP-2) may capture the requested video frames from the physical world based on any of ARD-2's cameras, since ARD-2 may move between location B and location C. ARD-2 (e.g., ARD-2's ARP-2) may perform some pre-processing to reduce the size of the original video frames. For example, if the ARRC indicates that it is not interested in pedestrians / sky, ARD-2 (e.g., ARD-2's ARP-2) may remove certain regions from one or more original video frames to reduce some communication costs (e.g., overhead) for delivering those original video frames from the ARRP to the ARRC.

[0261] In an example, in step 802, ARD-2 (e.g., ARP-2 of ARD-2) may deliver (e.g., transmit) the captured video frame(s) to the ARRSS (e.g., the ARRSS on ARD-1). For example, ARD-2 (e.g., ARP-2 of ARD-2) may deliver the video frames to the ARRSS continuously based on a preconfigured capture frequency. In another example, ARD-2 (e.g., ARP-2 of ARD-2) may accumulate one or more video frames and may deliver (e.g., transmit) them to the ARRSS in a batch (e.g., burst) manner. For example, ARD-2 (e.g., ARP-2 of ARD-2) may deliver the video frames to the ARRSS and include any of the following parameters:

[0262] In a first example, the video frame may be transmitted along with information indicating a list of video frames captured by ARD-2 (eg, ARP-2 of ARD-2).

[0263] In a second example, (e.g., each) captured video frame may be associated with information indicating any of: (i) the size of the video frame, (ii) the pose of the camera, which may include any of the position and orientation of the camera of the ARRP, and (iii) the sensor (e.g., camera) used to capture the video frames. For example, a high-resolution image may be captured by a red, green, blue (RGB) image capture camera, and an image with depth information may be captured by a depth camera.

[0264] In a third example, the video frame(s) may be transmitted along with information indicating other types of sensory data that may be captured and shared.

[0265] In an example, in step 803, the ARRSS (on ARD-2) may send an acknowledgement of the received video frame(s) to ARP-2.

[0266] For example, ARD-2 (e.g., ARRSS on ARD-2) may deliver (e.g., transmit) the requested AR resource(s) 804 as (e.g., original, raw) video frames to ARD-1 (e.g., ARRSC on ARD-1) based on the communication link established between ARD-1 and ARD-2. For example, the ARRSC on ARD-1 (as an ARRC) may establish a cross-user collaboration relationship with multiple ARDs (as ARRPs). For example, during the collaboration relationship establishment phase, ARD-1 may subscribe to more than one ARRP. For example, in the event that the ARRPs have video frames requested by the ARRCs, they may send a notification to the ARRCs so that the ARRCs may further obtain those video frames from the ARRPs (which may have already notified the ARRCs).

[0267] In an example, ARD-1 (e.g., ARRSC on ARD-1) may send an acknowledgment 805 to ARD-2 (e.g., ARRSS on ARD-2) for the received AR resource(s) 804 (such as, for example, video frame(s) or any other AR resource(s)).

[0268] In an example, in step 806, ARD-1 (e.g., ARRSC on ARD-1) may determine which video frames ARA-1 may expect. For example, ARRSC may deliver to ARA-1 the newly received video frame(s) shared by ARD-2. In an example, ARRSC may send to ARA-1 a notification regarding (e.g., indicating) the newly available video frames.

[0269] In an example, based on application logic, ARA-1 may process the received video frames (one or more) to extract physical world knowledge in step 807. For example, ARA-1 may use those video frames to detect (one or more) PTOs in physical environment X. In another example, ARA-1 may use those video frames to perform SLAM-related operations (e.g., generate a 3D map) to understand physical environment X.

[0270] In an example, ARA-1 may forward the video frame(s) along with processing instructions on how to extract physical world knowledge to ARP-1 in step 808. In an example, ARA-1 may send only (e.g., only) processing instructions to ARP-1 and may request ARRSC to deliver the video frames to ARP-1 for processing.

[0271] In an example, in step 809, the video frame(s) shared by ARD-2 may be processed by ARP-1 on ARD-1, and physical world knowledge may be extracted. For example, physical world knowledge may refer to (e.g., include, indicate) semantic perception of physical environment X via AI-enabled object detection and recognition (e.g., what POs may be in environment X, and what their characteristics may be, such as, for example, any of their 3D position, color, shape, orientation, brightness, natural features, etc.). For example, physical world knowledge may refer to (e.g., include, indicate) generated map information of physical environment X.

[0272] In an example, in step 810 , the extracted physical world knowledge may be returned to ARA-1.

[0273] In this example, in step 811, based on physical world knowledge about physical environment X, ARA-1 can dynamically design a customized AR scene for the user of ARD-1. For example, later, after the user of ARD-1 may have entered physical environment X, the designed AR scene can be executed or rendered to deliver a rich AR experience to the user. As described herein, designing an AR scene can involve two tasks:

[0274] In the first task, based on the knowledge of the physical world, ARA-1 can determine which detected POs may be PTOs that can be enhanced with AR effects.

[0275] In a second task, ARA-1 can design customized AR effects for (one or more) PTOs, which may include: a) determining which (one or more) VOs can be superimposed on (one or more) PTOs (e.g., (one or more) VOs can be downloaded), and b) configuring either the appearance of (one or more) VOs and the behavior of (one or more) VOs, for example, in the case of customized behavior.

[0276] In an example, in step 812, RSC-1 may monitor the sharing of video frames between ARRC and ARRP and may perform one or more operations based on one or more changes. For example, in the event that ARRP changes its movement path, RSC-1 may identify another (e.g., qualified) ARRP for serving ARRC. In another example, based on the received video frames, ARP-1 may evaluate the quality of the received video frames and may notify ARRP, for example, if an adjustment may be requested. For example, ARP-1 may dynamically determine which next video frames may be requested and may send a notification indicating new AR resource attributes (e.g., any of a new camera pose, a new resolution, etc.) to ARD-2.

[0277] The embodiments described herein are based on implementation examples involving separate modules for ARA and ARRSC and ARP. The embodiments may be applicable to other types of ARD embodiments with, for example, a single module, in which case the messages / information flowing between the described modules of the ARD may / may not be used.

[0278] Example of Type-2 Physical World Knowledge Sharing

[0279] In Type-2 sharing, the ARRC may intend to obtain, for example, physical world knowledge extracted from AR resources (e.g., video frames) from the ARRP, which can be captured by the ARRP. For example, depending on how the physical world knowledge can be utilized by the ARRC, many different processes are described in this article.

[0280] Example of cross-user collaborative AR processing for physical object detection and recognition

[0281] The following scenario is described herein: ARA-1 may expect an ARRP (e.g., ARD-2) to help it detect and identify one or more POs of interest in a (e.g., unknown) physical environment X (e.g., the street between location B and location C), which may be PTOs to be annotated with one or more AR effects. Such information may be used by ARA-1 to design a customized AR scenario for the user of ARD-1. The AR scenario may be executed when the user may later travel in the physical environment X. For example, during the cross-user collaborative relationship establishment phase (with Figure 7A During a request (e.g., during an AR resource sharing request 711 described in

[15] ), ARD-1 (e.g., ARA-1 on ARD-1) may indicate to an ARRP (e.g., ARD-2) that ARD-1 may rely on the ARRP to detect and identify one or more POs in physical environment X by analyzing video frames captured by the ARRP's built-in camera and any other type of AR resource. For example, ARD-1 (e.g., ARA-1 on ARD-1) may (e.g., further) indicate to the ARRP that ARD-1 may rely on the ARRP to extract map-related information about physical environment X, allowing ARD-1 to / may not create a map from scratch. For example, an accurate map may allow ARD-1 to accurately estimate the pose of its own camera, allowing VOs to be (e.g., perfectly) superimposed on the physical world. Based on sharing physical world knowledge about physical environment X (which may still be unknown to ARD-1), ARA-1 may, for example, gain pre-awareness and familiarity with physical environment X (e.g., what physical objects may be present) before ARD-1 can even reach physical environment X. For example, knowing the physical environment X in advance may allow ARD-1 to have more time to prepare and design a customized AR scene (which may be rendered to the user of ARD-1 when ARD-1 may later be in the physical environment X).

[0282] Figure 9 is a diagram illustrating an example process for enabling physical world knowledge sharing for physical object detection and recognition.

[0283] In the example, in step 900, ARD-1 may have identified ARD-2 via RSC-1, and ARD-1 (e.g., ARRSC on ARD-1) and ARD-2 (e.g., ARRSS on ARD-2) may have established a connection for AR collaboration. For example, ARP-2 may have been assigned an AR resource provisioning task (e.g., such as Figure 7A(described in step 713 of

[0066] ). In this example, the one or more AR resources to be shared may include physical object identification results extracted from one or more video frames captured by ARD-2. For example, ARD-1 may be located at location A and may be moving toward a (e.g., unknown) physical environment X (e.g., a street between locations B and C). For example, ARD-2, as an ARRP, may be located within physical environment X, for example, moving between locations B and C.

[0284] For example, Figure 7A and / or Figure 7B The process for establishing a cooperative relationship between ARRP and ARRC as illustrated in FIG can be adapted for use in this example. For example, since ARRP and ARRC can establish a cooperative relationship, the following adjustments can be made to the process included in FIG. Figure 7A The following two parameters in step 705 described in (eg, in the AR resource sharing request 707):

[0285] The parameter indicating one or more types of AR resources that may be requested (e.g., by ARA-1) may indicate a Type-2 type, such as physical world knowledge about physical environment X, e.g., based on a PO identification result. For example, ARA-1 may request ARD-2 to provide information about a map of physical environment X.

[0286] The information associated with the requested one or more AR resources may indicate a PO identification guide, which may indicate what kind of information may be returned to the ARRC to describe the PO if the PO is identified by the ARRP in the physical environment X. For example, the PO identification guide may indicate that the following information may be requested by the ARRC to describe the PO, including any one of the PO's 3D position, orientation, shape, color, brightness, movement speed, detection time, natural features (edge ​​geometry, point cloud, etc.), etc.

[0287] In the example, in step 901, based on the instructions of the assigned AR resource provisioning task (e.g., indicating when / how to provision / share (one or more) AR resources to the ARRC), the ARD-2 (e.g., the ARP-2 on the ARD-2) may obtain (e.g., capture) (one or more) video frames from the physical world using any one of the built-in cameras of the ARD-2 moving between position B and position C.

[0288] In an example, in step 902, ARD-2 (e.g., ARP-2 on ARD-2) may analyze (one or more) video frames for semantic perception of the physical world. For example, ARD-2 (e.g., ARP-2 on ARD-2) may detect and identify one or more POs. For example, ARD-2 (e.g., ARP-2 on ARD-2) may (e.g., further) perform SLAM-related operations to generate a map of the physical environment X (including the geometry and 3D model of the physical world).

[0289] In an example, in step 903 , ARD-2 (eg, ARP-2 on ARD-2) may generate a PO identification result, which may be expected by ARD-1 (eg, ARRSC on ARD-1) and the corresponding ARA-1.

[0290] In an example, in step 904 , ARD-2 (eg, ARP-2 on ARD-2) may deliver the PO identification result to the ARRSS.

[0291] In an example, in step 905, ARRSS may send an acknowledgement of the received PO identification result to ARP-2.

[0292] In an example, ARD-2 (e.g., ARRSS on ARD-2) may deliver (e.g., transmit) AR resource(s) 906 as PO identification results to ARD-1 (e.g., ARRSC on ARD-1). For example, for (e.g., each) identified PO, the PO identification result may indicate any of the PO's 3D position, orientation, shape, color, brightness, movement speed, detection time, natural features (edges of any of geometric shapes, point clouds, etc.), etc. In another example, during the collaboration establishment phase, ARD-1 (e.g., ARRSC on ARD-1) may transmit information indicating what types of POs may be requested by executing a subscription to ARD-2 (e.g., ARRSS on ARD-2). For example, if one or more POs of interest are identified, ARD-2 (e.g., ARRSS on ARD-2) may (e.g., only) send a notification to ARD-1 (e.g., ARRSS on ARD-1).

[0293] In an example, ARD-1 (eg, ARRSC on ARD-1) may send an acknowledgement 907 to ARD-2 (eg, ARRSS on ARD-2).

[0294] In an example, in step 908 , the ARRSC may notify ARA- 1 of the PO identification result of the unknown physical environment X.

[0295] In an example, in step 909 , ARA- 1 may send an acknowledgement to the ARRSC.

[0296] Steps 910 - 912 relate to how ARA-1 can perform dynamic and customized AR scene design based on the received knowledge shared by ARD-2.

[0297] In an example, in step 910, ARA-1 may analyze the PO identification results and may determine which POs may be PTOs that can be enhanced with one or more AR effects. For example, the determination may be made based on one or more factors, such as, for example, user preference, implementation complexity, application logic, etc.

[0298] In an example, in step 911, after one or more PTOs may have been determined, one or more AR effects may be determined. For example, for a PTO, it may be determined what type of VO may be used to annotate the PTO. This process may be customized. For example, different users may prefer to use different VOs for the PTO. For example, ARD-1 may / may not be able to store (e.g., all) VOs locally (due to storage limitations). For example, ARD-1 may (e.g., proactively) initiate a VO retrieval process and may pre-download the selected VO(s) (e.g., before ARD-1 may arrive in physical environment X).

[0299] In the example, in step 912, after one or more VOs may have been determined and downloaded, ARA-1 may configure the appearance of the VO(s), such as any of their color, brightness, etc. ARA-1 may further configure what kinds of actions or behaviors the VO(s) may have.

[0300] Steps 913-918 relate to how ARD-1 may deploy and implement the customized AR scene designed by ARD-1 so that AR effects may be displayed to the user of ARD-1 as ARD-1 may move in physical environment X.

[0301] In an example, in step 913, for example, later on, ARA-1 may approach physical environment X, for example, approach location B (e.g., Figure 3 ).

[0302] In an example, in step 914 , ARA-1 may deploy the customized AR scene to ARP-1 for execution and rendering. For example, ARA-1 may indicate the following information to ARP-1.

[0303] In an example, a detailed AR scene may be indicated to ARP-1, which may include what kind of PTOs may be annotated. To facilitate ARP processing, information about the PTOs may be included, such as, for example, any of the PTO's position, color, shape, geometry, orientation, image, etc. The detailed AR scene may further include information about how to annotate (e.g., each) PO, for example, what VO to use. The detailed AR scene may further include (e.g., complete) files of VO assets and their corresponding customized configuration details, such as any of the PO's appearance, behavioral design, etc.

[0304] In this example, ARA-1 may indicate to ARP-1 the applicable environment of the AR scenario to be applied. This may indicate, for example, when the AR scenario is to be executed. For example, the AR scenario may be executed when the AR is in physical environment X. Physical environment X may be described using geographic location and / or coordinates.

[0305] In an example, ARA-1 may indicate one or more triggers (e.g., conditions) for executing an AR scenario to ARP-1. For example, referring to the example described herein, where physical environment X may refer to a street between location B and location C, based on GPS location information, the trigger (e.g., condition) may be that the AR scenario may begin when the ARD is located at location B.

[0306] In an example, in step 915 , ARP- 1 may send confirmation of the AR scene deployment to ARA- 1 .

[0307] In the example, in step 916, where ARD-1 may be located in (e.g., and moving in) physical environment X (e.g., a street between location B and location C), video frames may be captured by ARD-1 itself (e.g., a built-in camera of ARD-1).

[0308] In this example, in step 917, ARP-1 may or may not process video frames and render AR scenes from scratch by leveraging the physical world knowledge received from ARD-2 (such as, for example, either PTO identification results or SLAM-related map information). For example, with the map information (e.g., physical world knowledge) shared by ARRP, ARP-1 can more quickly understand physical environment X. For example, ARP-1 can also use its own video frame(s) (captured by any of its own cameras) to further improve the accuracy of the map based on its own video frame(s) (captured by any of its own cameras). For example, for the video frame(s) captured by ARD-1 itself, ARP-1 can reduce the time required to verify the presence of PTO(s) in those video frames and the accuracy of the PTO identification results shared by ARD-2 (e.g., ARRP). In this way, ARP-1 can use less time and computing resources to understand physical environment X. For example, based on ARP-1 already knowing information about one or more PTOs (including any of 3D position, orientation, shape, color, natural features, etc.), ARP-1 can more quickly (e.g., efficiently) detect and locate the PTO in the current video frame captured by ARD-1. For example, ARD-1 may include odometry (e.g., positioning) capabilities to measure (e.g., determine) the geographic location of PTOs appearing in its current viewport. For example, the PTO identification results shared by ARD-2 may / may not be 100% accurate. Therefore, the AR scene may be calibrated (e.g., adjusted).

[0309] In an example, in step 918, ARP-1 may render using the (e.g., calibrated, adjusted) AR scene, for example, by annotating the PTO(s) appearing in the video frame(s) captured by ARD-1 via VO registration based on the prepared VO(s), so that the AR effect may be displayed to the user of ARD-1.

[0310] With the help of Figure 9 The steps 910-918 described can be viewed as Figure 8 Detailed description of step 811 described. In an example, steps 910-918 may be applicable to dynamic design of a customized AR scene based on (one or more) received AR assets as (one or more) original video frames.

[0311] Example of cross-user collaborative AR processing for knowledge accuracy and freshness verification

[0312] The following scenario is described herein. ARA-1 may approach and may soon arrive in physical environment X. Based on application logic, as ARD-1 may move through physical environment X (e.g., the street between location B and location C), ARA-1 may design a customized AR scenario for the user of ARD-1. For example, ARA-1 may (e.g., already) have some knowledge about physical environment X, such as, for example, a map of physical environment X and any of the Point of View (POs) in that physical environment X. For example, ARA-1 may have downloaded knowledge from a centralized world knowledge repository. In another example, ARD-1 may have traveled through physical environment X at an earlier time (e.g., a month ago). In yet another example, ARD-1 may have received physical world knowledge information from another ARD that traveled through physical environment X at an earlier time. For example, the physical world may be constantly changing, such that the physical world knowledge held by ARA-1 may / may not be valid or accurate, e.g., at least some of the knowledge may be outdated. For example, one or more Points of View (e.g., mobile Points of View) may / may no longer exist in physical environment X. For example, the characteristics of the PO (e.g., also) may change from time to time (e.g., the windows of a pizza shop may have neon lights during night time, which may be different from its appearance during day time). For example, ARA-1 can use the assistance of ARRP to verify the accuracy and freshness of the physical world knowledge currently held by ARA-1, so that the latest knowledge can be used to perform customized AR scene design.

[0313] Figure 10 is a diagram illustrating an example process for enabling physical world knowledge accuracy and freshness verification.

[0314] In step 1000, ARD-1 may be located at location A and may be moving toward physical environment X (e.g., a street between location B and location C). ARD-2, which may operate as an ARRP, may be located inside physical environment X, e.g., moving between location B and location C. ARD-1 may have (e.g., already) identified ARD-2 via RSC-1, and ARD-1 (e.g., ARRSC on ARD-1) and ARD-2 (e.g., ARRSS on ARD-2) may have used, e.g., Figure 7A and / or Figure 7B For example, ARP-2 may have been assigned the AR resource provisioning task during the collaborative relationship establishment phase (e.g., Figure 7A In this example, one or more video frames may be captured and shared by ARD-2 for the purpose of verifying the accuracy and freshness of ARD-1's physical world knowledge.

[0315] For example, Figure 7A and / or Figure 7B The process for establishing a cooperative relationship between ARRP and ARRC as shown in FIG can be adapted for use in this scenario. For example, since ARRP and ARRC can establish a cooperative relationship, the following adjustments can be made to include the following: Figure 7A The following two parameters in step 705 described in (eg, in the AR resource sharing request 707):

[0316] The parameters indicating one or more types of AR resources that may be requested (eg, by ARA-1) may indicate that accuracy and freshness verification results of physical world knowledge held by the ARRC may be requested.

[0317] The information associated with the requested AR resource may indicate the verification result format. For example, the information may indicate what kind of update information may be requested if the characteristics of the PO (e.g., any of the location, shape, color, orientation, natural features, etc.) have changed, such as, for example, any of the updated location, shape, color, etc. For example, the information may also indicate that the ARRC may also request the ARRP to provide the latest close-up image of the PO.

[0318] In an example, in step 1001 , ARA- 1 may determine that there may be some locally stored available knowledge about the physical environment X, such as, for example, any one of characteristic information about POs in the physical environment X and a map about the physical environment X.

[0319] In an example, in step 1002, ARA-1 may determine that knowledge about physical environment X may or may not be accurate or may be outdated. For example, ARA-1 may obtain (e.g., download) knowledge from a knowledge repository and mark the knowledge with an old (e.g., outdated) timestamp (indicating that the knowledge may be outdated). In another example, ARA-1 may have traveled through physical environment X some (e.g., a long) time ago. To design the AR scene, ARA-1 may use the latest and most accurate knowledge of physical environment X.

[0320] In an example, in step 1003 , ARA- 1 may ask ARRSC to verify the accuracy and freshness of its physical world knowledge about environment X.

[0321] In an example, ARD-1 (e.g., ARRSC on ARD-1) may send a physical world knowledge verification request 1004 to ARD-2 (e.g., ARRSS on ARD-2). For example, physical world knowledge verification request 1004 may indicate a list of point of interest (PO) characteristic information to ARD-2 (e.g., ARRSS on ARD-2). For example, for a PO, its characteristic information may indicate any of its location, shape, color, physical features, and orientation, which may be known to the ARRC and verifiable by the ARRP. In another example, if ARD-1 (e.g., ARRSC on ARD-1) frequently requests ARD-2 (e.g., ARRSS on ARD-2) to verify a list of POs, a versioning approach may be used. For example, for (e.g., each) PO verification result sent from ARD-2 (e.g., ARRSS on ARD-2) to ARD-1 (e.g., ARRSC on ARD-1), ARD-2 (e.g., ARRSS on ARD-2) may attach (e.g., include, indicate) a version number to the PO verification result. For example, at a later time, ARD-1 (e.g., ARRSC on ARD-1) may (e.g., request) re-verification of whether the PO verification result currently held by ARD-1 (e.g., ARRSC on ARD-1) is (e.g., still) fresh and / or accurate. ARD-1 (e.g., ARRSC on ARD-1) may indicate the version number of the PO verification result to ARD-2 (e.g., ARRSS on ARD-2), and (e.g., all) detailed description characteristics information may / may not be re-sent for each PO in the PO list. This may allow for reduced communication overhead. For example, if the PO verification result held by ARD-1 (e.g., ARRSC on ARD-1) is fresh or accurate, ARD-2 (e.g., ARRSS on ARD-2) may reply that the current version is valid (e.g., send information indicating that the current version is valid). Otherwise, ARRSS may return a new PO verification result to ARD-1 (e.g., ARRSC on ARD-1) (e.g., send information indicating the new PO verification result), and the new PO verification result may be associated with a new version number.

[0322] In an example, in step 1005 , the ARRSS may forward the physical world knowledge verification request to ARP- 2 .

[0323] In an example, in step 1006 , ARP-2 may capture video frame(s) from the physical world using any of its built-in cameras, for example, as ARP-2 may move in physical environment X (e.g., a street between location B and location C).

[0324] In an example, in step 1007, ARP-2 may analyze its video frame(s) and may check the physical world knowledge received from ARD-1. For example, ARP-2 may verify the presence and state (e.g., characteristics) of the PO as described (e.g., indicated) in the physical world knowledge received from ARD-1. For example, ARP-2 may (e.g., further) perform a new round of SLAM to update the map of the physical environment X.

[0325] In an example, in step 1008, ARP-2 may update and rectify the physical world knowledge received from ARD-1, e.g., in the event that the updated map of the physical environment differs from the physical world knowledge received from ARD-1, so that pieces of such knowledge may become fresh and accurate again.

[0326] In this example, in step 1009, ARP-2 may return a knowledge verification result (e.g., a response), which may include revised and updated physical world knowledge. For example, if ARP-2 determines that a first knowledge segment is still valid and accurate, the first knowledge segment may or may not be returned (e.g., sent back) to ARD-1, which may already hold the knowledge. If a second knowledge segment is updated, information indicating the updated second knowledge segment may be returned (e.g., transmitted) to ARD-1, allowing ARD-1 to update, for example, the AR scene design.

[0327] In an example, ARD-2 (e.g., ARRSS on ARD-2) can return (e.g., send information indicating the physical world knowledge verification result 1010) the physical world knowledge verification result 1010 to ARD-1 (e.g., ARRSC on ARD-1). For example, for a given PO to be verified, the following information can be included in the physical world knowledge verification result 1010.

[0328] For example, the physical world knowledge verification result 1010 may indicate whether the PO information received from ARD-1 (eg, ARRSC on ARD-1) is likely fresh or accurate. If not, the updated PO information may be included in the physical world knowledge verification result 1010.

[0329] For example, in the case of a version control method (e.g., as described for sending the physical world knowledge verification request 1004), the physical world knowledge verification result 1010 may indicate whether the current version of the PO information (e.g., held by the ARRSC) is valid. If not, a new version number may be indicated in the physical world knowledge verification result 1010, and the updated PO information of the new version may be included in the physical world knowledge verification result 1010.

[0330] For example, in step 1011, ARRSC may further forward the knowledge verification result to ARA-1. For example, if the knowledge verification result includes any knowledge update or revision, ARA-1 may update its physical world knowledge about environment X.

[0331] For example, in step 1012, ARA-1 may use the verified (e.g., updated) physical world knowledge to further design a customized AR scene, which may be used as a basis for ARD-1 to move in the physical environment X. Step 1012 may include Figure 9 Steps 910-914 described have similar features.

[0332] Example of cross-user collaborative AR processing for advance notification about mobile PO appearance

[0333] The following scenario is described herein: ARD-1 may be in a physical environment M (e.g., location A), and ARD-2 may be in another physical environment N (e.g., at location B). The two physical environments M, N may be close to each other, for example, interconnected via a road (e.g., a street). For example, one or more mobile POs may appear in the viewport of ARD-1. Examples of mobile POs may include vehicles traveling in the physical environment M. For example, (one or more) mobile POs may appear in the viewport of ARD-1 and then leave the viewport, for example, in an incoming and outgoing manner. Based on application logic, ARD-1 may overlay one or more AR effects onto those mobile POs, which may be displayed to the user of ARD-1. For example, (one or more) mobile POs may appear in the viewport of ARD-1 (e.g., only) for a limited period of time, such as, for example, five to ten seconds. ARA-1 can perform AR design and deployment, which may include identifying (one or more) mobile points of view (POs), determining whether the mobile points of view (POs) can be PTOs, determining the AR effects to be applied to the mobile PTOs, preparing (e.g., downloading) voice-activated video (VOs) (if not available locally), and overlaying the VOs onto the PTOs when the PTOs appear in ARD-1's viewport. Such AR processing with strict time constraints can be based on ARA-1 having access to high-performance computing capabilities, which may / may not be available (e.g., for resource-constrained ARDs). The embodiments described herein allow this challenge to be addressed from a different perspective by leveraging collaborative AR processing between different ARDs. The embodiments described herein utilize that mobile points of view (POs) can travel between different adjacent physical environments (e.g., physical environment M and physical environment N). For example, a vehicle appearing (e.g., traveling) at location B may arrive at location A very quickly, e.g., a minute or two later. The embodiments described herein propose that different ARDs can collaborate to exchange information about mobile points of view (POs), allowing the ARDs to have more time to design (e.g., prepare) the AR effects to be overlaid onto those mobile points of view, e.g., before the mobile points of view appear in their built-in camera's viewport. For example, ARD-1 (e.g., ARA-1 on ARD-1) (at location A) may utilize a video frame captured by another nearby ARD-2 (at location B) in the following manner: in the event that ARD-2 (operating as an ARRP) identifies a mobile PO that may be heading toward (e.g., a certain) direction (e.g., toward location A), ARD-2 may, for example, send a pre-notification indicating the detected mobile PO to ARD-1 before the mobile PO may appear in ARD-1's viewport. For example, ARD-1 may know in advance what kind of mobile PO may appear in its own viewport (e.g., soon) and may have more time to create and prepare AR effects to be overlaid on the mobile PO.

[0334] Figure 11 is a diagram illustrating an example process for enabling pre-notification associated with a mobile PO appearance.

[0335] For example, ARD-1 may be located in physical environment M (e.g., at location A), and ARD-2 may be located in physical environment N (e.g., at location B), which may not be far from physical environment M (e.g., adjacent to each other). ARD-1 and ARD-2 may be aware of the existence of RSC-1, which may be responsible for AR-related resource sharing coordination.

[0336] In this example, the cooperation establishment procedure between ARRP and ARRC may / may not be performed. For example, RSC-1 may receive pre-notifications from ARRP and may dispatch them to the appropriate ARRC.

[0337] In an example, ARP-2 may capture (one or more) video frames using any of its ARD-2's (e.g., built-in) cameras in step 1101. For example, ARD-2 may be on a busy street with a large number of mobile POs.

[0338] In an example, in step 1102, ARP-2 may analyze the video frame(s) and may perform image processing, such as, for example, object recognition. For example, ARP-2 may detect and recognize one or more mobile POs.

[0339] In an example, in step 1103, ARP-2 may generate a mobile PO identification report that may include (e.g., all) identified mobile POs. For example, the mobile PO identification report may include a detailed description of (e.g., each) PO, such as any one of type (e.g., car, food truck, or double-decker bus), color, shape, moving speed, moving direction, etc.

[0340] In an example, in step 1104, ARP-2 may deliver the mobile PO identification report to the ARRSS.

[0341] In an example, in step 1105, ARRSS may send an acknowledgment to ARP-2.

[0342] In an example, the ARRSS may determine to share the mobile PO identification report with other ARDs that may benefit from the report in step 1106. For example, ARD-2 (e.g., the ARRSS on ARD-2) may send information indicating the mobile PO identification report to RSC-1.

[0343] In an example, in step 1107 , RSC- 1 may send an acknowledgement to ARD- 2 (eg, ARRSS on ARD- 2 ).

[0344] In an example, RSC-1 may analyze the mobile PO identification report in step 1108. For example, for (eg, each) identified mobile PO, RSC-1 may check any one of its moving speed and moving direction.

[0345] In an example, in step 1109 , for (e.g., each) PO, RSC-1 may estimate its upcoming impact on the viewports of other ARDs, e.g., RSC-1 may estimate which mobile PO may (e.g., soon) appear in the viewports of which ARDs (e.g., ARD-1 in the adjacent physical environment M). For example, those ARDs (including ARD-1) may be notified about the upcoming appearance of the PO.

[0346] In an example, in step 1110, RSC-1 may send a mobile PO pre-notification to ARD-1 (e.g., ARRSC on ARD-1), along with (e.g., including detailed description of characteristic information of) (one or more) mobile POs. In another example, ARD-2 (e.g., ARRSS on ARD-2) may receive information from RSC-1 indicating which ARDs may be notified about the upcoming appearance of one or more POs. For example, ARD-2 (e.g., ARRSS on ARD-2) may send a mobile PO pre-notification to an ARD (e.g., ARD-1) (e.g., ARRSC on ARD). More generally, ARD-1 may subscribe (e.g., send information indicating the subscription) and may receive pre-notifications for (one or more) POs of interest from multiple ARDs.

[0347] In an example, in step 1111 ARD- 1 (eg, ARRSC on ARD- 1 ) may send an acknowledgement to RSC- 1 .

[0348] In an example, in step 1112, the ARRSC may forward the PO pre-notification to ARA-1.

[0349] In an example, in step 1113, ARA-1 may evaluate the mobile PO and, based on application logic, may determine whether the PO is a PTO (e.g., one that can be enhanced with AR effects). If so, ARA-1 may further determine what type of VO may be used to annotate the mobile PTO. In the event that the VO is not locally available at ARD-1, ARA-1 may download the VO. For example, AR design and preparation may be completed before the mobile PO may appear in ARD-1's viewport.

[0350] In an example, in step 1114 , ARA-1 may deploy the designed AR scene to ARP-1 and may instruct ARP-1 to detect the moving PTO (captured by any one of ARD-1's built-in cameras) from its own viewport.

[0351] In an example, in step 1116, ARP-1 may send an acknowledgment to ARA-1.

[0352] In an example, in step 1117 , ARP- 1 may capture real-time video frames from any of its own cameras and may (eg, closely) monitor and detect the moving PTO.

[0353] In an example, a PTO may be present and may be detected (based on the detailed characteristic information as indicated by ARA- 1 ) in step 1118. ARP- 1 may apply AR effects to the mobile PTO (eg, immediately) with reduced AR processing latency.

[0354] In the absence of an RSC, the process described herein may proceed in an ad hoc manner. For example, ARD-2 (as an ARRP) may use local broadcast or any other communication technique to send a pre-notification indicating a mobile PO to other nearby ARDs (as ARRCs), which may be on adjacent streets / roads, for example. For example, those nearby ARDs (as ARRCs) may subscribe to ARD-2 (as an ARRP), and (e.g., only) those ARDs that have subscribed to receive pre-notifications may receive a pre-notification indicating a mobile PO.

[0355] Example of 3GPP SA4 implementation

[0356] Figure 12

[0014] Figure 1 illustrates an example 3GPP SA4 implementation of an ARD architecture for supporting cross-user AR processing and resource sharing. 3GPP describes the 5G ARD as a WTRU. For example, a 5G AR WTRU in 3GPP may act as (e.g., operate as) either an ARRP or an ARRC as described herein.

[0357] The 5G AR WTRU 1201 operating as an ARRC may include an AR application 1202, which may include an AR controller for delivering AR effects to a user based on application logic. The AR application 1202 described in 3GPP may be considered as the ARA described herein. The 5G AR WTRU 1201 (as an ARRC) may include an AR runtime 1203, which may interact with various hardware (such as, for example, any of a display, a sensor, and a camera). The AR runtime 1203 may work with the AR scene manager 1204 of the 5G AR WTRU, which may retrieve a scene description from an AR scene provider (e.g., a server) to obtain scene content (e.g., data), may render it, and may send the rendered content to the AR runtime so that the rendered (one or more) virtual content may be sent to the display for presentation to the user. Based on the functionality of the AR runtime 1203 and the scene manager 1204 described in 3GPP SA4, the integration of those two components may be considered as an AR processor of the ARRC as described herein. The 5G AR WTRU 1201 may include a media access function (MAF) 1205 that may exchange AR / XR related content with other entities. The MAF 1205 in the 5G AR WTRU 1201 (operating as an ARRC) may be viewed as an implementation of the ARRSC of the ARRC described herein.

[0358] The 5G AR WTRU 1211 operating as an ARRP may include an AR runtime 1213 and a scene manager 1214. For example, the integration of those two components on the ARRP may be viewed as an AR processor for the ARRP as described herein. For example, the MAF 1215 in the 5G ARWTRU 1211 (in the role of the ARRP) may be viewed as an implementation of the ARRSS for the ARRP as described herein.

[0359] Example of 3GPP SA2 implementation

[0360] Figure 13 is a diagram illustrating an example 3GPP SA2 implementation for cross-user collaborative relationship establishment. For example, the ProSe function in 4G and / or the 5G Direct Discovery Name Management Function (DDNMF) can be used to enable a cross-user AR collaborative relationship between ARRP and ARRC. The RSC described in this document can be implemented as a Prose application server in 3GPP. Two models for D2D direct discovery are described in clause 6.3.1 "Proximitybased Services (ProSe) in the 5G System (5GS)" of 3GPP TS23.304V18.0.0(2022-12). Model B Direct Discovery is considered to be used to describe Figure 13 An example of establishing a cross-user collaborative relationship is shown in FIG. In 3GPP TS23.304 V18.0.0 (2022-12) Figure 6 3GPP Model B Direct Discovery is described in more detail in .3.1.3-1.

[0361] In an example, WTRU-1 and WTRU-2 may complete service authorization for using ProSe functionality in step 1300. For example, step 1300 may use the ProSe service authorization and provisioning procedures described in 3GPP TS 23.304 V18.0.0 (2022-12) “Proximity based Services (ProSe) in the 5G System (5GS)”.

[0362] In an example, in step 1301, WTRU-2 may send a (e.g., 3GPP D2D) discovery request, such as described in 3GPP TS 23.304 V18.0.0 (2022-12), to a first network element (e.g., operating a 5G DDNMF) for monitoring on PC5. For example, based on Model B direct discovery as described in 3GPP, WTRU-2 may operate as a discovered WTRU. For example, the (e.g., 3GPP D2D) discovery request may indicate the AR resource sharing capability of WTRU-2. For example, the (e.g., 3GPP D2D) discovery request may include, for example, Figure 7A Any one of the parameters indicated in the AR capability information described in step 701.

[0363] In the example, in step 1302, the first network element (e.g., running 5G DDNMF) may further forward the request to the second network element (e.g., running ProSe application server), and the second network element may include: Figure 7A Steps 1301-1302 may correspond to RSC-1 as shown in FIG. Figure 7A Step 701.

[0364] In an example, in step 1303, the second network element (e.g., running a ProSe application server) may check relevant policies (e.g., using a policy control function (PCF)) and may record the AR resource sharing capabilities of WTRU-2, e.g., indicating what types of AR resources can be shared by WTRU-2. Such information may be recorded as a registration record. For example, the second network element (e.g., running a ProSe application server) may authorize WTRU-2 to operate as an ARRP.

[0365] In an example, in step 1304, the second network element (e.g., running a ProSe application server) may send a confirmation of the AR resource sharing capability registration of WTRU-2 to the first network element (e.g., running a 5G DDNMF).

[0366] In an example, in step 1305, a first network element (e.g., running a 5G DDNMF) may assign (1) an AR resource solicitation filter and (2) an AR resource offer response code to WTRU-2. Their use is described herein with steps 1319-1323.

[0367] In an example, in step 1306, the first network element (e.g., running a 5G DDNMF) may send the assigned AR resource solicitation filter and the assigned AR resource offer response code back to WTRU-2. For example, WTRU-2 may receive first information from the first network element for communicating with another WTRU (such as, for example, WTRU-1). The first information may indicate the AR resource solicitation filter and the AR resource offer response code assigned to WTRU-2.

[0368] In an example, in step 1307, WTRU-2 may (eg, begin) monitoring a channel (eg, PC5) for an AR resource request code advertised on the channel (eg, PC5).

[0369] In an example, in step 1308 , WTRU- 1 may move (eg, quickly) through a (eg, unknown) physical environment X. For example, WTRU- 1 may determine that a customized AR scene is designed for physical environment X.

[0370] In an example, in step 1309 WTRU- 1 may determine to obtain some latest video frames captured from physical environment X.

[0371] In an example, in step 1310, WTRU-1 (eg, ARA-1 on WTRU-1) may send an AR resource sharing request to an ARRSC (eg, on WTRU-1).

[0372] In an example, in step 1311, WTRU-1 (e.g., ARRSC on WTRU-1) may determine to contact (e.g., send information to) a first network element (e.g., running a 5G DDNMF service) to identify (e.g., a target) ARRP, for example, using a ProSe direct discovery message.

[0373] In an example, WTRU-1 (e.g., ARRSC on WTRU-1) may send a discovery request 1312 to a first network element (e.g., running 5GDD NMF) to advertise its AR resource solicitation request on PC5, as well as a request for AR resources, e.g., indicating what kind of AR resources may be requested. Based on the Model B Direct Discovery described in 3GPP, WTRU-1 may operate as a discoverer WTRU. For example, the discovery request 1312 may include: Figure 7A Any of the parameters included in (eg, indicated by) the AR resource request 707 described in .

[0374] In an example, in step 1313 , the first network element (e.g., running a 5G DDNMF) may forward the request to the second network element (e.g., running a ProSe application server).

[0375] In an example, in step 1314, the second network element (e.g., running a ProSe application server) may check the relevant policies (e.g., with the PCF) and may authorize WTRU-1 to operate as an ARRC. The ProSe application server may check the ARRP registration record (e.g., created in step 1303) and may determine that the request for AR resources from WTRU-1 may be serviced by WTRU-2, which may be a registered ARRP (based on step 1303). Step 1313 may correspond to Figure 7A Step 708.

[0376] In an example, in step 1315, the second network element (e.g., running a ProSe application server) may send an acknowledgment to the first network element (e.g., running a 5G DDNMF) indicating that the AR resource sharing request from WTRU-1 may be served by WTRU-2.

[0377] In an example, in step 1316, the first network element (e.g., running a 5G DDNMF) may assign (1) an AR resource solicitation response filter and (2) an AR resource solicitation code to WTRU-1 based on the AR resource solicitation filter and AR resource solicitation response code assigned to WTRU-2 in step 1305 so that they may be matched together.

[0378] In an example, in step 1317, the first network element (e.g., running a 5G DDNMF) may send second information to WTRU-1 for communicating with WTRU-2. For example, the second information may indicate to WTRU-1 the allocated AR resource offer response filter and the allocated AR resource solicitation code.

[0379] In an example, in step 1318, WTRU-1 may (e.g., begin) announcing the received AR resource solicitation code (received in step 1317) on a channel (e.g., PC5) (e.g., transmit information indicating the received AR resource solicitation code). For example, the information indicating the received AR resource solicitation code may further include (e.g., indicate) the AR resource solicitation code received by Figure 7A Any parameters indicated by (eg, included in) the AR resource sharing request 711 described in .

[0380] In an example, in step 1319, WTRU-2 may determine that the AR resource solicitation code received from WTRU-1 may match its AR resource solicitation filter, which may indicate to WTRU-2 that WTRU-1 may operate as an ARRC to be served. WTRU-2 may react to WTRU-1's request. In the event that the notification transmitted by WTRU-1 in step 1318 includes information describing the requested AR resources, WTRU-1 may check again to determine whether the AR resource(s) requested by WTRU-1 can be provided based on either of its current workload and capabilities.

[0381] In the example, in step 1320, when WTRU-2 determines to serve WTRU-1 (e.g., provide the requested AR resources to WTRU-1), WTRU-2 may, for example, announce its AR resource provisioning code on PC5 (e.g., transmit information indicating its AR resource provisioning code).

[0382] In the example, in step 1321, WTRU-1 may determine that the AR resource provisioning code received from WTRU-2 may match its AR resource provisioning response filter (assigned in step 1316), so that WTRU-1 may understand that its AR resource sharing request may be served by WTRU-2 as an ARRP.

[0383] In this article, we use Figure 13 The examples described are based on monitoring / advertising techniques that enable ARD-1 and ARD-2 to find each other. For example, in the 3GPP SA2 example above, the D2D direct discovery mechanism may be used to enable WTRU-1 (e.g., ARD-1) to discover WTRU-2 (e.g., ARD-2). For example, monitoring / advertising techniques may be used to implement the process of how ARD-1 can discover ARD-2. Figure 13 The depicted example differs from the depicted example of FIG. 7 (eg, in step 709 ) in that RSC-1 may transmit information indicating the contact address of ARD-2 to ARD-1 so that ARD-1 may discover ARD-2 and establish a connection therewith. Figure 7AThe purpose of step 709 can be achieved by monitoring / notification technology (such as by Figure 13 1305, 1307, 1316, 1318, 1319, 1320 and 1321).

[0384] In an example, in step 1322, WTRU-1 (e.g., ARRSC on WTRU-1) may send a match report to the first network element (e.g., running 5G DDNMF). The match report may include details indicating how WTRU-1 and WTRU-2 may collaborate on AR processing, for example, by including Figure 7A Any of the parameters included in (eg, indicated by) the AR resource request 707 described.

[0385] In an example, in step 1323, the first network element (e.g., running the 5G DDNMF) may confirm the matching report (e.g., by sending a message (e.g., ack) indicating confirmation of the matching report).

[0386] In an example, in step 1324 , WTRU- 1 and WTRU- 2 may (eg, start) establishing a D2D link using ProSe direct communication.

[0387] Example of 3GPP SA6 implementation

[0388] Figure 14is a diagram illustrating an example 3GPP SA6 implementation aligned with the 3GPP SA6 architecture based on a client-server model. For example, an AR device may act as an ARRC and / or an ARRP (e.g., operate as an ARRC and / or an ARRP). An AR device acting as an ARRC 1401 (e.g., operating as an ARRC 1401) may include an ARA 1404, which may be a focused AR application, e.g., a consumer of AR resources shared by an ARRP 1411 (in this SA6 implementation, the ARA 1404 may be hosted on the ARRC 1401). The ARRC 1401 may include an ARP 1402. The ARP 1402 on the ARRC 1401 may operate two functions: 1) process the raw AR resources shared by the ARRP 1411 to extract physical world knowledge, and 2) perform intelligent AR processing (e.g., rendering) to display AR effects to the user. For example, the ARRC 1401 may include an ARRSC 1413, which may be considered a client interacting with the ARRP 1411. The ARD operating as an ARRP 1411 may include two modules. The ARRP 1411 may include an ARRSS 1413, which may be exposed as a service portal to receive one or more requests from the ARRSC(s) 1413 of the ARRC(s) 1401. The ARRP 1411 may include an ARP 1412, which may assist the ARRC 1401 in extracting physical world knowledge from the original or raw AR resources (such as, for example, (one or more) video frames) captured by the ARRP 1411. For example, the RSC 1430 may be deployed at the edge of the network, such as, for example, in any of the gNBs and network functions (NFs) in the core network (CN), for matching the ARRC 1401 with the ARRP 1411.

[0389] Example of ETSIARF implementation

[0390] Figure 15is a diagram illustrating an example ETSIARF implementation. The functional architecture of ETSIARF describes the following activities of AR applications, including: 1) physical world knowledge management, including physical world capture, physical world analysis, semantic understanding, and physical world knowledge storage; 2) AR scene management (such as scene update), including AR scene design (for example, using an AR authoring tool for designing AR scenes; a scene graph or scene description may include a tree-based data structure, which can be used to describe a scene, such as based on the glTF 2.0 format as described by MPEG) and AR asset preparation; 3) The designed scene can be rendered by a 3D rendering engine and presented to the user through one or more display devices (such as, for example, an optical see-through display, etc.). For example, the transmission part can allow AR-related data or content to be exchanged between different entities. For example, the ARA hosted on the ARRC described in this document can operate activities such as, for example, any of AR asset preparation, AR scene design, and scene management and physical world knowledge storage as described in the ETSI ARF (so that the ARA can use this knowledge to perform dynamic and customized scene design). The ARP hosted by the ARRC can operate activities such as any of world capture and world analysis as described in the ETSI ARF. For example, intelligent world analysis (such as fast PTO recognition) can be performed based on the physical world knowledge shared by the ARRP. For example, on the ARRP side, the ARP can operate activities such as any of world capture and world analysis to generate physical world knowledge about the physical environment X and share the knowledge with the ARRC. For example, the RSC can store world knowledge. For example, the ARRSC on the ARRC, the ARRSS on the ARRP, and the RSC can operate AR content sharing and exchange activities as described in the ETSI ARF.

[0391] Figure 16is a diagram illustrating an example method 1600 for enabling establishment of a cross-user collaborative relationship. The method 1600 may be implemented in a first WTRU. As shown at 1610, the first WTRU may send an AR resource request to a network element, the AR resource request indicating (1) an AR resource of the requested type and (2) location information associated with the first WTRU (e.g., and a physical environment). As shown at 1620, the first WTRU may receive an AR resource response from the network element, the AR resource response including first information for communicating with a second WTRU that is capable of providing one or more AR resources (e.g., and location information) based on the requested type. As shown at 1630, the first WTRU may send an AR resource sharing request to the second WTRU, the AR resource sharing request indicating, for example, at least one of an AR resource of the requested type and location information. As shown at 1640, the first WTRU may receive an AR resource sharing response from the second WTRU indicating agreement to provide AR resources of the requested type.

[0392] In an example, a first WTRU may receive one or more AR resources of a requested type, e.g., associated with a physical environment, from a second WTRU. In an example, in response to an AR resource sharing request, the first WTRU may not receive an AR resource sharing response indicating agreement to provide AR resources of the requested type. The first WTRU may receive one or more resources of the requested type in response to the AR resource sharing request, implicitly indicating agreement to provide AR resources of the requested type.

[0393] In an example, the first WTRU may determine an AR effect to apply to an AR scene (eg, of a physical environment) based on AR resources.

[0394] In an example, the positioning information associated with the first WTRU may indicate any one of a current location of the first WTRU, a travel path of the first WTRU, and a moving speed of the first WTRU.

[0395] In an example, the AR resource sharing request may be transmitted to the second WTRU via a device-to-device communication link established between the first WTRU and the second WTRU.

[0396] In an example, the one or more AR assets may include any of one or more video frames of the physical environment and physical world information indicating one or more physical objects of the physical environment.

[0397] In an example, the one or more AR resources may include physical world information indicating one or more physical objects of the physical environment. The first WTRU may be further configured to select a physical object from the one or more physical objects of the physical environment and apply the AR effect to the physical object.

[0398] In an example, the first WTRU may be further configured to download a virtual object associated with the selected physical object before the first WTRU may enter the physical environment.

[0399] In an example, the AR resource sharing request may indicate two different types of requested AR resources. For example, a first AR resource of a first type may include one or more video frames (e.g., of a physical environment), and a second AR resource of a second type may include physical world information indicating one or more physical objects (e.g., of a physical environment).

[0400] In an example, the first WTRU may be further configured to send a physical world knowledge verification request including physical object information to the second WTRU.

[0401] In an example, the physical object information may indicate one or more characteristics of one or more physical objects to be verified.

[0402] In an example, the one or more characteristics may include any of one or more positions, one or more shapes, one or more colors, and one or more orientations of the one or more physical objects.

[0403] In an example, the physical object information may indicate at least one version number associated with the at least one physical object.

[0404] In an example, the first WTRU may be further configured to receive a physical-world knowledge verification response including updated physical-world information from the second WTRU.

[0405] In an example, the first WTRU may update the AR effects to be applied to the AR scene based on the updated physical world information.

[0406] In an example, the first WTRU may be further configured to receive a notification from a network element indicating a moving physical object.

[0407] In an example, the notification may indicate that a mobile physical object is moving towards the first WTRU.

[0408] In an example, the first WTRU may be further configured to determine that the mobile physical object is to be augmented with a second AR effect.

[0409] In an example, the first WTRU may be further configured to determine a second AR effect.

[0410] In an example, the first WTRU may be further configured to capture one or more subsequent frames of the physical environment and detect the moving physical object in the captured one or more subsequent frames.

[0411] In an example, the first WTRU may be further configured to apply a second AR effect to the moving physical object, wherein the second AR effect may have been determined before the moving physical object is detected in one or more subsequent frames that are captured.

[0412] Figure 17 is a diagram illustrating another example method 1700 for enabling establishment of a cross-user collaborative relationship. The method 1700 may be implemented in a second WTRU. As shown at 1710, the second WTRU may send AR capability information to a network element, the AR capability information indicating: (1) at least one type of AR resource that the second WTRU may be able to provide, and (2) positioning information associated with the second WTRU (e.g., in a physical environment). As shown at step 1720, the second WTRU may receive an AR resource sharing request from the first WTRU, the AR resource sharing request indicating at least one type of AR resource that the second WTRU is able to provide. As shown at 1730, the second WTRU may determine to provide one or more AR resources of the at least one type of AR resource (e.g., associated with a physical environment) based on the availability of processing resources in the second WTRU. As shown at 1740, the second WTRU may send an AR resource sharing response to the first WTRU indicating agreement to provide the at least one type of AR resource.

[0413] In an example, a second WTRU may send one or more AR resources of at least one type of AR resources (e.g., associated with a physical environment) to a first WTRU. In an example, in response to an AR resource sharing request, the second WTRU may not send an AR resource sharing response indicating an agreement to provide at least one type of AR resources. The second WTRU may send one or more resources of at least one type in response to the AR resource sharing request, implicitly indicating an agreement to provide at least one type of AR resources.

[0414] In an example, the AR capability information may further indicate any one of a second WTRU identifier, AR hardware capabilities of the second WTRU, and AR software capabilities of the second WTRU.

[0415] In an example, the positioning information associated with the second WTRU may indicate any one of a current location of the second WTRU, a travel path of the second WTRU, and a moving speed of the second WTRU.

[0416] In an example, an AR resource sharing request may be received from a first WTRU via a device-to-device communication link established between the first WTRU and a second WTRU.

[0417] In an example, the second WTRU may capture one or more video frames from the physical environment.

[0418] In an example, the second WTRU may detect one or more physical objects in the physical environment based on one or more video frames captured from the physical environment.

[0419] In an example, the one or more AR assets may include physical world information indicative of one or more physical objects detected in the physical environment.

[0420] In an example, the physical world information may indicate any of a position, orientation, color, shape, and texture of at least one physical object among one or more physical objects detected in the physical environment.

[0421] In an example, the AR capability information may indicate two different types of AR resources. For example, a first AR resource of a first type may include one or more video frames (e.g., of a physical environment), and a second AR resource of a second type may include physical world information indicating one or more physical objects (e.g., of a physical environment).

[0422] In an example, the second WTRU may be further configured to receive a physical world knowledge verification request including physical object information from the second WTRU.

[0423] In an example, the physical object information may indicate one or more characteristics of one or more physical objects to be verified.

[0424] In an example, the one or more characteristics may include any of one or more positions, one or more shapes, one or more colors, and one or more orientations of the one or more physical objects.

[0425] In an example, the physical object information may indicate at least one version number associated with the at least one physical object.

[0426] In an example, the second WTRU may be further configured to determine updated physical world information.

[0427] In an example, the second WTRU may be further configured to send a physical-world knowledge verification response including updated physical-world information to the first WTRU.

[0428] In an example, the second WTRU may be further configured to capture one or more subsequent frames of the physical environment and detect the moving physical object in the captured one or more subsequent frames.

[0429] In an example, the second WTRU may be further configured to send a mobile physical object identification report indicating the mobile physical object to the network element.

[0430] In an example, the mobile physical object identification report may indicate a direction in which the mobile physical object may move.

[0431] In an example, the mobile physical object identification report may indicate a speed of the mobile physical object.

[0432] In an example, the second WTRU may be further configured to receive confirmation information from the network element confirming receipt of the mobile physical object identification report. Any features, variations, or embodiments described for the method are compatible with an apparatus including means for processing the disclosed method, with an apparatus including a processor and a transceiver operatively coupled to the processor configured to process the disclosed method, with a computer program product including program code instructions, and with a non-transitory computer-readable storage medium storing program instructions.

[0433] Although features and elements are provided above in specific combinations, it will be understood by those skilled in the art that each feature or element can be used alone or in any combination with other features and elements. The present disclosure is not limited to the aspects of the specific embodiments described in this application, which are intended to be illustrative of various aspects. Many modifications and variations can be made without departing from its spirit and scope, as will be clear to those skilled in the art. Unless explicitly provided as such, none of the elements, actions or instructions used in the specification of this application should be interpreted as being critical or essential to the present invention. Based on the foregoing description, functionally equivalent methods and devices within the scope of this disclosure, in addition to those enumerated herein, will be clear to those skilled in the art. Such modifications and variations are intended to fall within the scope of the appended claims. The present disclosure is limited only by the terms of the appended claims and the full scope of equivalents to which such claims are entitled. It is to be understood that the present disclosure is not limited to a specific method or system.

[0434] For simplicity, the foregoing embodiments are discussed with respect to the terminology and structure of devices with infrared capabilities (i.e., infrared transmitters and receivers). However, the embodiments discussed are not limited to these systems, but may be applied to other systems that use other forms of electromagnetic waves or non-electromagnetic waves (such as sound waves).

[0435] It is also understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, the term "video" or the term "image" may mean any of a snapshot, a single image, and / or a plurality of images displayed on a temporal basis. As another example, when referred to herein, the term "user equipment" and its abbreviation "UE", the term "remote" and / or the term "head mounted display" or its abbreviation "HMD" may mean or include (i) a wireless transmitting and / or receiving unit (WTRU); (ii) any of a plurality of embodiments of a WTRU; (iii) in particular a wireless-capable and / or wired-capable (e.g., tetherable) device configured with some or all of the structure and functionality of a WTRU; (iii) a wireless-capable and / or wired-capable device configured with less than all of the structure and functionality of a WTRU; or (iv) the like. Figures 1A-1D Details of an example WTRU are provided that can represent any WTRU described herein. As another example, various disclosed embodiments herein, both above and below, are described as utilizing a head-mounted display. Those skilled in the art will recognize that devices other than head-mounted displays can be utilized and that some or all of the present disclosure and various disclosed embodiments can be modified accordingly without undue experimentation. Examples of such other devices may include drones or other devices configured to stream information for providing an adapted reality experience.

[0436] In addition, the methods provided herein can be implemented in a computer program, software, or firmware incorporated into a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted via a wired or wireless connection) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, read-only memory (ROM), random access memory (RAM), registers, cache memory, semiconductor memory devices, magnetic media (such as internal hard disks and removable disks), magneto-optical media, and optical media (such as CD-ROMs and digital versatile disks (DVDs)). A processor associated with the software can be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.

[0437] Variations of the methods, apparatus, and systems provided above are possible without departing from the scope of the present invention. In view of the wide variety of embodiments that may be applied, it should be understood that the illustrated embodiments are merely examples and should not be considered as limiting the scope of the following claims. For example, the embodiments provided herein include handheld devices that may include or be used with any suitable voltage source (such as a battery and the like) that provides any suitable voltage.

[0438] In addition, in the embodiments provided above, processing platforms, computing systems, controllers and other devices including processors are mentioned. These devices may include at least one central processing unit ("CPU") and memory. According to the practice of those skilled in the art of computer programming, reference to the actions and symbolic representations of operations or instructions may be performed by various CPUs and memories. Such actions and operations or instructions may be referred to as being "executed," "computer-executed," or "CPU-executed."

[0439] Those skilled in the art will understand that the actions and symbolically represented operations or instructions comprise manipulations of electrical signals by the CPU. The electrical system represents data bits that can cause a resulting transformation or reduction of the electrical signals and the maintenance of the data bits at memory locations in the memory system to thereby reconfigure or otherwise alter the operation of the CPU and other processing of signals. The memory locations where the data bits are maintained are physical locations having specific electrical, magnetic, optical, or organic properties that correspond to or represent the data bits. It should be understood that the embodiments are not limited to the platforms or CPUs mentioned above, and that other platforms and CPUs can support the provided methods.

[0440] The data bits may also be maintained on computer-readable media, including magnetic disks, optical disks, and any other volatile (e.g., random access memory (RAM)) or non-volatile (e.g., read-only memory (ROM)) mass storage systems readable by a CPU. The computer-readable media may include cooperating or interconnected computer-readable media that reside solely on a processing system or distributed across multiple interconnected processing systems that may be local or remote to the processing system. It should be understood that the embodiments are not limited to the memories mentioned above, and other platforms and memories may support the provided methods.

[0441] In an illustrative embodiment, any of the operations, processes, etc. described herein may be implemented as computer-readable instructions stored on a computer-readable medium. The computer-readable instructions may be executed by a processor of a mobile unit, a network element, and / or any other computing device.

[0442] There is little distinction between hardware and software implementations of various aspects of the system. The use of hardware or software is typically (but not always, as the choice between hardware and software may become important in certain contexts) a design choice that represents a cost-efficiency trade-off. There can be a variety of carriers (e.g., hardware, software, and / or firmware) by which the processes and / or systems and / or other techniques described herein can be implemented, and the preferred carrier can vary depending on the context in which the processes and / or systems and / or other techniques are deployed. For example, if the implementer determines that speed and accuracy are most important, the implementer may choose a primarily hardware and / or firmware carrier. If flexibility is most important, the implementer may choose a primarily software implementation. Alternatively, the implementer may choose some combination of hardware, software, and / or firmware.

[0443] The foregoing detailed description has been described using block diagrams, flow charts, and / or examples to illustrate various embodiments of the device and / or process. Where such block diagrams, flow charts, and / or examples include one or more functions and / or operations, it will be understood by those skilled in the art that each function and / or operation within such block diagrams, flow charts, or examples may be implemented individually and / or collectively by a variety of hardware, software, firmware, or any combination thereof. In an embodiment, several portions of the subject matter described herein may be implemented via application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), digital signal processors (DSPs), and / or other integrated forms. However, those skilled in the art will recognize that some aspects of the embodiments disclosed herein may be implemented in whole or in part in an integrated circuit as one or more computer programs running on one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more processors (e.g., as one or more programs running on one or more microprocessors), as firmware, or as any combination thereof, and that designing circuitry and / or writing code for software and / or firmware, in accordance with the present disclosure, will be entirely within the skill of those skilled in the art. Furthermore, those skilled in the art will appreciate that the mechanisms of the subject matter described herein may be distributed as a program product in a variety of forms, and that the illustrative embodiments of the subject matter described herein are applicable regardless of the particular type of signal-bearing medium used to actually perform the distribution. Examples of signal-bearing media include, but are not limited to, the following: recordable media such as floppy disks, hard drives, CDs, DVDs, digital tapes, computer memory, and the like; and transmission media such as digital and / or analog communication media (e.g., fiber optic cables, waveguides, wired communication links, wireless communication links, and the like).

[0444] Those skilled in the art will recognize that it is common in the art to describe equipment and / or process in the manner set forth herein, and thereafter use engineering practice to integrate the equipment and / or process described thus into a data processing system. That is, at least a portion of the equipment and / or process described herein can be integrated into a data processing system via a reasonable number of experiments. Those skilled in the art will recognize that a typical data processing system can generally include one or more of the following: a system unit housing, a video display device, a memory (such as volatile and non-volatile memory), a processor (such as a microprocessor and a digital signal processor), a computing entity (such as an operating system, a driver, a graphical user interface (GUI) and an application), one or more interactive devices (such as a touchpad or touch screen) and / or a control system including a feedback loop and a control motor (for example, for sensing position and / or speed feedback, for moving and / or adjusting a control motor of a component and / or amount). A typical data processing system can utilize any suitable commercially available component to realize, such as those components typically found in data computing / communication and / or network computing / communication systems.

[0445] The subject matter described herein sometimes illustrates different components that are included in or connected to different other components. It is to be understood that such depicted architectures are merely examples, and in fact many other architectures that achieve the same functionality can be implemented. In a conceptual sense, any arrangement of components that achieve the same functionality is effectively "associated" so that the desired functionality can be achieved. Therefore, any two components that are combined to achieve a specific functionality herein can be considered to be "associated" with each other so that the desired functionality is achieved, regardless of the architecture or intermediate components. Similarly, any two components that are so associated can also be considered to be "operably connected" or "operably coupled" to each other to achieve the desired functionality, and any two components that can be so associated can also be considered to be "operably coupled" to each other to achieve the desired functionality. Specific examples of operable coupling include, but are not limited to, components that can physically cooperate and / or physically interact and / or components that can wirelessly interact and / or wirelessly interact and / or components that can logically interact and / or components that can logically interact.

[0446] With respect to the use of substantially any plural and / or singular terms herein, those skilled in the art can appropriately translate from the plural to the singular and / or from the singular to the plural as appropriate to the context and / or application. For clarity, various singular / plural arrangements may be expressly set forth herein.

[0447] Those skilled in the art will understand that, in general, the terms used herein and particularly in the appended claims (e.g., the bodies of the appended claims) are generally intended as "open-ended" terms (e.g., the term "including" should be interpreted as "including but not limited to," the term "having" should be interpreted as "having at least," the term "comprising" should be interpreted as "including but not limited to," etc.). Those skilled in the art will further understand that if a specific number of claim recitations is intended, such intent will be explicitly recited in the claim, and in the absence of such recitation, no such intent is present. For example, where only one item is intended, the term "single" or similar language may be used. To aid understanding, the following appended claims and / or the description herein may include the use of the introductory phrases "at least one" and "one or more" to introduce claim recitations. However, the use of such phrases should not be interpreted as implying that a claim recitation introduced by the indefinite article "a" or "an" will limit any particular claim including such introduced claim recitation to embodiments including only one such recitation, even when the same claim includes the introductory phrases "one or more" or "at least one" and an indefinite article, such as "a" or "an" (e.g., "a" and / or "an" should be interpreted as meaning "at least one" or "one or more"). The same applies to the use of definite articles to introduce claim recitations. Furthermore, even if a specific number of introduced claim recitations is explicitly recited, those skilled in the art will recognize that such recitation should be interpreted to mean at least the recited number (e.g., the bare recitation of "two recitations" without other modifiers means at least two recitations, or two or more recitations). Furthermore, in those instances where a convention similar to “at least one of A, B, and C, etc.” is used, generally speaking, a person skilled in the art will understand that such construction is intended in the sense of the convention (e.g., “a system having at least one of A, B, and C” will include but is not limited to systems having A alone, B alone, C alone, both A and B, both A and C, both B and C, and / or both A, B, and C, etc.). In those instances where a convention similar to “at least one of A, B, or C, etc.” is used, generally speaking, a person skilled in the art will understand that such construction is intended in the sense of the convention (e.g., “a system having at least one of A, B, or C” will include but is not limited to systems having A alone, B alone, C alone, both A and B, both A and C, both B and C, and / or both A, B, and C, etc.). Those skilled in the art will further understand that, in fact, any disjunctive words and / or phrases presenting two or more alternative terms, whether in the specification, claims, or drawings, should be understood to contemplate the possibility of including one, either, or both of the terms.For example, the phrase "A or B" will be understood to include the possibilities of "A" or "B" or "A and B." Furthermore, as used herein, the term "any of" followed by a listing of multiple items and / or categories of items is intended to include "any of," "any combination," "any multiples," and / or "any combination of multiples" of the items and / or categories of items, alone or in combination with other items and / or categories of items. Furthermore, as used herein, the term "set" is intended to include any number of items, including zero. Additionally, as used herein, the term "number" is intended to include any number, including zero. And the term "plurality," as used herein, is intended to be synonymous with "multiple."

[0448] In addition, where features or aspects of the disclosure are described in terms of Markush groups, those skilled in the art will recognize that the disclosure is also described in terms of any individual member or subgroup of members of the Markush group.

[0449] As will be understood by those skilled in the art, for any and all purposes (such as with respect to providing a written description), all ranges disclosed herein also encompass any and all possible subranges and combinations thereof. Any listed range can be easily identified as fully describing and enabling the same range to be divided into at least equal halves, one-third, one-quarter, one-fifth, one-tenth, etc. As non-limiting examples, each range discussed herein can be easily divided into lower one-third, middle one-third, and upper one-third, etc. As will be understood by those skilled in the art, all languages ​​such as "up to," "at least," "greater than," "less than," and the like include the numerals described and refer to the scopes that can subsequently be divided into subranges as discussed above. Finally, as will be understood by those skilled in the art, ranges include each individual numeral. Therefore, for example, a group having 1-3 units refers to a group having 1, 2, or 3 units. Similarly, a group having 1-5 units refers to a group having 1, 2, 3, 4, or 5 units, etc.

[0450] Furthermore, the claims should not be read as limited to the order or elements provided unless otherwise stated. Furthermore, the use of the term "means for..." in any claim is intended to invoke 35 U.S.C. § 112, or means-plus-function claim format, and any claim without the term "means for..." is not intended to be so.

Claims

1. A first wireless transmit / receive unit (WTRU) comprising a processor and a transceiver operatively coupled to the processor, the processor and the transceiver being configured to: sending an augmented reality (AR) resource request to a network element, the augmented reality (AR) resource request indicating: (1) a requested type of AR resource, and (2) positioning information associated with a first WTRU; receiving an AR resource response from a network element, the AR resource response including first information for communicating with a second WTRU capable of providing one or more AR resources based on the requested type and positioning information; sending an AR resource sharing request to a second WTRU, the AR resource sharing request indicating at least one of an AR resource of a requested type and positioning information; as well as An AR resource sharing response is received from the second WTRU indicating agreement to provide AR resources of the requested type.

2. The first WTRU of claim 1 , wherein the processor and transceiver are further configured to receive one or more AR resources of the requested type from a second WTRU.

3. The first WTRU of claim 2, wherein the processor and the transceiver are further configured to determine an AR effect to be applied to the AR scene based on one or more AR resources.

4. The first WTRU of any one of claims 1 to 3, wherein the positioning information associated with the first WTRU indicates any one of a current location of the first WTRU, a travel path of the first WTRU, and a moving speed of the first WTRU.

5. The first WTRU of any one of claims 1 to 4, wherein the AR resource sharing request is transmitted to the second WTRU via a device-to-device communication link established between the first WTRU and the second WTRU.

6. The first WTRU according to any one of claims 1 to 5, wherein the one or more AR resources include any one of one or more video frames of a physical environment and physical world information indicating one or more physical objects of the physical environment.

7. The first WTRU of claim 6, wherein the physical world information indicates any one of a position, an orientation, a color, a shape, and a texture of at least one physical object of the one or more physical objects of the physical environment.

8. The first WTRU according to any one of claims 1 to 5, wherein the one or more AR resources include one or more video frames of a physical environment, and wherein the processor and the transceiver are further configured to detect one or more physical objects in the physical environment based on the one or more video frames of the physical environment.

9. A first WTRU according to any one of claims 3 to 5, wherein the one or more AR resources include physical world information indicating one or more physical objects of a physical environment, and wherein the processor and the transceiver are further configured to select a physical object from the one or more physical objects of the physical environment and apply the AR effect to the physical object.

10. The first WTRU of claim 9, wherein the processor and transceiver are further configured to download the virtual object associated with the selected physical object before the first WTRU enters the physical environment.

11. A first WTRU according to any one of claims 1 to 5, wherein the AR resource sharing request indicates two different types of requested AR resources, wherein a first AR resource of a first type includes one or more video frames, and wherein a second AR resource of a second type includes physical world information indicating one or more physical objects.

12. A second WTRU comprising a processor and a transceiver operably coupled to the processor, the processor and the transceiver configured to: sending AR capability information to a network element, the AR capability information indicating: (1) at least one type of AR resource that the second WTRU is capable of providing, and (2) positioning information associated with the second WTRU; receiving, from a first WTRU, an AR resource sharing request indicating at least one type of AR resource that a second WTRU is capable of providing; Determining to provide one or more AR resources of at least one type of AR resources based on availability of processing resources in the second WTRU; as well as An AR resource sharing response is sent to the first WTRU indicating agreement to provide at least one type of AR resource.

13. The second WTRU of claim 12, wherein the processor and transceiver are further configured to send one or more AR resources of at least one type of AR resources to the first WTRU.

14. The second WTRU according to any one of claims 12 to 13, wherein the AR capability information further indicates any one of a second WTRU identifier, AR hardware capabilities of the second WTRU, and AR software capabilities of the second WTRU.

15. The second WTRU of any one of claims 12 to 14, wherein the positioning information associated with the second WTRU indicates any one of a current location of the second WTRU, a travel path of the second WTRU, and a moving speed of the second WTRU.

16. The second WTRU of any one of claims 12 to 15, wherein the processor and transceiver are configured to receive an AR resource sharing request from the first WTRU: The processor and transceiver are configured to receive an AR resource sharing request over a device-to-device communication link established between a first WTRU and a second WTRU.

17. The second WTRU of any one of claims 12 to 16, wherein the processor and transceiver are further configured to capture one or more video frames from the physical environment.

18. The second WTRU of claim 17, wherein the one or more AR resources include one or more video frames captured from a physical environment.

19. The second WTRU of any one of claims 17 to 18, wherein the processor and transceiver are further configured to detect one or more physical objects in the physical environment based on one or more video frames captured from the physical environment.

20. The second WTRU of claim 19, wherein the one or more AR resources include physical world information indicative of one or more physical objects detected in the physical environment.

21. The second WTRU of claim 20, wherein the physical world information indicates any one of a position, an orientation, a color, a shape, and a texture of at least one physical object among the one or more physical objects detected in the physical environment.

22. A second WTRU according to any one of claims 12 to 16, wherein the AR capability information indicates two different types of AR resources, wherein a first AR resource of a first type includes one or more video frames, and wherein a second AR resource of a second type includes physical world information indicating one or more physical objects.

23. A method implemented in a first wireless transmit / receive unit (WTRU), the method comprising: sending an augmented reality (AR) resource request to a network element, the augmented reality (AR) resource request indicating: (1) a requested type of AR resource, and (2) positioning information associated with a first WTRU; receiving an AR resource response from the network element, the AR resource response indicating that the second WTRU is capable of providing one or more AR resources based on the requested type and positioning information; sending an AR resource sharing request to a second WTRU, the AR resource sharing request indicating at least one of an AR resource of a requested type and positioning information; as well as An AR resource sharing response is received from the second WTRU indicating agreement to provide AR resources of the requested type.

24. The method of claim 23, further comprising receiving one or more AR resources of the requested type from the second WTRU. The method of claim 24 , further comprising determining an AR effect to be applied to the AR scene based on the one or more AR resources.

26. A method implemented in a second WTRU, the method comprising: sending AR capability information to a network element, the AR capability information indicating: (1) at least one type of AR resource that the second WTRU is capable of providing, and (2) positioning information associated with the second WTRU; receiving, from a first WTRU, an AR resource sharing request indicating at least one type of AR resource that a second WTRU is capable of providing; Determining to provide one or more AR resources of at least one type of AR resources based on availability of processing resources in the second WTRU; as well as An AR resource sharing response is sent to the first WTRU indicating agreement to provide at least one type of AR resource.

27. The method of claim 26, further comprising sending one or more AR resources of at least one type of AR resources to the first WTRU.

Images