Methods and apparatuses for blockchain-enabled quality management for resource sharing
Blockchain-enabled quality management addresses the lack of quality control in AR resource sharing by using smart contracts and reputation systems, ensuring reliable and high-quality AR resource sharing between AR devices.
Patent Information
- Application Number
- PCT/US2024/052545
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-10-23
- Filing Date
- 2024-10-23
- Publication Date
- 2025-05-01
AI Technical Summary
There is no existing system or method to manage the sharing of knowledge and data between augmented reality (AR) devices, nor is there a technique to check and manage the quality of such sharing.
The use of blockchain-enabled quality management for decentralized AR resource sharing between AR resource providers and consumers, which includes smart contract-based and endorsement-based quality management mechanisms.
Enables reliable and trustworthy AR resource sharing with desired quality, ensuring that AR resources meet the requirements of consumers and maintaining a high reputation within the AR sharing community.
Smart Images

Figure US2024052545_01052025_PF_FP_ABST
Abstract
Description
METHODS AND APPARATUSES FOR BLOCKCHAIN-ENABLED QUALITY MANAGEMENT FOR RESOURCE SHARINGCROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 545,299, filed on October 23, 2023, the contents of which are incorporated herein by reference.BACKGROUND
[0002] Augmented reality (AR) applications may render one or more AR scenes to a user. In the one or more AR scenes, one or more virtual objects (VOs) may be superimposed on one or more physical objects (POs). Hence, for generating the one or more AR scenes, an AR device needs to possess knowledge and / or data related to the one or more POs. However, this knowledge and / or data may not be readily available to the AR device. In some cases, this may be because the AR device may be unable to capture the data due to positional and / or technical limitations. The AR device may request another AR device for the knowledge and / or data necessary to generate the one or more AR scenes. However, there is no existing system or method that can manage such sharing of the knowledge and / or data. Moreover, there exists no technique to check and / or manage quality of such sharing. Therefore, there is a need for a technique to control and / or manage the quality of the knowledge and / or data shared between the AR devices.SUMMARY
[0003] This disclosure relates to communication networks, including wireless and / or wired communication networks For example, one or more embodiments disclosed herein are related to methods and apparatuses for blockchain-enabled quality management for resource sharing (e.g., augmented reality (AR) resource sharing) in the communication networks.
[0004] In various embodiments, one or more implementations of the blockchain-based quality management for cross-user AR resource sharing and / or any other resource sharing in 5G and / or 6G applications are disclosed.
[0005] In one example, one or more improved decentralized AR resource sharing procedures are provided. In an example, the resource sharing may be conducted in an ad-hoc mode manner. In an example, the resource sharing may be conducted in a direct manner between one or more AR resource consumers (ARRCs) and one or more AR resource providers (ARRPs) without a need for a centralized AR resource repository. When the AR resource sharing is conducted in a decentralized way, one or more corresponding AR resource quality management mechanisms may be used. The ARRPs and / or the ARRCs may be personal user devices and may not trust each other. As such, the AR resource sharing may be spontaneous, ad hoc, and at times random, for example. The ARRPs may not have obligations (e.g., service agreements) for serving the ARRCs. As a consequence, it may be possible that a quality of one or more AR resources shared by the ARRPs may notalways fulfill the requirements of the ARRCs. To address this need, the blockchain-based quality management for supporting the decentralized AR resource sharing between the ARRPs and the ARRCs is provided.
[0006] Various embodiments disclosed herein are to realize the AR resource sharing with a desired quality between an ARRP and an ARRC.
[0007] In one embodiment, a smart contract-based AR resource quality management is provided. In that, the quality management of the one or more AR resources is realized through establishing a collaboration relationship and / or a collaboration agreement (e.g. a smart contract) between the ARRP and the ARRC such that the ARRP may comply with the collaboration agreement for providing the one or more AR resources with a quality desired by the ARRC.
[0008] In one embodiment, an endorsement-based AR resource quality management is provided. In that, the quality management of the one or more AR resources is realized without establishing the collaboration agreement between the ARRP and the ARRC. Instead, a blockchain-based reputation system is leveraged such that each party participating in the AR resource sharing may perform well in order to maintain a high reputation in an AR sharing community. For example, higher reputation may increase a chance of an AR device (ARD) to be involved in future AR resource sharing events, to serve one or more ARDs for potential rewards and / or to obtain help from the one or more ARDs.
[0009] In one or more embodiments, a method performed by a wireless transmit / receive unit (WTRU) is provided. The method includes receiving, from a consumer device, an AR resource solicitation request comprising one or more smart contract terms and one or more quality requirements associated with one or more AR resources. The method further includes determining that one or more device capabilities of the WTRU meet the one or more quality requirements. The method further includes generating a smart contract based on the one or more smart contract terms. The method further includes generating the one or more AR resources based on the smart contract. The method further includes transmitting the one or more AR resources to the consumer device.
[0010] In an embodiment, the one or more quality requirements include a resource type indicative of at least one of: a first AR resource type indicative of one or more original AR resources, or a second AR resource type indicative of one or more processed AR resources.
[0011] In an embodiment, generating one or more AR resources includes generating sensory data using one or more sensors of the WTRU.
[0012] In an embodiment, the WTRU transmits the sensory data as the one or more AR resources when the resource type is indicative of the first AR resource type.
[0013] In an embodiment, the WTRU processes the sensory data when the resource type is indicative of the second AR resource type. The WTRU transmits processed sensory data as the one or more AR resources.
[0014] In an embodiment, the one or more quality requirements comprise a freshness requirement associated with a time of generation of the one or more AR resources.
[0015] In an embodiment, the WTRU transmits a timestamp indicative of the time of generation of the one or more AR resources.
[0016] In an embodiment, the one or more smart contract terms comprise at least one of: a service fee associated with the smart contract or a penalty fee associated with the smart contract.
[0017] In an embodiment, the AR resource solicitation request further comprises at least one of: one or more physical target objects to be captured, one or more geographical locations associated with generation of the one or more AR resources, or one or more requirements on technical specifications associated with the one or more AR resources to be generated.
[0018] In an embodiment, the one or more requirements on technical specifications are indicative of at least one of: a camera orientation, a camera pose, or one or more camera settings.
[0019] In an embodiment, the WTRU negotiates the smart contract with the consumer device. The WTRU generates a final smart contract based on the negotiation.
[0020] In an embodiment, the one or more AR resources are generated based on the final smart contract.
[0021] In an embodiment, the final smart contract is stored in a blockchain network.
[0022] In one or more embodiments, a method performed by a WTRU is provided. The method includes discovering a plurality of provider devices. The method further includes transmitting, to the plurality of provider devices, an AR resource solicitation request indicative of a plurality of quality requirements. The method further includes receiving a smart contract from at least one provider device of the plurality of provider devices. The method further includes transmitting the smart contract to a blockchain network. The method further includes receiving one or more AR resources based on the smart contract.
[0023] In an embodiment, the WTRU determines whether the smart contract is acceptable. The WTRU identifies one or more parts of the smart contract for negotiation if the smart contract is not acceptable. The WTRU transmits to the at least one provider device, a negotiation indication associated with the one or more parts of the smart contract.
[0024] In an embodiment, the WTRU generates an AR scene based on the one or more AR resources.
[0025] In an embodiment, the WTRU evaluates the one or more AR resources based on the plurality of quality requirements. The WTRU generates, based on the evaluation, a quality evaluation result indicative of at least one of: one or more quality requirements of the plurality of quality requirements that are met by the one or more AR resources, or one or more quality requirements of the plurality of quality requirements that are not met by the one or more AR resources. The WTRU transmits the quality evaluation result to the blockchain network.
[0026] In an embodiment, the smart contract is executed based on the quality evaluation result.
[0027] In an embodiment, evaluating the one or more AR resources comprises generating sensory data using one or more sensors of the WTRU, comparing the sensory data with the one or more AR resources received from the at least one provider device, and determining an accuracy of the one or more AR resourcesbased on the comparison, wherein the quality evaluation result is indicative of the accuracy of the one or more AR resources.
[0028] In an embodiment, the WTRU re-calibrates the AR scene based on the accuracy of the one or more AR resources.
[0029] In an embodiment, evaluating the one or more AR resources comprises determining whether the one or more AR resources are indicative of one or more physical target objects indicated by the AR resource solicitation request.BRIEF DESCRIPTION OF THE DRAWINGS
[0030] A more detailed understanding may be had from the detailed description below, given by way of example in conjunction with drawings appended hereto. Figures in such drawings, like the detailed description, are examples. As such, the Figures (FIGs ) and the detailed description are not to be considered limiting, and other equally effective examples are possible and likely. Furthermore, like reference numerals ("ref.") in the FIGs. indicate like elements, and wherein:
[0031] FIG. 1A is a system diagram illustrating an example communications system in which one or more disclosed embodiments may be implemented;
[0032] FIG. 1 B is a system diagram illustrating an example wireless transmit / receive unit (WTRU) that may be used within the communications system illustrated in FIG. 1A according to an embodiment;
[0033] FIG. 1C is a system diagram illustrating an example radio access network (RAN) and an example core network (CN) that may be used within the communications system illustrated in FIG. 1A according to an embodiment;
[0034] FIG. 1D is a system diagram illustrating a further example RAN and a further example CN that may be used within the communications system illustrated in FIG. 1A according to an embodiment;
[0035] FIG. 2 is a diagram illustrating an example workflow of a blockchain system, according to one or more embodiments;
[0036] FIG. 3A is a diagram illustrating an example first use case of an augmented reality (AR) experience in a physical environment, according to one or more embodiments;
[0037] FIG. 3B is a diagram illustrating an example second use case of an AR experience in a physical environment, according to one or more embodiments;
[0038] FIG. 4A is a signal flowchart illustrating an example procedure of an AR resource sharing collaboration establishment using a smart contract (SC) for quality management, according to one or more embodiments;
[0039] FIG. 4B is a signal flowchart illustrating an example procedure of an AR resource sharing collaboration establishment using a SC for quality management, according to one or more embodiments;
[0040] FIG. 5A is a signal flowchart illustrating an example procedure of an AR resource sharing execution with quality management enforcement, according to one or more embodiments;
[0041] FIG. 5B is a signal flowchart illustrating an example procedure of an AR resource sharing execution with quality management enforcement, according to one or more embodiments;
[0042] FIG. 6A is a signal flowchart illustrating an example procedure of a reputation-based collaborative AR resource sharing, according to one or more embodiments;
[0043] FIG. 6B is a signal flowchart illustrating an example procedure of a reputation-based collaborative AR resource sharing, according to one or more embodiments;
[0044] FIG. 7A is a signal flowchart illustrating an example procedure of a device to device (D2D) AR resource sharing collaboration establishment for quality management, according to one or more embodiments;
[0045] FIG. 7B is a signal flowchart illustrating an example procedure of a D2D AR resource sharing collaboration establishment for quality management, according to one or more embodiments;
[0046] FIG. 8A is a signal flowchart illustrating an example procedure of an AR resource sharing execution with quality management enforcement, according to one or more embodiments;
[0047] FIG. 8B is a signal flowchart illustrating an example procedure of an AR resource sharing execution with quality management enforcement, according to one or more embodiments;
[0048] FIG. 8C is a signal flowchart illustrating an example procedure of an AR resource sharing execution with quality management enforcement, according to one or more embodiments;
[0049] FIG. 8D is a signal flowchart illustrating an example procedure of an AR resource sharing execution with quality management enforcement, according to one or more embodiments;
[0050] FIG. 9A is a signal flowchart illustrating an example procedure of an AR resource sharing collaboration establishment using smart contract for quality management, according to one or more embodiments;
[0051] FIG. 9B is a signal flowchart illustrating an example procedure of an AR resource sharing collaboration establishment using smart contract for quality management, according to one or more embodiments;
[0052] FIG. 10A is a signal flowchart illustrating an example procedure of an AR resource sharing collaboration establishment using smart contract for quality management, according to one or more embodiments;
[0053] FIG. 10B is a signal flowchart illustrating an example procedure of an AR resource sharing collaboration establishment using smart contract for quality management, according to one or more embodiments;
[0054] FIG. 11 is an example system diagram of an AR resource sharing system, according to one or more embodiments;
[0055] Fig 12 is an example flowchart of a process performed by a provider device, according to one or more embodiments; and
[0056] FIG. 13 is an example flowchart of a process performed by a consumer device, according to one or more embodiments.DETAILED DESCRIPTION
[0057] As discussed herein, one or more abbreviations in the following (non-exhaustive) list, shown in Table 1, may be used herein.Table 1
[0058] In the following detailed description, numerous specific details are set forth to provide a thorough understanding of embodiments and / or examples disclosed herein. However, it will be understood that such embodiments and examples may be practiced without some or all of the specific details set forth herein. In other instances, well-known methods, procedures, components and circuits have not been described in detail, so as not to obscure the following description. Further, embodiments and examples not specifically described herein may be practiced in lieu of, or in combination with, the embodiments and other examples described, disclosed or otherwise provided explicitly, implicitly and / or inherently (collectively "provided") herein. Althoughvarious embodiments are described and / or claimed herein in which an apparatus, system, device, etc. and / or any element thereof carries out an operation, process, algorithm, function, etc. and / or any portion thereof, it is to be understood that any embodiments described and / or claimed herein assume that any apparatus, system, device, etc. and / or any element thereof is configured to carry out any operation, process, algorithm, function, etc. and / or any portion thereof
[0059] The methods, apparatuses and systems provided herein are well-suited for communications involving both wired and wireless networks. An overview of various types of wireless devices and infrastructure is provided with respect to FIGs. 1A-1D, where various elements of the network may utilize, perform, be arranged in accordance with and / or be adapted and / or configured for the methods, apparatuses and systems provided herein.
[0060] FIG. 1A is a diagram illustrating an example communications system 100 in which one or more disclosed embodiments may be implemented The communications system 100 may be a multiple access system that provides content, such as voice, data, video, messaging, broadcast, etc., to multiple wireless users. The communications system 100 may enable multiple wireless users to access such content through the sharing of system resources, including wireless bandwidth. For example, the communications systems 100 may employ one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), singlecarrier FDMA (SC-FDMA), zero-tail unique-word discrete Fourier transform Spread OFDM (ZT-UW-DFT-S- OFDM), unique word OFDM (UW-OFDM), resource block-filtered OFDM, filter bank multicarrier (FBMC), and the like.
[0061] As shown in FIG. 1A, the communications system 100 may include wireless transmit / receive units (WTRUs) 102a, 102b, 102c, 102d, a radio access network (RAN) 104, a core network (ON) 106, a public switched telephone network (PSTN) 108, the Internet 110, and other networks 112, though it will be appreciated that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements Each of the WTRUs 102a, 102b, 102c, 102d may be any type of device configured to operate and / or communicate in a wireless environment. By way of example, the WTRUs 102a, 102b, 102c, 102d, any of which may be referred to as a station (STA), may be configured to transmit and / or receive wireless signals and may include a user equipment (UE), a mobile station, a fixed or mobile subscriber unit, a subscription-based unit, a pager, a cellular telephone, a personal digital assistant (PDA), a smartphone, a laptop, a netbook, a personal computer, a wireless sensor, a hotspot or Mi-Fi device, an Internet of Things (loT) device, a watch or other wearable, a head-mounted display (HMD), a vehicle, a drone, a medical device and applications (e.g., remote surgery), an industrial device and applications (e.g., a robot and / or other wireless devices operating in an industrial and / or an automated processing chain contexts), a consumer electronics device, a device operating on commercial and / or industrial wireless networks, and the like. Any of the WTRUs 102a, 102b, 102c and 102d may be interchangeably referred to as a UE.
[0062] The communications systems 100 may also include a base station 114a and / or a base station 114b. Each of the base stations 114a, 114b may be any type of device configured to wirelessly interface with at least one of the WTRUs 102a, 102b, 102c, 102d to facilitate access to one or more communication networks, such as the CN 106, the Internet 110, and / or the other networks 112. By way of example, the base stations 114a, 114b may be a base transceiver station (BTS), a NodeB, an eNode B (eNB), a Home Node B, a Home eNode B, a next generation NodeB, such as a gNode B (gNB), a new radio (NR) NodeB, a site controller, an access point (AP), a wireless router, and the like. While the base stations 114a, 114b are each depicted as a single element, it will be appreciated that the base stations 114a, 114b may include any number of interconnected base stations and / or network elements.
[0063] The base station 114a may be part of the RAN 104, which may also include other base stations and / or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), relay nodes, and the like. The base station 114a and / or the base station 114b may be configured to transmit and / or receive wireless signals on one or more carrier frequencies, which may be referred to as a cell (not shown). These frequencies may be in licensed spectrum, unlicensed spectrum, or a combination of licensed and unlicensed spectrum A cell may provide coverage for a wireless service to a specific geographical area that may be relatively fixed or that may change over time. The cell may further be divided into cell sectors. For example, the cell associated with the base station 114a may be divided into three sectors. Thus, in one embodiment, the base station 114a may include three transceivers, i.e., one for each sector of the cell. In an embodiment, the base station 114a may employ multiple-input multiple output (MIMO) technology and may utilize multiple transceivers for each sector of the cell. For example, beamforming may be used to transmit and / or receive signals in desired spatial directions.
[0064] The base stations 114a, 114b may communicate with one or more of the WTRUs 102a, 102b, 102c, 102d over an air interface 116, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, centimeter wave, micrometer wave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 116 may be established using any suitable radio access technology (RAT).
[0065] More specifically, as noted above, the communications system 100 may be a multiple access system and may employ one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, and the like. For example, the base station 114a in the RAN 104 and the WTRUs 102a, 102b, 102c may implement a radio technology such as Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), which may establish the air interface 116 using wideband CDMA (WCDMA). WCDMA may include communication protocols such as High-Speed Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA may include High-Speed Downlink (DL) Packet Access (HSDPA) and / or High-Speed Uplink (UL) Packet Access (HSU PA).
[0066] In an embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), which may establish the air interface 116 using Long Term Evolution (LTE) and / or LTE-Advanced (LTE-A) and / or LTE-Advanced Pro (LTE-A Pro).
[0067] In an embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as NR Radio Access , which may establish the air interface 116 using NR.
[0068] In an embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement multiple radio access technologies. For example, the base station 114a and the WTRUs 102a, 102b, 102c may implement LTE radio access and NR radio access together, for instance using dual connectivity (DC) principles. Thus, the air interface utilized by WTRUs 102a, 102b, 102c may be characterized by multiple types of radio access technologies and / or transmissions sent to / from multiple types of base stations (e g., an eNB and a gNB).
[0069] In other embodiments, the base station 114a and the WTRUs 102a, 102b, 102c may implement radio technologies such as IEEE 802.11 (i.e., Wireless Fidelity (WiFi), IEEE 802.16 (i.e., Worldwide Interoperability for Microwave Access (WiMAX)), CDMA2000, CDMA2000 1X, CDMA2000 EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), GSM EDGE (GERAN), and the like.
[0070] The base station 114b in FIG. 1A may be a wireless router, Home Node B, Home eNode B, or access point, for example, and may utilize any suitable RAT for facilitating wireless connectivity in a localized area, such as a place of business, a home, a vehicle, a campus, an industrial facility, an air corridor (e.g., for use by drones), a roadway, and the like. In one embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.11 to establish a wireless local area network (WLAN). In an embodiment, the base station 114b and the WTRUs 102c, 102d may implement a radio technology such as IEEE 802.15 to establish a wireless personal area network (WPAN) In yet another embodiment, the base station 114b and the WTRUs 102c, 102d may utilize a cellular-based RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, LTE-A Pro, NR etc.) to establish a picocell or femtocell. As shown in FIG. 1A, the base station 114b may have a direct connection to the Internet 110. Thus, the base station 114b may not be required to access the Internet 110 via the CN 106.
[0071] The RAN 104 may be in communication with the CN 106, which may be any type of network configured to provide voice, data, applications, and / or voice over internet protocol (VoIP) services to one or more of the WTRUs 102a, 102b, 102c, 102d. The data may have varying quality of service (QoS) requirements, such as differing throughput requirements, latency requirements, error tolerance requirements, reliability requirements, data throughput requirements, mobility requirements, and the like. The CN 106 may provide call control, billing services, mobile location-based services, pre-paid calling, Internet connectivity, video distribution, etc., and / or perform high-level security functions, such as user authentication. Although not shown in FIG. 1A, it will be appreciated that the RAN 104 and / or the CN 106 may be in direct or indirect communication with other RANs that employ the same RAT as the RAN 104 or a different RAT. For example, in addition to being connected to the RAN 104, which may be utilizing a NR radio technology, the CN 106 may also be in communication with another RAN (not shown) employing a GSM, UMTS, CDMA 2000, WiMAX, E-UTRA, or WiFi radio technology.
[0072] The CN 106 may also serve as a gateway for the WTRUs 102a, 102b, 102c, 102d to access the PSTN 108, the Internet 110, and / or the other networks 112. The PSTN 108 may include circuit-switched telephone networks that provide plain old telephone service (POTS). The Internet 110 may include a global system of interconnected computer networks and devices that use common communication protocols, such as the transmission control protocol (TCP), user datagram protocol (UDP) and / or the internet protocol (IP) in the TCP / IP internet protocol suite. The networks 112 may include wired and / or wireless communications networks owned and / or operated by other service providers. For example, the networks 112 may include another CN connected to one or more RANs, which may employ the same RAT as the RAN 104 or a different RAT.
[0073] Some or all of the WTRUs 102a, 102b, 102c, 102d in the communications system 100 may include multi-mode capabilities (e.g., the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers for communicating with different wireless networks over different wireless links). For example, the WTRU 102c shown in FIG. 1A may be configured to communicate with the base station 114a, which may employ a cellularbased radio technology, and with the base station 114b, which may employ an IEEE 802 radio technology.
[0074] FIG. 1 B is a system diagram illustrating an example WTRU 102. As shown in FIG. 1 B, the WTRU 102 may include a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a keypad 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power source 134, a global positioning system (GPS) chipset 136, and / or other peripherals 138, among others. It will be appreciated that the WTRU 102 may include any sub-combination of the foregoing elements while remaining consistent with an embodiment.
[0075] The processor 118 may be a general purpose processor, a special purpose processor, a conventional processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), any other type of integrated circuit (IC), a state machine, and the like The processor 118 may perform signal coding, data processing, power control, input / output processing, and / or any other functionality that enables the WTRU 102 to operate in a wireless environment. The processor 118 may be coupled to the transceiver 120, which may be coupled to the transmit / receive element 122. While FIG. 1 B depicts the processor 118 and the transceiver 120 as separate components, it will be appreciated that the processor 118 and the transceiver 120 may be integrated together in an electronic package or chip.
[0076] The transmit / receive element 122 may be configured to transmit signals to, or receive signals from, a base station (e.g., the base station 114a) over the air interface 116. For example, in one embodiment, the transmit / receive element 122 may be an antenna configured to transmit and / or receive RF signals In an embodiment, the transmit / receive element 122 may be an emitter / detector configured to transmit and / or receive IR, UV, or visible light signals, for example. In yet another embodiment, the transmit / receive element 122 may be configured to transmit and / or receive both RF and light signals. It will be appreciated that the transmit / receive element 122 may be configured to transmit and / or receive any combination of wireless signals.
[0077] Although the transmit / receive element 122 is depicted in FIG. 1 B as a single element, the WTRU 102 may include any number of transmit / receive elements 122. More specifically, the WTRU 102 may employ MIMO technology. Thus, in one embodiment, the WTRU 102 may include two or more transmit / receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 116.
[0078] The transceiver 120 may be configured to modulate the signals that are to be transmitted by the transmit / receive element 122 and to demodulate the signals that are received by the transmit / receive element 122. As noted above, the WTRU 102 may have multi-mode capabilities. Thus, the transceiver 120 may include multiple transceivers for enabling the WTRU 102 to communicate via multiple RATs, such as NR and IEEE 802.11 , for example.
[0079] The processor 118 of the WTRU 102 may be coupled to, and may receive user input data from, the speaker / microphone 124, the keypad 126, and / or the display / touchpad 128 (e.g , a liquid crystal display (LCD) display unit or organic light-emitting diode (OLED) display unit). The processor 118 may also output user data to the speaker / microphone 124, the keypad 126, and / or the display / touchpad 128. In addition, the processor 118 may access information from, and store data in, any type of suitable memory, such as the non-removable memory 130 and / or the removable memory 132. The non-removable memory 130 may include random-access memory (RAM), read-only memory (ROM), a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, and the like. In other embodiments, the processor 118 may access information from, and store data in, memory that is not physically located on the WTRU 102, such as on a server or a home computer (not shown).
[0080] The processor 118 may receive power from the power source 134, and may be configured to distribute and / or control the power to the other components in the WTRU 102. The power source 134 may be any suitable device for powering the WTRU 102. For example, the power source 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li- ion), etc.), solar cells, fuel cells, and the like.
[0081] The processor 118 may also be coupled to the GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 102 In addition to, or in lieu of, the information from the GPS chipset 136, the WTRU 102 may receive location information over the air interface 116 from a base station (e.g., base stations 114a, 114b) and / or determine its location based on the timing of the signals being received from two or more nearby base stations. It will be appreciated that the WTRU 102 may acquire location information by way of any suitable location-determination method while remaining consistent with an embodiment.
[0082] The processor 118 may further be coupled to other peripherals 138, which may include one or more software and / or hardware modules that provide additional features, functionality and / or wired or wireless connectivity. For example, the peripherals 138 may include an accelerometer, an e-compass, a satellite transceiver, a digital camera (for photographs and / or video), a universal serial bus (USB) port, a vibrationdevice, a television transceiver, a hands free headset, a Bluetooth® module, a frequency modulated (FM) radio unit, a digital music player, a media player, a video game player module, an Internet browser, a Virtual Reality and / or Augmented Reality (VR / AR) device, an activity tracker, and the like. The peripherals 138 may include one or more sensors. The sensors may be one or more of a gyroscope, an accelerometer, a hall effect sensor, a magnetometer, an orientation sensor, a proximity sensor, a temperature sensor, a time sensor; a geolocation sensor, an altimeter, a light sensor, a touch sensor, a magnetometer, a barometer, a gesture sensor, a biometric sensor, a humidity sensor and the like.
[0083] The WTRU 102 may include a full duplex radio for which transmission and reception of some or all of the signals (e g, associated with particular subframes for both the UL (e.g, for transmission) and DL (e.g, for reception) may be concurrent and / or simultaneous. The full duplex radio may include an interference management unit to reduce and or substantially eliminate self-interference via either hardware (e.g., a choke) or signal processing via a processor (e.g., a separate processor (not shown) or via processor 118). In an embodiment, the WTRU 102 may include a half-duplex radio for which transmission and reception of some or all of the signals (e.g , associated with particular subframes for either the UL (e.g., for transmission) or the DL (e.g., for reception)).
[0084] FIG. 1C is a system diagram illustrating the RAN 104 and the ON 106 according to an embodiment. As noted above, the RAN 104 may employ an E-UTRA radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 116. The RAN 104 may also be in communication with the ON 106.
[0085] The RAN 104 may include eNode-Bs 160a, 160b, 160c, though it will be appreciated that the RAN 104 may include any number of eNode-Bs while remaining consistent with an embodiment. The eNode-Bs 160a, 160b, 160c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116. In one embodiment, the eNode-Bs 160a, 160b, 160c may implement MIMO technology. Thus, the eNode-B 160a, for example, may use multiple antennas to transmit wireless signals to, and / or receive wireless signals from, the WTRU 102a
[0086] Each of the eNode-Bs 160a, 160b, 160c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the UL and / or DL, and the like. As shown in FIG. 1C, the eNode-Bs 160a, 160b, 160c may communicate with one another over an X2 interface.
[0087] The CN 106 shown in FIG. 1C may include a mobility management entity (MME) 162, a serving gateway (SGW) 164, and a packet data network (PDN) gateway (PGW) 166. While the foregoing elements are depicted as part of the CN 106, it will be appreciated that any of these elements may be owned and / or operated by an entity other than the CN operator.
[0088] The MME 162 may be connected to each of the eNode-Bs 162a, 162b, 162c in the RAN 104 via an S1 interface and may serve as a control node. For example, the MME 162 may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, bearer activation / deactivation, selecting a particular serving gateway during an initial attach of the WTRUs 102a, 102b, 102c, and the like. The MME 162 may provide a control planefunction for switching between the RAN 104 and other RANs (not shown) that employ other radio technologies, such as GSM and / or WCDMA
[0089] The SGW 164 may be connected to each of the eNode Bs 160a, 160b, 160c in the RAN 104 via the S1 interface. The SGW 164 may generally route and forward user data packets to / from the WTRUs 102a, 102b, 102c. The SGW 164 may perform other functions, such as anchoring user planes during inter-eNode B handovers, triggering paging when DL data is available for the WTRUs 102a, 102b, 102c, managing and storing contexts of the WTRUs 102a, 102b, 102c, and the like.
[0090] The SGW 164 may be connected to the PGW 166, which may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices.
[0091] The CN 106 may facilitate communications with other networks. For example, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional land-line communications devices. For example, the CN 106 may include, or may communicate with, an IP gateway (e.g., an IP multimedia subsystem (IMS) server) that serves as an interface between the CN 106 and the PSTN 108. In addition, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to the other networks 112, which may include other wired and / or wireless networks that are owned and / or operated by other service providers.
[0092] Although the WTRU is described in FIGS. 1A-1D as a wireless terminal, it is contemplated that in certain representative embodiments that such a terminal may use (e.g., temporarily or permanently) wired communication interfaces with the communication network.
[0093] In representative embodiments, the other network 112 may be a WLAN.
[0094] A WLAN in Infrastructure Basic Service Set (BSS) mode may have an Access Point (AP) for the BSS and one or more stations (STAs) associated with the AP. The AP may have access or an interface to a Distribution System (DS) or another type of wired / wireless network that carries traffic in to and / or out of the BSS Traffic to STAs that originates from outside the BSS may arrive through the AP and may be delivered to the STAs. T raffic originating from STAs to destinations outside the BSS may be sent to the AP to be delivered to respective destinations. Traffic between STAs within the BSS may be sent through the AP, for example, where the source STA may send traffic to the AP and the AP may deliver the traffic to the destination STA. The traffic between STAs within a BSS may be considered and / or referred to as peer-to-peer traffic. The peer-to- peer traffic may be sent between (e.g., directly between) the source and destination STAs with a direct link setup (DLS). In certain representative embodiments, the DLS may use an 802 11e DLS or an 802.11 z tunneled DLS (TDLS). A WLAN using an Independent BSS (IBSS) mode may not have an AP, and the STAs (e.g., all of the STAs) within or using the IBSS may communicate directly with each other. The IBSS mode of communication may sometimes be referred to herein as an “ad-hoc” mode of communication.
[0095] When using the 802.11ac infrastructure mode of operation or a similar mode of operations, the AP may transmit a beacon on a fixed channel, such as a primary channel. The primary channel may be a fixedwidth (e.g., 20 MHz wide bandwidth) or a dynamically set width. The primary channel may be the operating channel of the BSS and may be used by the STAs to establish a connection with the AP In certain representative embodiments, Carrier Sense Multiple Access with Collision Avoidance (CSMA / CA) may be implemented, for example in 802.11 systems. For CSMA / CA, the STAs (e.g., every STA), including the AP, may sense the primary channel. If the primary channel is sensed / detected and / or determined to be busy by a particular STA, the particular STA may back off. One STA (e.g., only one station) may transmit at any given time in a given BSS.
[0096] High Throughput (HT) STAs may use a 40 MHz wide channel for communication, for example, via a combination of the primary 20 MHz channel with an adjacent or nonadjacent 20 MHz channel to form a 40 MHz wide channel
[0097] Very High Throughput (VHT) STAs may support 20MHz, 40 MHz, 80 MHz, and / or 160 MHz wide channels The 40 MHz, and / or 80 MHz, channels may be formed by combining contiguous 20 MHz channels. A 160 MHz channel may be formed by combining 8 contiguous 20 MHz channels, or by combining two noncontiguous 80 MHz channels, which may be referred to as an 80+80 configuration. For the 80+80 configuration, the data, after channel encoding, may be passed through a segment parser that may divide the data into two streams. Inverse Fast Fourier Transform (IFFT) processing, and time domain processing, may be done on each stream separately. The streams may be mapped on to the two 80 MHz channels, and the data may be transmitted by a transmitting STA. At the receiver of the receiving STA, the above described operation for the 80+80 configuration may be reversed, and the combined data may be sent to the Medium Access Control (MAC).
[0098] Sub 1 GHz modes of operation are supported by 802.11 af and 802.11 ah. The channel operating bandwidths, and carriers, are reduced in 802.11 af and 802.11 ah relative to those used in 802.11n, and 802.11 ac. 802.11 af supports 5 MHz, 10 MHz, and 20 MHz bandwidths in the TV White Space (TVWS) spectrum, and 802.11ah supports 1 MHz, 2 MHz, 4 MHz, 8 MHz, and 16 MHz bandwidths using non-TVWS spectrum. According to a representative embodiment, 802.11ah may support Meter Type Control / Machine- Type Communications (MTC), such as MTC devices in a macro coverage area. MTC devices may have certain capabilities, for example, limited capabilities including support for (e.g., only support for) certain and / or limited bandwidths The MTC devices may include a battery with a battery life above a threshold (e.g., to maintain a very long battery life).
[0099] WLAN systems, which may support multiple channels, and channel bandwidths, such as 802.11n, 802.11 ac, 802.11af, and 802.11 ah, include a channel which may be designated as the primary channel. The primary channel may have a bandwidth equal to the largest common operating bandwidth supported by all STAs in the BSS. The bandwidth of the primary channel may be set and / or limited by a STA, from among all STAs in operating in a BSS, which supports the smallest bandwidth operating mode. In the example of 802.11 ah, the primary channel may be 1 MHz wide for STAs (e.g., MTC type devices) that support (e.g., only support) a 1 MHz mode, even if the AP, and other STAs in the BSS support 2 MHz, 4 MHz, 8 MHz, 16 MHz,and / or other channel bandwidth operating modes. Carrier sensing and / or Network Allocation Vector (NAV) settings may depend on the status of the primary channel. If the primary channel is busy, for example, due to a STA (which supports only a 1 MHz operating mode) transmitting to the AP, all available frequency bands may be considered busy even though a majority of the available frequency bands remains idle.
[0100] In the United States, the available frequency bands, which may be used by 802.11 ah, are from 902 MHz to 928 MHz. In Korea, the available frequency bands are from 917.5 MHz to 923.5 MHz. In Japan, the available frequency bands are from 916.5 MHz to 927.5 MHz. The total bandwidth available for 802.11 ah is 6 MHz to 26 MHz depending on the country code.
[0101] FIG. 1 D is a system diagram illustrating the RAN 104 and the CN 106 according to an embodiment. As noted above, the RAN 104 may employ an NR radio technology to communicate with the WTRUs 102a, 102b, 102c over the air interface 116. The RAN 104 may also be in communication with the CN 106.
[0102] The RAN 104 may include gNBs 180a, 180b, 180c, though it will be appreciated that the RAN 104 may include any number of gNBs while remaining consistent with an embodiment. The gNBs 180a, 180b, 180c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, 102c over the air interface 116. In one embodiment, the gNBs 180a, 180b, 180c may implement Ml MO technology. For example, gNBs 180a, 108b may utilize beamforming to transmit signals to and / or receive signals from the gNBs 180a, 180b, 180c. Thus, the gNB 180a, for example, may use multiple antennas to transmit wireless signals to, and / or receive wireless signals from, the WTRU 102a. In an embodiment, the gNBs 180a, 180b, 180c may implement carrier aggregation technology. For example, the gNB 180a may transmit multiple component carriers to the WTRU 102a (not shown). A subset of these component carriers may be on unlicensed spectrum while the remaining component carriers may be on licensed spectrum. In an embodiment, the gNBs 180a, 180b, 180c may implement Coordinated Multi-Point (CoMP) technology. For example, WTRU 102a may receive coordinated transmissions from gNB 180a and gNB 180b (and / or gNB 180c).
[0103] The WTRUs 102a, 102b, 102c may communicate with gNBs 180a, 180b, 180c using transmissions associated with a scalable numerology. For example, the OFDM symbol spacing and / or OFDM subcarrier spacing may vary for different transmissions, different cells, and / or different portions of the wireless transmission spectrum. The WTRUs 102a, 102b, 102c may communicate with gNBs 180a, 180b, 180c using subframe or transmission time intervals (TTIs) of various or scalable lengths (e.g., containing a varying number of OFDM symbols and / or lasting varying lengths of absolute time).
[0104] The gNBs 180a, 180b, 180c may be configured to communicate with the WTRUs 102a, 102b, 102c in a standalone configuration and / or a non-standalone configuration. In the standalone configuration, WTRUs 102a, 102b, 102c may communicate with gNBs 180a, 180b, 180c without also accessing other RANs (e.g., such as eNode-Bs 160a, 160b, 160c). In the standalone configuration, WTRUs 102a, 102b, 102c may utilize one or more of gNBs 180a, 180b, 180c as a mobility anchor point. In the standalone configuration, WTRUs 102a, 102b, 102c may communicate with gNBs 180a, 180b, 180c using signals in an unlicensed band. In a non-standalone configuration WTRUs 102a, 102b, 102c may communicate with / connect to gNBs 180a, 180b,180c while also communicating with / connecting to another RAN such as eNode-Bs 160a, 160b, 160c. For example, WTRUs 102a, 102b, 102c may implement DC principles to communicate with one or more gNBs 180a, 180b, 180c and one or more eNode-Bs 160a, 160b, 160c substantially simultaneously. In the non- standalone configuration, eNode-Bs 160a, 160b, 160c may serve as a mobility anchor for WTRUs 102a, 102b, 102c and gNBs 180a, 180b, 180c may provide additional coverage and / or throughput for servicing WTRUs 102a, 102b, 102c.
[0105] Each of the gNBs 180a, 180b, 180c may be associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, scheduling of users in the UL and / or DL, support of network slicing, DC, interworking between NR and E-UTRA, routing of user plane data towards User Plane Function (UPF) 184a, 184b, routing of control plane information towards Access and Mobility Management Function (AMF) 182a, 182b and the like. As shown in FIG 1D, the gNBs 180a, 180b, 180c may communicate with one another over an Xn interface.
[0106] The CN 106 shown in FIG. 1D may include at least one AMF 182a, 182b, at least one UPF 184a, 184b, at least one Session Management Function (SMF) 183a, 183b, and possibly a Data Network (DN) 185a, 185b. While the foregoing elements are depicted as part of the CN 106, it will be appreciated that any of these elements may be owned and / or operated by an entity other than the CN operator.
[0107] The AMF 182a, 182b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 104 via an N2 interface and may serve as a control node. For example, the AMF 182a, 182b may be responsible for authenticating users of the WTRUs 102a, 102b, 102c, support for network slicing (e g., handling of different protocol data unit (PDU) sessions with different requirements), selecting a particular SMF 183a, 183b, management of the registration area, termination of non-access stratum (NAS) signaling, mobility management, and the like. Network slicing may be used by the AMF 182a, 182b in order to customize CN support for WTRUs 102a, 102b, 102c based on the types of services being utilized WTRUs 102a, 102b, 102c. For example, different network slices may be established for different use cases such as services relying on ultra-reliable low latency (URLLC) access, services relying on enhanced massive mobile broadband (eMBB) access, services for MTC access, and the like. The AMF 182a, 182b may provide a control plane function for switching between the RAN 104 and other RANs (not shown) that employ other radio technologies, such as LTE, LTE-A, LTE-A Pro, and / or non-3GPP access technologies such as WiFi.
[0108] The SMF 183a, 183b may be connected to an AMF 182a, 182b in the CN 106 via an N11 interface. The SMF 183a, 183b may also be connected to a UPF 184a, 184b in the CN 106 via an N4 interface. The SMF 183a, 183b may select and control the UPF 184a, 184b and configure the routing of traffic through the UPF 184a, 184b. The SMF 183a, 183b may perform other functions, such as managing and allocating UE IP address, managing PDU sessions, controlling policy enforcement and QoS, providing DL data notifications, and the like A PDU session type may be IP-based, non-IP based, Ethernet-based, and the like.
[0109] The UPF 184a, 184b may be connected to one or more of the gNBs 180a, 180b, 180c in the RAN 104 via an N3 interface, which may provide the WTRUs 102a, 102b, 102c with access to packet-switchednetworks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices. The UPF 184, 184b may perform other functions, such as routing and forwarding packets, enforcing user plane policies, supporting multi-homed PDU sessions, handling user plane QoS, buffering DL packets, providing mobility anchoring, and the like.
[0110] The CN 106 may facilitate communications with other networks. For example, the CN 106 may include, or may communicate with, an IP gateway (e.g., an IP multimedia subsystem (IMS) server) that serves as an interface between the CN 106 and the PSTN 108. In addition, the CN 106 may provide the WTRUs 102a, 102b, 102c with access to the other networks 112, which may include other wired and / or wireless networks that are owned and / or operated by other service providers. In one embodiment, the WTRUs 102a, 102b, 102c may be connected to a local DN 185a, 185b through the UPF 184a, 184b via the N3 interface to the UPF 184a, 184b and an N6 interface between the UPF 184a, 184b and the DN 185a, 185b.
[0111] In view of FIGs. 1A-1 D, and the corresponding description of FIGs. 1A-1D, one or more, or all, of the functions described herein with regard to one or more of: WTRU 102a-d, Base Station 114a-b, eNode-B 160a-c, MME 162, SGW 164, PGW 166, gNB 180a-c, AMF 182a-b, UPF 184a-b, SMF 183a-b, DN 185a-b, and / or any other device(s) described herein, may be performed by one or more emulation devices (not shown). The emulation devices may be one or more devices configured to emulate one or more, or all, of the functions described herein. For example, the emulation devices may be used to test other devices and / or to simulate network and / or WTRU functions.
[0112] The emulation devices may be designed to implement one or more tests of other devices in a lab environment and / or in an operator network environment. For example, the one or more emulation devices may perform the one or more, or all, functions while being fully or partially implemented and / or deployed as part of a wired and / or wireless communication network in order to test other devices within the communication network. The one or more emulation devices may perform the one or more, or all, functions while being temporarily implemented / deployed as part of a wired and / or wireless communication network. The emulation device may be directly coupled to another device for purposes of testing and / or performing testing using over-the-air wireless communications.
[0113] The one or more emulation devices may perform the one or more, including all, functions while not being implemented / deployed as part of a wired and / or wireless communication network. For example, the emulation devices may be utilized in a testing scenario in a testing laboratory and / or a non-deployed (e.g., testing) wired and / or wireless communication network in order to implement testing of one or more components. The one or more emulation devices may be test equipment. Direct RF coupling and / or wireless communications via RF circuitry (e.g., which may include one or more antennas) may be used by the emulation devices to transmit and / or receive data.
[0114] Augmented reality (AR) aims to integrate virtual world with physical world by applying and / or overlaying virtual information, assets, objects, and / or scenes etc., for example, on the physical world. In typical AR applications, one or more real world objects and the virtual objects coexist in same viewport of a user,thereby augmenting the user’s real sensory experience. Overall, the AR applications have three major features or characteristics. A first feature is a combination of the virtual world and the real world. A second feature is a real-time interaction with the user. The user of the AR applications may interact with an AR system in real-time using gestures, voice, and / or touch, etc., for example. A third feature is virtual object registration. The virtual object registration refers to tracking and / or positioning of the virtual assets and / or the virtual objects in the real world so that the virtual information and / or the virtual assets and / or the virtual objects may be perfectly superimposed onto the real world. One or more enabling technologies for the AR applications include but are not limited to multimedia, three-dimensional (3D) modeling, sensors, and / or computer vision (such as object recognition and / or object detection, real-time object tracking, and / or virtual assets registration etc., for example), and / or video rendering etc., for example.
[0115] In a typical AR processing pipeline, first, a camera of an AR device (ARD) captures one or more video frames from the real world. Then, the AR device may extract one or more features from the one or more video frames (an image and a video frame are used interchangeably in this disclosure). Using an extracted feature information, the AR device performs object recognition to recognize various one or more physical objects (POs) In particular, based on a business logic and / or a user input, the AR device may detect and / or identify the one or more POs as physical target objects (PTOs). The PTOs are the POs to be annotated with one or more AR effects. For example, a marker-based technique or a marker-less technique may be used for determining placement of a virtual object (VO) in the physical environment. Overall, an AR effect is a perceived AR experience of the user. Augmenting the PTOs with the AR effect is usually performed by superimposing the virtual assets and / or the virtual objects onto the PTOs in the real world. The virtual assets and the virtual objects are used interchangeably in this disclosure.
[0116] In some examples, simultaneous localization and mapping (SLAM) may be performed to establish a map while at the same time locating the AR device in the map. One or more operations and / or processes are performed to enable the AR device to extract useful knowledge about the real world and / or the physical world. Based on a business logic, the AR device may select an appropriate virtual asset and / or a VO to be superimposed onto a PTO. For example, the virtual asset and / or the VO may include but is not limited to a static sign, a dynamic cartoon character, and / or a decoration, etc. During this process, the VO needs to maintain a precise alignment with the PTO to be annotated in the physical environment. This may be achieved through 3D tracking and registration. Formally, a positioning process of overlaying the VO in an exact and / or perfect position in the real world is termed as the VO registration It may be seen that success of the VO registration depends on an accuracy of a knowledge of the physical world known to the AR device For example, a 3D map and / or model about the physical world generated by the AR device via SLAM may be useful to accurately estimate a pose (i.e. position and / or orientation etc.) of a camera of the AR device, which is a piece of essential information for the VO registration. The AR device may also track and / or react to various real-time changes (such as but not limited to the user’s current position, head angle, gestures, and / or motion, etc , for example) in order to re-establish and / or align a coordinate system based on to the user's current viewport tomaintain the VO in a correct position. In addition, in order to avoid repetitive PTO detection in every video frame, the AR device may use an initial video frame as an input and perform object tracking between multiple frames. Finally, the AR device may perform rendering to create a final AR effect, which may be delivered and / or shown on a display of the AR device for presenting to the user as the AR output.
[0117] In general, for a given physical environment “X”, a typical AR scene design is to design an AR scene, which is to be applied for creating and / or delivering a desired AR experience to the user when the user and / or the AR device is in the physical environment . The AR scene design may involve two major tasks, viz, (i) PTO determination for the physical environment X and (ii) VO selection and / or VO preparation.
[0118] For the PTO determination in the physical environment X, the PTO may be the PO in the real world that is to be augmented with the AR effect. In other words, the AR device may superimpose the virtual information (e g. the VO) onto the PTO so that from the user’s viewport, the physical world is integrated and / or enhanced with the virtual asset, information, and / or contents etc., for example. Which of the POs are to be considered as the PTOs may be determined based on the business logic and / or an AR application. Detecting and / or recognizing the PTOs often needs a support of various techniques such as but not limited to object detection and / or object recognition technologies in computer vision domain For example, in the marker-based VO registration approach, the AR device may capture, recognize, and / or locate a visual marker (e.g. a fiducial marker) in the video frames generated by the camera. Then, the VO may be annotated onto the visual marker. In an example using the marker-less VO registration approach, one or more applications may support multiple AR effects and may leverage one or more natural features to detect one or more human faces so that the user may add different face effects in the real-time (such as but not limited to adding a hat, beards, and / or other virtual objects etc., for example).
[0119] For the VO selection and / or preparation, the AR device may decide for a given PTO, what kind of VO may be superimposed on to the PTO. Typically, the VO is a 3D virtual asset, which may be represented as a file, which includes one or more data structures and / or information about the VO, such as but not limited to shape, geometry, color, and / or texture etc., for example (i.e. deciding an appearance of the VO). A behavior of the VO may also be configured. The appearance of the VO may refer to what the VO looks like (e.g. a Hello Kitty cat, a 3D hat, and / or a cartoon character, etc.) including corresponding shape, color, and / or texture, etc., for example. Typically, a VO designer may use an AR authoring tool to design the 3D virtual asset and / or the VO, which may be published in a 3D asset store for usage by the AR application. The VO may also have a planned behavior, and / or the VO may provide one or more application programming interfaces (APIs) so that the behavior of the VO may be configured when the VO is used in the AR scene. In conventional AR applications, preparing the VO is rudimentary. For example, the AR application may add a face effect to the user’s face. The VOs (which are typically basic cartoon-like effects, such as but not limited to a hat, and / or a beard etc., for example) are often stored locally and in a small size (e.g., as small add-ons and / or additions etc.) in the conventional AR applications.
[0120] In some examples, in the AR scene design, the VO selection and / or preparation may be conducted first, and then the PTO determination may be performed. For example, the AR design may first decide what VO is to be used, and depending on the behavior of the VO, corresponding PTOs may be selected and / or decided.
[0121] 3rd Generation Partnership Project (3GPP) Service Architecture 4 (SA4) has published some specifications focusing on the AR, virtual reality (VR), and extended reality (XR). For example, TR 26.918 includes a VR audio and video content production process, VR business use cases, VR audio and video quality assessment, etc. TS 26.118 defines certain operation points for VR streaming media services, as well as a VR client reference architecture for the VR streaming media services, etc. TR26 928 introduces XR-related content, including various XR use cases, XR basic technical background (e.g. XR delivery, XR rendering, and / or XR visual formats, etc.), and how the XR delivery may be realized under a 5G system architecture. TR 26.998 focuses on a design of AR glasses and defines three AR user equipment (UE) architectures. For example, Type-1 AR devices conduct standalone AR processing. However, Type-2 and Type-3 AR devices need to leverage computing resources on an associated UE (or wireless transmit / receive unit (WTRU)) or in an edge processor and / or a cloud processor. In addition, latest 3GPP standardization activities are ongoing such as TS 26.119 about certain media capabilities for the AR and TR 26.806 which focuses on smart tethering for the AR glasses
[0122] In addition, an SA1 study item related to metaverse (e.g., TR 22.856) focuses on a feasibility study about how to offer shared and / or interactive user experience of local AR contents and / or services, among one or more users in proximity.
[0123] Blockchain technology is a technology that jointly leverages and builds on top of a few existing techniques such as but not limited to cryptography, hashing, Merkle tree, distributed ledgers, peer-to-peer (P2P) networking, and / or consensus protocols etc., for example. The blockchain technology integrates the techniques together to enable a system that may provide advanced features such as but not limited to decentralization, immutability, transparency, and / or security etc., for example. A blockchain system may be referred to as any system using the blockchain technology. Multiple blockchain nodes (BCN) are connected via peer-to-peer (P2P) links and form a mesh P2P network, over which one or more transactions and / or one or more blocks are broadcast among all the blockchain nodes Each blockchain node may connect to multiple other blockchain nodes as neighbors or neighboring blockchain nodes. Any applications using and / or supported by the blockchain system may be referred to as blockchain applications. The blockchain system is underpinned by one or more underlying blockchain networks which include multiple participating blockchain nodes. Each blockchain node may host one or more distributed blockchains (e.g., a form of distributed ledgers) and may participate in the blockchain system. For example, the blockchain nodes may broadcast one or more blockchain transactions and / or the one or more blocks among each other using P2P networking. The blockchain nodes may also perform one or more consensus protocols with other blockchain nodes to enable distributed trust without relying on a centralized party. Examples of a blockchain transaction include but are not limited to adigital representation of a real-world transaction, a digital record of physical assets, a digital record of a physical event, a digital record of any action in an information system, a digital payment, and / or a digital smart contract etc., for example. A block may group multiple blockchain transactions together. The blockchain is a data structure to chain a growing number of blocks.
[0124] In an example, FIG. 2 illustrates a general workflow of a blockchain system. FIG. 2 also shows multiple existing techniques that are associated with the blockchain system.
[0125] The blockchain system is accessible and / or accessed by multiple users. At (1), each participating user generates new transactions independently. Each user is associated with a corresponding user identifier and / or a corresponding account identifier, which may include a hash of a public key of the user. Each new transaction may be signed using a private key of the user. After a new transaction is generated, the user may transmit and / or share the new transaction to the blockchain network.
[0126] At (2), the new transaction may be first received by one or more blockchain nodes. The one or more blockchain nodes may verify an integrity of the new transaction using the public key of the user, which may be included in the transaction. After the verification and if the new transaction is valid, the new transaction may be relayed and / or broadcasted within the blockchain network Eventually, all blockchain nodes in the blockchain network may receive and store a copy of all newly generated and valid transactions.
[0127] At (3), certain blockchain nodes (referred to as mining Nodes and / or full nodes) may group multiple newly generated and pending transactions together to generate a new block. The new block may include a block header and a block body. The block header may include a hash of a current block, a hash of a previously confirmed block, and a hash of all included transactions (e.g. the Merkle tree). The block header may include additional information based on a consensus protocol used by the blockchain network. The block body may include content of all the included transactions. Each mining node may independently attempt to create a new block.
[0128] In (4), the mining nodes may independently attempt to create the new block. The mining nodes may use same consensus protocol (e.g., Proof-of-Work (POW)) and reach an agreement on who (i.e., a winner) is allowed to insert the block into the existing blockchain. The winner of the consensus protocol may share and / or transmit the newly generated block to the blockchain network. The new block may be broadcasted to all other mining nodes. The other mining nodes may receive the new block and verify the new block.
[0129] At (5), after the newly generated block is verified, the new block may be successfully appended to an existing blockchain, since the new block includes a hash of the previous block (i.e., the last block of the previous blockchain).
[0130] Wearable AR devices, such as but not limited to AR glasses etc., for example, may be a next big trend. Similar to smartphones, the AR glasses may support most functions and / or operations that may be done with the smartphones, such as but not limited to making phone calls, and / or accessing the Internet, etc , for example. People may wear the AR glasses all the time, and the AR glasses may capture multiple real-time images and / or videos using one or more built-in forward high-definition (HD) cameras without any humanintervention. The captured images and / or videos may include outdoor street views and / or indoor views, e.g., if a human user is walking inside a shopping mall, etc. The AR glasses may superimpose various virtual assets onto the physical world in the user’s viewport, thereby enabling a vivid AR experience.
[0131] Conventional AR experiences are basic and rudimentary. For example, in some conventional implementations, a given AR experience is typically realized by a pre-designed AR scene, e.g. the PTOs to be annotated are pre-defined (e.g. a marker is placed as an anchor etc.), the VOs are also pre-selected and / or pre-configured. However, the predefined AR scene implementations may be inefficient and may not provide dynamic and / or customized AR scene designs. For example, it may not be possible to pre-design the AR scene for an unknown and / or new physical environment since no prior knowledge (e.g., knowledge regarding what kinds of POs are present in the environment and characteristics of the PCs, such as but not limited to locations, colors, and / or orientations of the POs, etc.) can be leveraged by the AR device for completing the AR scene design beforehand. In future, the AR devices are expected to bring sophisticated and customized AR experiences to the users even when the users enter into the new and / or unknown physical environments for a first time, where no pre-defined AR solution is available for use. This may require the AR device to dynamically (i.e., in an online and / or real-time manner) complete a rich AR scene design for the unknown and / or new physical environment. The dynamic and / or customized AR scene design may also imply more complexity, compared to current pre-designed AR solutions. For example, the dynamic and / or customized AR scene design may involve dynamic PTO detection and / or recognition associated with physical world understanding. Based on an understanding of the physical world, the dynamic and / or customized AR scene design may involve dynamic VO selection (e.g. an ARD may superimpose a specific type of VO onto a specific physical object, based on a user preference etc.). However, a desired VO and / or the specific type of VO may not be locally available to the ARD, and some of the VOs may be huge in size and may need to be downloaded from a VO repository in time. For example, a typical 3D human character published by an example VO repository may be in a size of 300-500 MB. The size may grow to over 1 GB and the AR scene may require many such large-size VOs In addition, the VO behavior preparation may also be more computationally complex. For example, different users may want the VO to have different customized behaviors (for e.g. based on the user preference etc.) All of these contribute to an increased complexity of future AR scene design and may cause processing delays, thereby leading to unsatisfactory AR experience
[0132] FIG. 3A illustrates a first example use case of a dynamic and / or customized AR experience according to one or more embodiments. In this example, the dynamic and / or customized AR experience relates to exploring a new and exciting city street, however it may be understood that the dynamic and / or customized AR scene design may be used for any types of use cases, and is not limited to the example use case shown in FIG. 3A. In this use case, a user, Lisa 310 may wear an AR glasses (also denoted as “AR Device-1”, or “ARD-1 320” at multiple instances) and may visit a new shopping street (just built in a downtown area of a tourist city). While walking down the street, the ARD-1 320 captures one or more video frames from the physical world and analyzes the one or more video frames in real-time using techniques such as but no limited to imagerecognition and / or video processing, for example Then, a real-time and / or dynamic AR scene design is constructed by considering needs and / or user preferences of Lisa 310 (which may be based on the information stored in a user profile corresponding to Lisa 310 and / or could be indicated by Lisa 310) For example, the ARD-1 320 needs to understand the physical world, then determine and / or identify one or more POs as one or more PTOs that need the AR effects (based on user preferences indicated by Lisa 310, for example). For example, Lisa 310 may prefer to see a lively AR effect in which a cartoon character may jump here and there. Therefore, the ARD-1 320 may identify one or more jumping points as the POs, such as the roof of a street truck 340, or the roof of a street building 350 as the PTOs Once the POs that are the PTOs are decided, the ARD-1 320 decides what kinds of the VOs may be selected, what types of actions and / or behaviors may be assigned to the VOs, etc. For example, a 3D virtual character (e.g. a virtual superman character, also referred to as “superman 330”) may be adopted based on the user preferences indicated by Lisa 310. Also, the superman 330 may have a customized behavior such as walking along with Lisa 310 and introducing a popular local pizza style served by a nearby pizza shop in the street building 350. In addition, Lisa 310 may prefer the superman 330 to jump around Lisa 310 in order to have a more vivid AR experience. During the dynamic and / or customized AR scene design, the ARD-1 320 may download (if not stored locally) and prepare a high-fidelity VO (i.e. a 3D superman asset, for example) with a detailed behavior as discussed above. Also, one or more VO parameters such as but not limited to body color, brightness, and / or texture of the superman 330 may be configured to match a current daylight brightness of the street.
[0133] The previously described use case demonstrates an example AR experience with the dynamic and / or customized AR scene design for the new and / or unknown environment. To realize such an advanced AR experience, various AR scene design-related tasks, such as but not limited to understanding the new and / or unknown physical environment (e.g., identifying the PTOs etc.), selecting the VOs, and / or preparing the VOs may be performed in a quick manner, without causing any perceivable delay by the user.
[0134] In many embodiments, a collaborative AR resource sharing system and a collaborative AR resource sharing method may be provided to facilitate the dynamic and / or customized AR scene designs in the new and / or unknown environments In that, understanding the physical environment is essential for initiating the AR scene design (for the new and / or unknown environment etc., for example). One way of learning the new and / or unknown physical environment is for the ARD (e.g. the ARD-1) to enter into the physical environment and conduct real-time understanding and / or learning (such as conducting SLAM and / or PTO determination etc., for example). However, such an approach may require the ARD-1 to have sophisticated implementation complexity and very efficient and / or powerful AR processing, which may not be available to the ARD-1. Further, preparing and / or downloading the large-size (e g., more than 1 GB) VOs may also cause more delay. For such cases, the collaborative AR resource sharing method may be used among multiple ARDs including the ARD- 1 . In the collaborative AR resource sharing, different ARDs may share different types of AR resources, insights, and / or AR assets with each other. For example, other ARDs in a physical environment and / or location may share with the ARD-1 (i.e Lisa) one or more AR resources which may be useful and / or which may be relatedto the physical environment (which is still unknown and / or new to the ARD-1). Examples of the one or more types of the AR resources include but are not limited to any type of data and / or information describing the physical environment, which may refer to one or more fresh and / or raw images captured in the physical environment by other ARDs, and / or a knowledge about the physical environment, such but not limited to one or more maps, the POs detected and / or extracted from the one or more images of the physical environment, etc. Through the AR resource sharing (either raw image sharing and / or knowledge sharing), the ARD-1 may perform the dynamic and / or customized AR scene design for the new and / or unknown physical environment significantly earlier (for e.g. even before the ARD-1 has arrived and / or entered into the new and / or unknown physical environment). As a result, the ARD-1 would have sufficient and / or more time for the dynamic and / or customized AR scene design and the ARD-1 may not need to possess high-end and / or powerful AR processing capability, thereby enabling new ARD form-factors such as light-weight AR glasses.
[0135] FIG. 3B illustrates a second example use case of a dynamic and / or customized AR experience according to one or more embodiments. In FIG. 3B, an example of the collaborative AR resource sharing technique is presented. As shown in FIG. 3B, Mike 315 is walking from a Location B to a Location C. Lisa 310 is 300 meters behind Mike 315 and is currently at a Location A but also walking toward the Location B. Soon thereafter, Lisa 310 is navigated by her smartphone and also plans to walk from the Location B to the Location C. In this example, the street between the Locations B and C is defined as the physical environment X, which is the new and / or unknown physical environment to Lisa 310 and the corresponding ARD-1 320. An AR device (e.g. ARD-2 325) worn by Mike 315 captures images from the physical environment X. The captured images may be shared by the ARD-2 325 of Mike 315 as one or more AR resources 335 with the ARD-1 320 of Lisa 310. The one or more AR resources 335 may facilitate the dynamic and / or customized AR scene design by the ARD-1 320. That is, Lisa 310 may be presented with the dynamic and / or customized AR scene generated by the ARD-1 320 using the one or more AR resources 335 received from the ARD-2 325 when Mike 315 travels through the physical environment X.
[0136] In this example, two logical roles are involved: an AR resource provider (ARRP) and an AR resource consumer (ARRC). Here, the ARD-2 325 worn by Mike 315 is the ARRP since the ARD-2 325 shares the images (i e. the one or more AR resources 335) with the ARD-1 320 worn by Lisa 310 which is the ARRC. The collaborative AR resource sharing may provide effective quality of service (QoS) management for the AR resource sharing between the ARRP and the ARRC. Typically, the ARDs (e g. the ARDs functioning as ARRPs and / or ARRCs) may be personal devices, and hence, the ARDs may not trust each other (e.g., no prior service agreement may be defined between the ARDs). Similarly, the ARRPs may not have obligations for serving the ARRCs, and the ARRPs may not have an incentive to function as professional resource providers with promised QoS guarantees. As a result, a quality of the AR resources shared by the ARRPs cannot be assumed to meet a specific quality desired by the ARRCs. For example, the resolution, focus, and / or size of the images captured by an ARRP may not be as good as expected by corresponding ARRC. In another example, the images shared by the ARRP may be out of date, i.e. a freshness of the AR resources may be another issue.Since different ARDs may not trust each other and the AR resource sharing usually may happen on an ad hoc basis, an effective quality management solution is needed for supporting AR resource quality management in the collaborative AR resource sharing.
[0137] Therefore, there is a need for new and enhanced methods, procedures, and / or systems to enable trust between the ARDs so that the AR resources may be shared and to manage the quality of the AR resources to satisfy the expectations and / or requirements of the ARRCs.
[0138] Accordingly, in many embodiments, the present disclosure provides various techniques for the collaborative AR resource sharing that facilitate quality management and / or quality enforcement.
[0139] In various embodiments, a decentralized AR resource sharing procedure is provided. The decentralized AR resource sharing procedure uses the blockchain to facilitate quality management and / or quality enforcement of the AR resources shared between the ARDs
[0140] In one embodiment, an AR resource sharing procedure may be conducted in an ad-hoc mode or in direct manner between the ARRCs and the ARRPs (without requiring a centralized AR resource repository in the direct manner). The decentralized AR resource sharing procedure includes an AR resource quality management mechanism. As such, the AR resource sharing may be spontaneous, ad hoc, and at times random. The ARRPs may not have obligations (e.g., service agreements etc.) for serving the ARRCs. As a consequence, it may be possible that the quality of the AR resources shared by the ARRPs may not always meet the quality requirements of the ARRCs. To address this need, the disclosure provides the quality management and / or quality enforcement mechanisms using the blockchain network. That is, the disclosure provides a blockchain-based quality management and / or enforcement mechanisms for the decentralized AR resource sharing procedures between the ARRPs and the ARRCs. Overall, the blockchain may support quality management during the AR resource sharing procedure. Adopting the blockchain has advantages including but not limited to enabling trust among untrusted entities (such as the ARRPs and the ARRCs) so that the ARRPs and the ARRCs may build a trustworthy collaboration relationship for the AR resource sharing, providing data tamper-proof support (so that one or more contracts between the ARRPs and the ARRCs for the quality management may be effectively enforced since the one or more contracts are permanently recorded in the blockchain network, which enables non-repudiation), supporting traceability (e.g. how the AR resources are generated and / or shared by the ARRPs may be traced), supporting native incentive mechanisms with reward allocation. For example, the ARD may function as the ARRP to earn rewards and / or service fees for serving other ARDs. In turn, such rewards and / or service fees may be used by the ARD when the ARD functions as the ARRC for requesting other AR resources from other ARDs
[0141] Multiple embodiments are disclosed herein to describe new and improved techniques to realize the AR resource sharing with desired quality between the ARRPs and the ARRCs.
[0142] In one embodiment, a smart contract-based AR resource quality management is provided. In an example, the AR resource quality management is realized by establishing a collaboration relationship (e.g. thesmart contract) between the ARRP and the ARRC so that the ARRP complies with the collaboration agreement for providing the AR resources with the quality desired by the ARRC.
[0143] In another embodiment, an endorsement-based AR resource quality management is provided. In an example, the AR resource quality management is realized without establishing the collaboration agreement between the ARRP and the ARRC Instead, a blockchain-based reputation system is leveraged by assuming that each party (i.e. the ARRP and / or the ARRC) participating in the AR resource sharing may want to perform well in order to maintain a high reputation in an AR sharing community (for e.g. the higher the reputation is, the more are the chances that the ARD can be involved in future AR resource sharing events to serve other ARDs for potential rewards and / or to obtain the AR resources from the other ARDs).
[0144] In various embodiments, blockchain technology is used as a generic term to represent much broader distributed ledger technology. That is, the blockchain technology and distributed ledger technology are used synonymously or interchangeably in this disclosure. As such, the proposed ideas in this disclosure are also applicable to any specific blockchain technology and / or distributed ledger technology.
[0145] This disclosure uses the 3GPP 5G system as an example of a wireless system. However, the embodiments discussed herein may also be used for the future generations of 3GPP system (such as 6G and beyond), as well as any other types of wireless and / or fixed access systems (such as Wi-Fi etc., for example).
[0146] When explaining the details of the embodiments, the WTRUs, the UEs, the base stations, and / or the network functions are used as examples of the entities in the wireless system, and it may be understood that the embodiments are not limited to these examples. The embodiments discussed herein may be applied to any terminals, devices, and / or entities, such as but not limited to laptops, Internet-of-Things (loT) devices, equipment, future cellphones, drones, roadside units, TV set-top boxes, gateways, access points, satellites, sensor nodes, robots, machines, routers, base stations, radio access network central units, radio access network distribution units, radio access network radio units, network functions in 5GS and / or 6GS, etc , for example.
[0147] In addition to the examples of one ARRP and one ARRC, the embodiments discussed in this disclosure may also be applicable to other contexts, for example, one ARRC may be served by multiple ARRPs, and / or one ARRP may serve multiple ARRCs.
[0148] Various embodiments are presented using a scenario where the ARRC needs to obtain the AR resources associated with the new and / or unknown physical environment before the ARRC visits the new and / or unknown physical environment (e.g., the ARRC knows nothing about the physical environment before visiting). One or more embodiments may also be applicable to scenarios where the ARRC is already in the physical environment, however, the ARRC may not have a capacity and / or capability to obtain the AR resources required by the ARRC, and therefore may need to request other ARRPs for an AR resource provisioning.
[0149] Although this disclosure focuses on the resource sharing in the application of AR, the embodiments discussed herein may also be applicable to resource sharing in other types of applications. To generalize the embodiments discussed in this disclosure, the ARRP may be generalized as a resource provider (RP), whichcould provision various types of resources. The ARRC may be generalized as a resource consumer (RC), which needs certain desired resources, meaning that the RC may need to consume resources provisioned by the RP In an example, the methods and / or systems in this disclosure may also be used in one or more artificial intelligence (Al) and / or machine learning (ML) applications. An AI / ML RP may provide desired AI / ML model inputs (as AI / ML resources) to a given AI / ML RC, which cannot obtain the AI / ML model inputs. Similarly, the RP not only can share original AI / ML resources to the RC, but also can conduct certain pre-processing on the original AI / ML resources and then deliver the pre-processed AI / ML resources to the RC (e.g., so that the RC may directly consume the pre-processed AI / ML resources).
[0150] This disclosure considers the AR resource sharing between the ARRPs and the ARRCs, which are two logical roles of ARDs. However, a specific ARD may function as the ARRP in one resource sharing event and may function as the ARRC in another (or the same) resource sharing event.
[0151] The representative AR resources focused by this disclosure mainly refer to the following two types of resources that are used for understanding a physical environment (which is an essential information input for the AR scene design), viz. Type-1 AR resources such as the original AR resources and Type-2 AR resources such as the processed (and / or pre-processed) AR resources. In an example, the AR resources may also include metadata and / or other information associated with the AR resources.
[0152] Examples of the Type-1 AR resources, i.e., the original AR resources may include but are not limited to original video frames and / or original images captured by the ARRP in a specific physical environment. This forms a most basic form of the AR resource sharing. In this type of the AR resource sharing, the ARRC may request the one or more ARRPs to share the original and / or raw video frames, video and / or images captured at a desired location, which may be unknown to the ARRC. In this case, the ARRP does not need to conduct any processing on the captured video frames, video, and / or images which may be delivered to the ARRC via a communication link (e.g., 3GPP Device-to-Device (D2D) communication, 5G-NR, customer premises network (CPN), local area network (LAN), future 6G wireless communication link, Wi-Fi, and / or any other communication links etc.). This type of AR resource sharing may be applicable to several scenarios, e.g. when the ARRP currently does not have sufficient computing resources to help the ARRC for processing the video frames, the video, and / or the images. However, since the original video frames, video, and / or images could be in a large size (e.g. having a high resolution), this approach may incur high communication traffic for transferring the original video frames to the ARRC. In this approach, the ARRC is responsible for processing the original video frames, video, and / or images to extract useful physical world knowledge. After that, the extracted physical world knowledge may be used by the ARRC for performing the dynamic and / or customized AR scene design for the physical environment. Here, the Type-1 AR Resources may not be limited to the video frames, video, and / or images. Instead, the Type-1 AR resources may be any AR-related original resources that may be beneficial and / or useful for the ARRC, such as but not limited to audio, radar, lidar, and / or other sensory data.
[0153] The Type-2 AR resources such as the processed resources and / or the pre-processed resources may include the physical world knowledge extracted from the original AR resources, such as but not limited toa map of the physical environment and / or a semantic perception of the physical environment (e.g. the detected POs in the physical environment and corresponding characteristics of the POs and / or PTOs). In the Type-2 AR resource sharing, certain processing may be conducted by the ARRP. For example, the ARRC may send an AR resource-sharing request (ARRSR) to the ARRP, by indicating needs and / or requirements of the ARRC, e.g. what kinds of physical world knowledge is needed by the ARRC. Accordingly, once the ARRP decides to serve the resource sharing request, the ARRP may process the raw video frames, video, images, and / or other raw AR resource data. An output of the processing may be the physical world knowledge needed by the ARRC. Finally, information indicative of the physical world knowledge may be delivered to the ARRC as the AR resources. The knowledge may be used by the ARRC for performing the dynamic and / or customized AR scene design.
[0154] In an embodiment, for the Type-1 AR resources (i.e. the original and / or raw AR resources), a quality of the AR resources may include one or more technical specifications associated with the AR resources and / or a freshness of the AR resources.
[0155] In an example, the technical specifications may include but are not limited to one or more physical targets to be captured by the ARRP. For example, in the use case discussed above, the ARD-1 may check Lisa’s user profile and may determine that the superman cartoon may be the VO needed and / or preferred by Lisa, and that Lisa may need and / or prefer the superman to jump around when the superman is introducing interesting physical objects (e.g. the street shops and / or restaurants) to Lisa. To realize such a customized AR scene design, the ARD-1 may obtain and / or request to obtain the images of the buildings (such as the pizza shop building) along the street between the Location B and the Location C (i.e. the buildings are the physical targets to be captured in this example). Not only that, some zoom-in images may also be useful if certain physical objects on those buildings may be interesting and / or useful for the AR scene design (e.g. a specific showcase window on a first floor of the pizza shop building). In an example, the physical target to be captured may not be indicated and the ARRC may request the ARRP to capture one or more images at a specific location using a specific camera pose and / or orientation (as indicated by one or more ARD technical settings), etc.
[0156] In an example, the technical specifications may include one or more technical settings for creating and / or capturing the AR resources. For example, the technical settings may indicate a required resolution, brightness, size of a video frame and / or an image to be generated by the ARRP using a camera of the ARRP.
[0157] In an example, the Type-1 AR resources may also include the freshness of the AR resources For example, the ARRP may generate and transmit the image to the ARRC which may be captured in the physical environment X with the desired technical specifications (e.g., needed image resolution, ideal camera focus, and / or image size etc.). However, the image may be captured in past (e.g., last night). Therefore, the image may be less useful or even useless to the ARRC if the ARRC needs images captured in daytime (e.g. the street may have different appearances during daytime and nighttime especially if various neon lighting or signs are used etc.).
[0158] In an embodiment, for the Type-2 AR resources, i.e. the processed AR resources and / or the pre- processed AR resources, the quality may include but is not limited to an accuracy of the AR resources and the freshness of the AR resources.
[0159] In an example, the accuracy of the AR resources may be determined and shared by the ARRP. For example, the ARRP may share a set of knowledge about the physical environment X with the ARRC (e.g., the physical environment that may be new and / or unknown to the ARRC). However, inaccurate knowledge may be not useful or may even be useless. Therefore, the accuracy determined and shared by the ARRP may be used to determine a relevancy of the AR resources.
[0160] In an example, the ARRP may determine the freshness of the AR resources. The physical world knowledge set may be stale. For example, the physical world knowledge about the shopping street generated in the daytime may not be applicable to the nighttime. Therefore, the freshness determined and shared by the ARRP may be used to determine the relevancy of the AR resources.
[0161] In an example, for a piece of an AR resource created by the ARRP, the ARRP may generate and / or share corresponding AR resource metadata (ARRM), which may include information describing the AR resource and one or more quality-related characteristics associated with the AR resource. The ARRM may include an identifier associated with the AR resource. The identifier may indicate which piece of the AR resource is associated with the ARRM. For example, the ARRP may create an image resource. The image resource may have a string-based identifier, which may include an ARRP identifier (ID) and / or a local image file ID associated with the image resource. The ARRM may include a timestamp of when the corresponding AR resource was created. The timestamp may reflect and / or indicate the freshness of the AR resource. The ARRM may include a geo-location of where the AR resource is created, e.g. a geographical location, network location, a civic address, an indoor location, camera pose and / or orientation, etc The geo-location may reflect and / or indicate whether the AR resource is captured in the physical environment X and at a desired location.
[0162] In an embodiment, for the Type-1 AR resources (e.g., the video frames, the videos, and / or the images), the ARRP may capture information such as but not limited to the physical targets being captured (e g. if the corresponding ARRC of the Type-1 AR resources has indicated to the ARRP about one or more types of the target objects are to be captured). Since the ARRP captures the video frames, the videos, and / or the images, the ARRP may identify and / or determine the physical targets captured in the video frames, the videos, and / or the images The ARRP may share an adopted technical setting for capturing and / or creating the AR resources. For example, for the given image captured by the ARRP, the technical setting may indicate the resolution, brightness, and / or size of the image and / or a camera pose (such as position, direction, and / or orientation etc., for example) information associated with the image.
[0163] In an embodiment, for the Type-2 AR resources (e.g a piece of physical world knowledge), the ARRP may capture information such as the timestamp of when the knowledge is created and / or the confidence of the knowledge. It is possible that the physical world knowledge extracted from the raw AR resources maynot be 100% accurate. Therefore, the ARD-2 (as the ARRP) may indicate how accurate the knowledge is, which may be a useful reference for the ARD-1 (as the ARRC).
[0164] For the AR resources (including the Type-1 and / or the Type-2 AR resources), the ARD-2 may also evaluate a freshness dynamics and indicate such information in the corresponding ARRM. The freshness dynamics may be implemented using a “valid period” parameter, meaning that after the AR resource is created, the AR resource is deemed to be static and / or non-changing within a time period as indicated by the valid period parameter. The reason is that the physical world keeps changing. Some of the physical objects may stay stable for a long time, while some of the physical objects may have significant changes in a short time, including their locations, colors, appearances, shapes, etc Accordingly, when the ARD-2 is generates the AR resource and / or the piece of the AR resource, the ARD-2 may identify, measure and / or determine the freshness dynamics. The freshness dynamics including the valid period parameter may be shared with the ARD-1 For example, the image of a building may not have significant changes in a short time, e.g. ten minutes. As such, the image may have low freshness dynamics. In comparison, the image of a busy road intersection may have big changes due to moving traffic flow on the road intersection, therefore, the image may have high freshness dynamics. The ARRC (i.e. the ARD-1) may consider the freshness dynamics for the AR scene design. For example, if the ARD-2 indicates that the image about the building is relatively stable in a near term (e.g within ten minutes), then the ARD-1 may have a more concrete AR design of the building. However, if the ARD-2 indicates that the image of the busy road interaction changes frequently, then the ARD-1 may just complete a preliminary AR scene of the road intersection Then, ARD-1 may conduct further enhancement and / or adjustment at a later stage (e.g., when the ARD-1 arrives at the road interaction), which may only cost a small amount of processing time (e.g., can be done in a real time manner).
[0165] In one embodiment, the AR resource quality management may be realized by establishing the collaboration agreement between the ARRP and the ARRC. In other words, when the ARRC wants to obtain the AR resources (e.g., either the Type-1 AR resources (original AR resources) or the Type-2 AR resources (processed AR resources)) from the ARRP, the ARRC and the ARRP may first establish the AR resource sharing agreement and / or the collaboration agreement. The collaboration agreement may be realized in form of the smart contract that may be deployed in the blockchain system. The smart contract not only records details of the needs and / or requirements of the ARRC (e g. what types of AR resources are needed by the ARRC etc.) but also prescribes the quality requirements for the AR resources desired by the ARRC. Due to the immutability and non-repudiation characteristics of the blockchain system, once the smart contract is recorded in the blockchain, the smart contract may be automatically executed. The smart contract may also describe the service fees and / or reward allocation instructions so that both the ARRP and the ARRC have obligations and / or incentives to enforce the collaboration agreement. For example, the ARRC may have to deposit certain payments and / or funds in the smart contract so that the ARRP does not have to worry about the ARRC for repudiating the service fee to be paid to the ARRP. Once the ARRP provides a desired AR resource, the smart contract may be triggered for automatic execution, i.e. the payment, the service fee, the rewards and / or thefunds may be automatically transferred to the ARRP. Similarly, the ARRC may assume that the ARRP has the incentive to provide the AR resources with the quality desired by the ARRC. For example, if the ARRP provides the AR resources with undesired quality, the smart contract may not allocate and / or transfer any payment, service fee, reward, and / or funds. In an example, the ARRP may also be required to either agree to one or more penalty policies as described in the smart contract (e.g. the penalty may be automatically deducted from an account of the ARRP if the ARRP consistently produces the AR resources with undesired quality) or even may even make certain deposits into the smart contract in advance so that the ARRP may try its best to serve the needs of the ARRC in order to avoid any penalty (e.g., if failing to provide the AR resource with the quality desired by the ARRC). For example, once the ARRP fails to provide the AR resources to the ARRC, the smart contract may automatically deduct the penalty from the account of the ARRP, which may be used as a compensation to the ARRC for the time lost and / or wasted.
[0166] In an example, the smart contract-based AR resource quality management procedure may include two stages, viz, enabling the AR resource sharing collaboration establishment using the smart contract (as shown in FIG. 4) and the AR resource sharing process with the smart contract enforcement (as shown in FIG. 5).
[0167] FIGS. 4A-4B illustrate an example process 400 of an AR resource sharing collaboration establishment using the smart contract for quality management, according to one or more embodiments. The process 400 may be performed between a first WTRU (WTRU-1 402), a second WTRU (WTRU-2404), and a blockchain node (BCN-1 406). The WTRU-1 402 may function as a consumer device (i.e. the ARRC) and the WTRU-2 404 may function as a provider device (i.e. the ARRP).
[0168] At 410, the WTRU-1 402 and the WTRU-2 404 may access the blockchain system and / or a blockchain infrastructure via the BCN-1 406. For example, the BCN-1 406 may be one of the operating blockchain nodes deployed close to the WTRU-1 402 and the WTRU-2 404.
[0169] At 411, the WTRU-1 402 will travel to the new and / or unknown physical environment X soon. Based on the business logic, the user of the WTRU-1 402 may need the AR experience when traveling in the physical environment X. Accordingly, the WTRU-1 402 needs to dynamically design the customized AR scene and make the customized AR scene ready for presenting to the user when arriving in the physical environment X.
[0170] At 412, to design the customized AR scene for the physical environment X, the WTRU-1 402 needs the AR resources captured from the physical environment X. Here, the AR resources may refer to either the Type-1 AR resources (such as the raw video frames, video, and / or images captured from the physical environment X) or the Type-2 AR resources (e.g. the physical world knowledge about the physical environment X).
[0171] At 413, the WTRU-1 402 generates and transmits a first AR resource solicitation request-1 (ARRSR- 1) to one or more candidate ARRPs (such as the WTRU-2 404) by indicating the needs and / or the quality requirements. For example, the WTRU-1 402 as the ARRC may use one or more different approaches to discover the candidate ARRPs for delivering the ARRSR-1.
[0172] In an example, the an ARRC-ARRP pairing engine (PE) may be included in the AR resource sharing system and the ARDs (including the ARRPs and the ARRCs) may be provided with a contact address of the PE, e.g. through pre-configuration, such as address provisioning. Accordingly, the ARDs (including the ARRPs and the ARRCs) may register themselves to the PE as the ARRPs and / or the ARRCs or both. The WTRU-1 402 may transmit the ARRSR-1 to the PE so that the PE may forward the ARRSR-1 to one or more potential candidate ARRPs, such as those ARDs that currently are in the physical environment X.
[0173] The WTRU-1 402 may also rely on the 3GPP D2D direct discovery approach for identifying the potential candidate ARRPs. For example, certain ARRPs may monitor for ARRSR requests on a PC5 channel, and accordingly, when the WTRU-1 402 as the ARRC announces the ARRSR-1 on the channel, multiple candidate ARRPs may receive the ARRSR-1 and evaluate whether they want to serve the ARRSR-1. If so, the ARRP (such as the WTRU-2 404) may transmit a response so that the ARRC and the ARRP may start to build a D2D communication link for the AR resource sharing.
[0174] In the 3GPP D2D embodiment, the ARRSR-1 , may include multiple parameters associated with the needs and / or requirements (e.g. the quality requirements) of the WTRU-1 402. The ARRSR-1 may include an identifier of the WTRU-1 402. Examples of the identifier include but are not limited to WTRU-ID, an IP address, a blockchain account address, and / or a decentralized identifier, etc. The ARRSR1- may include a current location of the WTRU-1 402. The ARRSR-1 may also include a moving speed of the WTRU-1 402 and / or other mobility-related information, such as but not limited to a planned travel path, a list of geo-locations, a time of arrival, etc. The ARRSR1- may include information and / or description of an interested physical environment. This may indicate one or more locations and / or physical environments in which requested AR resources may be collected and / or captured. For example, as the use case discussed above, Lisa is currently at the Location A and is planning to travel the street between the Location B and the Location C soon. The physical environment X, in this case, may relate to the street between the Locations B and C, which may be new and / or unknown to Lisa. Therefore, the AR resources needed by Lisa may be captured from the street. For example, the physical environment may be described using a list of geographical locations (such as but not limited to geo-locations, co-ordinates, and / orGPS locations etc., for example), or a list of road network names (e.g. street and / or road etc.), or a list of landscapes names (such as but not limited to the shops, restaurants in a shopping mall and / or on the street etc., for example). The ARRSR-1 may include the AR resources that are to be captured from the physical environment interested by the WTRU-1 402. This may indicate what types of the AR resources are needed by the WTRU-1 402. For example, two types of the AR resources may be required by the WTRU-1 402, i e. the Type-1 AR resources (i.e. the original AR resources) and / or the Type-2 AR resources (i.e. the processed AR resources). The WTRU-1 402 may request two kinds of resources in a single ARRSR- 1.
[0175] The ARRSR-1 may include the quality requirements for the AR resources that are to be captured from the physical environment interested by the WTRU-1 402. The quality requirements may indicate the AR resource quality desired by the WTRU-1 402, such as, for the Type-1 AR resources (i.e. the original ARresources (e.g. the raw video frames, videos, and / or images or other types of sensory data captured in the physical environment X), the quality requirements may include the requirements related to what is to be captured. For the Type-1 AR resources, the quality requirements may indicate what kinds of physical world objects and / or physical target objects may be captured in the AR resources (If this requirement is present, the ARRP (i.e. the WTRU-2 404) may include such information in the corresponding ARRM of each Type-1 AR resources captured and / or generated by the ARRP). The quality requirements may include one or more technical parameters indicative of information which the ARRP may use to operate the camera to capture and / or generate desired images. For example, in the use case discussed above, the ARD-1 may want certain images about all the street buildings (such as the pizza shop building) between the Location B and the Location C. The ARRSR-1 may include the requirements related to the technical specifications of the AR resources, i e. how the AR resources may be captured. For example, this may indicate camera pose and / or orientation, the desired resolution, brightness, and / or size of the video frame, videos, and / or the images. The ARRSR-1 may include the requirements related to the freshness of the AR resource. For example, the WTRU-1 402 may only want images that are captured in the last 5 minutes. The ARRSR-1 may also include other non-technical requirements. For example, the WTRU-1 402 may indicate how many images i.e. a number of images that are needed from the physical environment X. In an example, the other requirements may include for a given physical object (such as the building), a number of images to capture, and / or direction, orientation, and / or pose etc. of the images In addition, the WTRU-1 402 may also indicate a sampling rate so that the ARRP may conduct continuous image capture when walking on the street in the physical environment X.
[0176] For the Type-2 AR resources (i.e. the processed AR resources (e g. the physical world knowledge extracted from raw video frames, video, and / or images captured from the physical environment X etc.)), the quality requirements may include the requirements related to what is to be captured. For example, for the Type- 2 AR resources, the requirements may indicate the knowledge that may be related to what kinds of physical world objects and / or physical target objects. For example, the WTRU-1 402 may indicate a list of potential interested POs, so that the ARRP may pay more attention to detecting such POs and generate corresponding knowledge. The requirements may be used when evaluating the accuracy of the Type-2 AR resources. The ARRSR-1 may also include the requirements related to the freshness of the resource. For example, the WTRU- 1 402 may only want fresh knowledge about the physical environment X, e.g. the knowledge may be extracted from the raw video frames, videos, and / or images that were captured in the last 5 minutes. The requirements may be used when evaluating the freshness of the Type-2 AR resources.
[0177] The ARRSR-1 may include one or more blockchain capabilities. The blockchain capabilities may indicate that the WTRU-1 402 has access to the blockchain system. For example, the WTRU-1 402 may indicate a specific blockchain system that the WTRU-1 402 would prefer to use for establishing a formal AR resource sharing agreement with the ARRP.
[0178] The ARRSR-1 may include an indicator for using the smart contract. The indicator may indicate that the WTRU-1 402 desires to sign the smart contract for establishing the formal AR resource sharing relationship with the one or more ARRPs.
[0179] The ARRSR-1 may further include one or more clauses, smart contract terms, and / or contents to be included in the smart contract The smart contract terms may indicate what contents may be included in the smart contract and how the smart contract may be executed. For example, the smart contract may record all the quality requirements associated with the AR resources so that the ARRP may obtain the corresponding service fee and / or rewards only if the ARRP provides the AR resources with the desired quality. Some popular clauses, smart contract terms, and / or contents may be reused on multiple occasions (i.e. in multiple smart contracts between different ARRPs and ARRCs), therefore such common and / or popular clauses, smart contract terms, and / or contents may be stored in the location (e.g. in the blockchain) and the ARRC may indicate a reference and / or a link to the location.
[0180] The ARRSR-1 may include, in the smart contract terms, the service fee to be paid by the ARRC. The ARRSR-1 may indicate how much service fee the WTRU-1 402 would like to pay if the WTRU-1 402 obtains the desired AR resources.
[0181] The ARRSR-1 may include, in the smart contract terms, the penalty fee to be paid by a serving ARRP. To enable the ARRP to have the obligation and / or the incentive to serve the ARRC, the ARRC may also propose some penalty-related policies with the ARRP. For example, a certain penalty may be incurred for the ARRP (e.g. the WTRU-2 404 ) if the ARRP has accepted the resource sharing request from the WTRU-1 402 (as the ARRC) but at a later time, the WTRU-2 404 decides not to serve this request (the penalty can be a compensation to the ARRC since the ARRP wastes the time of the ARRC) or if the ARRP irresponsibly accepts the ARRSR-1 but the ARRP does not have corresponding capability and / or capacity for serving the ARRSR-1.
[0182] The ARRSR-1 may include a blockchain account information. The blockchain account information may indicate the blockchain account of the ARRC so that other ARDs may check the account to make sure whether the ARRC has sufficient funds to pay the service fee for the AR resources.
[0183] At 414, many potential ARRPs (including the WTRU-2 404) may receive the ARRSR-1 Upon receiving the ARRSR-1, the WTRU-2404 may evaluate the ARRSR-1 to determine whether the WTRU-2404 is in the physical environment desired by the WTRU-1 402. The WTRU-2404 may evaluate whether the WTRU- 2 404 has the capability to generate the AR resources desired by the WTRU-1 402. The WTRU-2 404 may evaluate whether WTRU-2404 has the confidence to meet the quality requirements as indicated by the WTRU- 1 402. For example, the WTRU-2 404 may check a history work log to see if previously the WTRU-2404 was able to generate similar AR resources The WTRU-2404 may evaluate whether the WTRU-2404 is satisfactory with the service fee to be paid by the WTRU-1 402. The WTRU-2 404 may evaluate whether the penalty fee as indicated in the ARRSR-1 is affordable and / or acceptable to the WTRU-2 404.
[0184] If the WTRU-2 404 does not want to serve the ARRSR-1 , the WTRU-2 404 may send a reject response or just keep silent, i.e. not send any response. In this procedure, it may be assumed that the WTRU- 2 404 decides to serve the ARRSR-1
[0185] At 415, the WTRU-2 404 creates a draft smart contract (SC-1 ) for describing a detailed AR resource sharing agreement. For example, all the information described in 413 may be included. The SC-1 is to be deployed into the blockchain system. In an example, at 413, the ARRSR-1 may also be directly created in form of a smart contract, meaning that the SC-1 may also be created by the WTRU-1 402.
[0186] At 416, the WTRU-2 404 transmits a response to the WTRU-1 402 and confirms that the ARRSR-1 will be served. The draft SC-1 may also be sent back to the WTRU-1 402 in response.
[0187] At 417, the WTRU-1 402 may check the draft SC-1 and evaluate whether the SC-1 is acceptable to the WTRU-1 402. If not, the WTRU-1 402 and the WTRU-2404 may conduct a further negotiation until reaching a consensus on the details of the SC-1 . For example, the WTRU-1 402 may indicate to the WTRU-2404 which parts of the SC-1 may be rectified before the consensus may be reached. In this procedure, it may be assumed that the WTRU-1 402 (as the ARRC) may be responsible for deployment of the SC-1. Alternatively, this may also be performed by the WTRU-2 404. In that case, 419-421 may be performed by the WTRU-2 404 (as the ARRP). It may be possible that the WTRU-1 402 may receive responses from multiple ARRPs. If that is the case, the WTRU-1 402 may evaluate and select the one (or more) ARRP that has best chances to provide the AR resources with the quality desired by the WTRU-1 402. For the selected ARRP (e.g. the WTRU-2404), the WTRU-1 402 may conduct further negotiation until reaching a consensus on the details of SC-1. For other ARRPs that were not selected by the WTRU-1 402, the WTRU-1 402 may send a notification to terminate further interaction regarding the provisioning of the AR resources required by the WTRU-1 402.
[0188] At 418, the WTRU-1 402 deploys the SC-1 to the blockchain system via the BCN-1 406. All the useful information as described in 413 may be recorded in the blockchain system via the BCN-1 406
[0189] At 419, the BCN-1 406 records the SC-1 into the blockchain system (e.g. based on one or more consensus protocols). Based on the details of the SC-1, certain deposits may be deducted from the blockchain accounts of the WTRU-1 402 and / or the WTRU-2 404. For example, the service fee to be paid by the WTRU- 1 402 may be deducted from the account of the WTRU-1 402 as a deposit. Similarly, the potential penalty may be deducted from the account of the WTRU-2404 The SC-1 may become active once deployed into blockchain network. In an example, it is also possible that the SC-1 currently may not be in an active status and may need activation from the WTRU-2 404.
[0190] At 420, the BCN-1 406 may transmit an acknowledgment to the WTRU-1 402 that the SC-1 has been successfully recorded in the blockchain system.
[0191] At 421, the WTRU-1 402 may inform the WTRU-2 404 that the SC-1 has been recorded in the blockchain system. The WTRU-2404 may access the blockchain system to check existence and / or correctness of the SC-1 If the deployed SC-1 is as same as the final version that the WTRU-1 402 and the WTRU-2 404 reached in 418, the WTRU-2 404 may transmit a trigger to the SC-1 so that the SC-1 can be in an active status.Otherwise, if the deployed SC-1 is different from the final version that the WTRU-1 402 and the WTRU-2 404 have reached in 418, the WTRU-2 404 may not transmit the trigger to activate the SC-1.
[0192] At 422, once the SC-1 is in the active status, the WTRU-2 404 may create a corresponding job for serving the ARRSR-1, based on all the requirements as indicated in the ARRSR-1 . Such a job may be executed by the WTRU-2 404 for generating the AR resources required by the WTRU-1 402.
[0193] There may be some alternatives to the procedures illustrated in FIG. 4. In an example, it may be possible that the ARRC may have already deployed the smart contract (e.g. SC-2) in the blockchain system. Then, the ARRP may check the contents of the SC-2. If the ARRP agrees with the SC-2, the SC-2 can now be used for collaboration establishment between the ARRC and the ARRP. Similarly, it is possible that the ARRP may have already deployed the smart contract (e.g. SC-3) in the blockchain system. Then, the ARRC may check the contents of the SC-3. If the ARRC agrees with the SC-3, the SC-3 may be used for collaboration establishment between the ARRC and the ARRP. In both cases, 415-420 may not be needed.
[0194] In the procedure in 413, the ARRC may transmit the ARRSR to the ARRP. Alternatively, the ARRP may also transmit an AR resource advertisement request (ARRAR) to the ARRC. In an example, the ARRAR may indicate but is not limited to one or more of the types of AR resources that can be generated by the ARRP, the expected quality of the AR resources, the capabilities of the ARRP, the mobility plan of the ARRP, and / or any other useful information regarding the ARRP
[0195] Then, on the ARRC side, the ARRC may decide whether to agree to a particular ARRP for receiving the AR resources. If so, the smart contract may be established between the ARRP and the ARRC.
[0196] FIGS. 5A-5B illustrate an example process 500 of an AR resource sharing execution with quality management enforcement. The process 500 may be performed between a WTRU-1 502 (as the ARRC), a WTRU-2 504 (as the ARRP) and a blockchain node BCN-1 506.
[0197] At 510, the WTRU-1 502 may establish the AR resource sharing relationship with the WTRU-2 504 (as the ARRP) using the procedure illustrated in the AR resource sharing collaboration establishment using smart contract for AR resource quality management process. The smart contract SC-1 (established between the WTRU-1 502 and the WTRU-2 504 ) may be recorded in the blockchain system. Since the ARRP may be the personal WTRU and may not be able to act as a professional service and / or resource providers, the ARRC may need to conduct an AR resource quality evaluation upon obtaining the resources.
[0198] At 511, the WTRU-2 504 (as the ARRP) may be traveling in the physical environment X. Based on the SC-1 , the WTRU-2 504 may generate the AR resources required by the WTRU-1 502. For each piece of generated AR resource, the WTRU-2 504 may also create the corresponding ARRM.
[0199] At 512, the WTRU-2 504 may deliver the generated AR resources to the WTRU-1 502, along with the corresponding ARRMs.
[0200] At 513, the WTRU-1 502 may transmit an acknowledgment to the WTRU-2 504 about the reception of the AR resources.
[0201] At 514, the WTRU-1 502 receives the AR resources and evaluates the quality of the resources, based on the quality definition discussed in the definition of the AR resource quality and the ARRM.
[0202] In an example, when the shared AR resource is the Type-1 AR resource (e.g., the original raw video frames, videos, and / or images), the WTRU-1 502 may evaluate the technical specifications of the AR resources, which may be found in the corresponding ARRM. For example, for a given piece of the image, the WTRU-1 502 may partially evaluate all the quality aspects as described in the ARRM.
[0203] The WTRU-1 502 may compare an image and its corresponding ARRM in order to determine, verify and / or check (e g., to ensure) that the ARRM truly reflects the one or more technical specifications and / or characteristics of the image For example, the WTRU-1 502 may first check the captured physical targets as listed in the ARRM of the image. Then, the WTRU-1 502 may examine the image to determine whether the image has really captured those physical objects as listed in the ARRM. For example, if the WTRU-1 502 requires capturing images of street buildings, the WTRU-1 502 may evaluate whether the image is about a building of interest.
[0204] Similarly, the WTRU-1 502 may evaluate the adopted ARD technical setting for creation and / or generation of the AR resources. For example, the WTRU-1 502 may also evaluate the specific technical specifications of the image, such as its resolution, brightness, and / or size of an image (similarly, the WTRU-1 may also examine the image to validate whether the one or more technical specifications included in the corresponding ARRM are correct). Then, the WTRU-1 502 may further decide whether the technical specifications are compliant with the requirements as specified in the smart contract SC-1. Also, the WTRU-1 502 may evaluate whether the object is clear enough, i e. whether the WTRU-2504 used the appropriate and / or correct camera pose (e g. position, direction, and / or orientation) and focus, etc. The WTRU-1 502 may not be able to evaluate AR resource freshness until the WTRU-1 502 arrives and / or enters into the physical environment X.
[0205] In another example, when the shared AR resource is the Type-2 resource, the WTRU-1 502 may evaluate some general information, including but not limited to the physical objects being described in the knowledge and / or the timestamp of when the knowledge was created, which is to evaluate the freshness of the knowledge. Also, at this time point, the WTRU-1 502 may not be able to evaluate the accuracy of the Type-2 resource until the WTRU-1 502 arrives at the physical environment X.
[0206] At 515, the WTRU-1 502 may start to use the AR resources for the AR scene design, by assuming that the freshness of the Type-1 AR resources or the accuracy and / or freshness of the Type-2 AR resources meets the requirements as described in the SC-1 since the WTRU-2504 may not want to face a penalty. During the AR scene design, the WTRU-1 502 may determine the VOs that are needed (e.g. based on the user preference and / or other customization needs) and if the VOs are not locally available, the WTRU-1 502 may need to download the VOs from a VO repository (e.g. a cloud server and / or a cloud storage etc.). The WTRU- 1 502 may also determine where to superimpose the VOs (e.g. locations and / or positions of the VOs in the AR scene and / or relative positions of the VOs to the physical objects) on the POs in the physical environment Xbased on the physical world knowledge extracted from the AR resources shared by the WTRU-2504. After the AR scene design is complete, the AR scene may be used and / or rendered by the WTRU-1 502 when the WTRU-1 502 arrives at the physical environment X, which may be finally used to present the AR effect to the user of the WTRU-1 502. In addition, during the AR scene design, the WTRU-1 502 may also need to consider the resource change dynamics. For example, if the PO keeps changing, the WTRU-1 502 may just complete a preliminary AR design for this PO and prepare to have a further calibration and / or adjustment at a later time (e.g. at 516), before presenting the final AR effect to the user of the WTRU-1 502.
[0207] At 516, at a later time, the WTRU-1 502 may arrive and / or enter the physical environment X. The WTRU-1 502 may further evaluate and / or validate the freshness of the Type-1 AR resources, or the freshness and / or accuracy of the Type-2 AR resources provided by the WTRU-2 504 previously, which the WTRU-1 502 was not able to evaluate during 514 (since the WTRU-1 502 had not arrived in the physical environment X yet in 514). For example, for the image describing the building, the WTRU-1 502 may be stricter when evaluating the freshness of the image since the building is supposed to have low change dynamics in a short time (e g. five minutes). In comparison, for the image or the piece of knowledge describing the busy traffic intersection, the WTRU-1 502 may be more flexible and / or tolerant due to the high change dynamics. Depending on the quality evaluation (i.e. whether the AR resources shared by the WTRU-2 504 are fresh and / or accurate), the WTRU-1 502 may dynamically calibrate the AR scene design performed in 515, and then render the final AR scene in order to generate the desired AR effects.
[0208] For each piece of the AR resource received in 512, the WTRU-1 502 may complete all the quality evaluations on every aspect (e.g. the quality evaluations done in 514 and / or 516).
[0209] At 517, based on the overall quality evaluation as conducted in 514 and / or 517, the WTRU-1 502 may create an AR resource quality evaluation result based on the AR resource quality requirements as described in the SC-1.
[0210] At 518, the WTRU-1 502 may report the AR resource quality evaluation result to the SC-1 via the BCN-1 506, which may specify for the given AR resource, which quality requirements are met, and which are not. For example, in 518, the AR resource quality evaluation result is described as a blockchain transaction (e.g. transaction-1) submitted to the BCN-1 506. Once the transaction is recorded in the blockchain system based on certain consensus protocols, each BCN node in the blockchain system (including the BCN-1 506) may record a copy of the transaction-1. In the meantime, each BCN in the blockchain system has already hosted a copy of the SC-1 during the previous smart contract deployment. As a result, each copy of the transaction-1 will further trigger the smart contract execution on every BCN in the blockchain system (including the BCN-1 506).
[0211] At 519, the SC-1 deployed in the blockchain system determines that the quality evaluation result needs to be confirmed by WTRU-2 504. The SC-1 may generate a notification to be sent to the WTRU-1 502.
[0212] At 520, the BCN-1 506 transmits a notification to the WTRU-2 504 (generated by the SC-1) about the quality evaluation result reported by the WTRU-1 502. Alternatively, in 518, the WTRU-1 502 may alsotransmit a copy of the AR resource quality evaluation result to the WTRU-2 504 directly. Alternatively, at 511, the WTRU-2 504 may proactively send a report to the SC-1 regarding what kinds of the one or more AR resources were generated, and the quality summary information (then, 520 and 521 may not be needed in this case). Then, if the SC-1 determines that there are significant discrepancies between the quality summary information reported by the WTRU-2 504 and the WTRU-1 502, 522 may be executed.
[0213] At 521 , the WTRU-2 504 decides whether the WTRU-2 504 agrees with the result.
[0214] At 522, if not agreed in 521, an additional arbitration process may be needed. For example, it is possible that the WTRU-2 504 does not agree with the WTRU-1 502 about the freshness of the image. This is particularly true if the physical objects in the images keep changing. Therefore, it is hard for both the ARRP and the ARRC to prove by themselves who is correct. In this case, one approach is to contact the other ARDs for their opinions and / or evaluations. For example, if a third ARD-3 also captures an image (at a similar or same location and at a similar or same time), then the ARD-3 may function as a quality validator and decide whether the ARD-3 agrees with the WTRU-2 504 (e.g. if the image is highly similar to what has been captured by the WTRU-2 504) or not.
[0215] At 523, if the WTRU-2504 agrees with the evaluation result of the WTRU-1 502 (in 521), the WTRU- 2 504 may transmit an acknowledgment to the SC-1 via the BCN-1 506.
[0216] At 524, when the BCN-1 506 receives the acknowledgement from both the WTRU-1 502 (at 518) and the WTRU-2 504 (at 523) or when the WTRU-1 502 and the WTRU-504 reach consensus on the quality evaluation result, the BCNs in the blockchain system (e g. the BCN-1 506) may execute the provisions in the SC-1. For example, based on the quality evaluation results, the SC-1 automatically executes the service fee payment settlement and / or clearing between the WTRU-1 502 and the WTRU-2 504. In other words, for each piece of the AR resources, the SC-1 not only calculates how much service fee the WTRU-1 502 shall pay to the WTRU-2504 but also calculates how much potential penalty the WTRU-2504 incurred. Ultimately, the SC- 1 may decide how much in total the WTRU-1 502 shall pay to the WTRU-2 504 and the fund transfer may be accordingly conducted automatically between the blockchain accounts of the WTRU-1 502 and the WTRU-2 504.
[0217] At 525, the SC-1 may transmit an acknowledgment to the WTRU-1 502 that the AR resource quality management has been conducted. The SC-1 may also transmit a notification to WTRU-2504 for its awareness.
[0218] In one embodiment, the procedure in FIGS. 5A-5B may have one or more of the following alternatives In 520 of FIGS. 5A-5B, the blockchain system may notify the ARRP regarding the quality evaluation result reported by the ARRC. Alternatively, the ARRP may proactively report its own quality evaluation result to the SC-1 in the blockchain system, before the ARRP delivers the AR resources to the ARRC. The ARRC may obtain the AR resources from ARRP and also report its quality evaluation result. The SC-1 may decide whether the two quality evaluation results reported by the ARRP and the ARRC can reach a consensus. If so, the SC-1 may trigger the settlement of the payment. If not, the arbitration process may be needed. There may be the case that one party (the ARRP or the ARRC) does not agree on the evaluatedquality evaluation result reported by the other party. In such a case, a certain reputation mechanism can be further leveraged. For example, the ARRC may only choose the ARRP with a high reputation, and vice versa.
[0219] In one embodiment, the AR resource quality management may not be realized through a smart contract between the ARRP and the ARRC as in the smart contract-based AR resource quality management discussed herein. Accordingly, the AR resource sharing collaboration relationship establishment may be skipped. However, new technical challenges need to be solved. For example, in the smart contract-based AR resource quality management, the ARRP has the obligation to serve the ARRC due to the agreed smart contract. However, when the smart contract is not used, the ARRP may not have the obligation for providing specific quality of the AR resources. Similarly, in the smart contract-based AR resource quality management, it is a responsibility of the ARRC for evaluating the AR resource quality (i e. the AR resource quality evaluation is conducted at the resource receiver side) This means that the AR resources (no matter having desired quality or not) have to be first delivered from the ARRP to the ARRC. If ultimately the ARRC decides that the quality of AR resources is not desired, the communication effort used for the AR resource delivery may be wasted (e.g. especially when the AR resources are video frames, videos, and / or images etc. having large sizes). In the meantime, the ARRC may not be able to judge the freshness of the Type-1 AR resources and / or the freshness and the accuracy of the Type-2 AR resources before the ARRC arrives at the physical environment X. Therefore, there may be a delay before the ARRC evaluates the freshness and / or accuracy of AR resources.
[0220] In an example, a home buyer (as the ARRC) may not have the capability to inspect the house sold by the existing homeowner (as the ARRP). Instead, the home buyer may rely on 3rd party licensed home inspector (as a quality validator (QV)) for inspecting the quality of the house. Accordingly, a third logical role may be defined for the ARD, as the QV. In other words, a given ARD may be the ARRP and / or the ARRC and / or the QV. In an embodiment, a reputation system may be implemented using the blockchain system so that the behaviors of the ARDs (functioning as the ARRCs, the ARRPs, and / or the QVs) may have an impact on corresponding reputation scores and / or reputation points, which may be dynamically updated. It may be assumed that such a reputation is important for the ARDs since the reputation may decide whether the ARD may serve other ARDs for rewards and / or obtain help from other ARDs in the AR sharing community, which may be regarded as a collaborative AR resource sharing ecosystem. In an embodiment, the ARRC may want to choose a high-reputation ARRP for certain AR resources. Similarly, the ARRP may refuse to serve the ARRC with a low reputation. In other words, since a given ARD may perform one or more roles (as the ARRP, and / or as the ARRC, and / or as the QV) in different AR resource sharing events, the physical ARD may have three reputations for the three roles, respectively.
[0221] In an example, when the ARRP generates the AR resources, the ARRP may also generate the corresponding ARRM, which may be in a small size compared to the original AR resource (especially for the Type-1 AR resources). The ARRP may store the original AR resources locally but may store the ARRM in the blockchain system and / or any other shared storage medium, such as off-chain storage The quality requirements and / or needs indicated by the ARRC may also be stored in the blockchain system and / or anyother shared storage medium. For quality evaluation, the ARRC may request one or more high-reputation QVs in the physical environment X for validating the AR resource quality based on the ARRM information. Multiple QVs may submit their endorsement decisions to the blockchain system for the same piece of AR resources (e.g. having desired quality or not), and the blockchain system may form a final quality evaluation result by integrating opinions, evaluations, and / or endorsements from multiple QVs. If the quality is deemed to be desired by the ARRC, the blockchain system may inform the ARRC to retrieve the AR resource, which may be communication efficient (since the ARRC will not retrieve resources having undesired quality). In the meantime, since the QVs may be directly on-site (i.e. in the physical environment X), the freshness of the Type-1 AR resources and / or the freshness and the accuracy of the Type-2 AR resources may be evaluated without any delay. Further, the reputation system may be updated periodically based on the behavior of the ARDs (e.g., based on the roles and / or performances of the ARDs as the ARRP, the ARRC, and / or the QV, as described below).
[0222] FIGS. 6A-6B illustrate an example process 600 using a reputation-based collaborative AR resource sharing management according to one or more embodiments. The process 600 may be performed between a WTRU-1 602 (as the ARRC), a WTRU-2 604 (as the ARRP), a WTRU-3 606 (as the QV), and a BCN-1 608.
[0223] At 610, the process 600 may initiate the reputation based collaborative AR resource sharing framework. In this framework, the ARDs may participate in the collaborative AR resource sharing events based on the blockchain system, so that a specific ARD may collaboratively help other ARDs and / or receive help from the other ARDs. The ARD may function as the ARRP, the ARRC, and / or the QV. The collaborative AR resource sharing may be performed by leveraging the reputation system maintained by the blockchain system so that the higher the reputation of the ARD (e.g. WTRU-1 602) is, the easier it is for the WTRU-1 602 to receive more accreditation and / or appreciation by the other ARDs for collaborating in future AR resource sharing events. For the given ARD functioning as the ARRP, the reputation for being the ARRP may describe whether the ARD provides the AR resources with the quality desired by the ARRCs. For the given ARD functioning as the ARRC, the reputation as the ARRC may describe whether the ARD is a good customer in a sense that the ARD does not have bad records for any arrears of payment to the ARRPs For the given ARD functioning as the QV, the reputation as the QV may describe whether the ARD reports accurately the quality of the AR resources. As an example, it may be assumed the WTRU-1 602 has discovered multiple ARDs residing in the physical environment X as the potential ARRPs and could potentially provide the desired AR resources (including the Type-1 and / orType-2 AR resources) to the WTRU-1 602. The ARRP selection may be based on the reputation system. As an example, the WTRU-1 602 may want to choose the WTRU-2 604 having a high reputation as the ARRP so that the WTRU-1 602 may largely expect that the WTRU-1 602 may provide requested quality of the AR resources. Here, it may be assumed that the WTRU-1 602 has discovered the WTRU-2 604 and selected the WTRU-2 604 for collecting and / or generating the AR resources associated with the physical environment X, which may be new and / or unknown to the WTRU-1 602. Also, the WTRU-1 602 may have already conveyed the AR resource requirements and / or needs to the WTRU-2 604 (e.g. the WTRU-1 602 mayindicate all the information and / or parameters as listed in 413 to the WTRU-2604), during the ARRP discovery and / or selection process.
[0224] For quality evaluation, the WTRU-1 602 may not be involved in the evaluation process. Instead, the WTRU-1 602 may rely on one or more 3rd-party ARDs (acting as the QVs) to evaluate the AR resources generated by the WTRU-2 604 (as the ARRP). When the ARRP (e.g. the WTRU-2 604) generates the AR resources, the ARRPs may need to record the corresponding ARRMs (usually in a small size) in the blockchain system or in any other shared off-chain storage medium. In the meantime, the ARRC may submit the quality requirements to the blockchain system or to the any other shared off-chain storage medium. The QVs (e.g. the WTRU-3 606) may subscribe to one or more notifications related to new ARRMs. For example, if the ARRM is stored in the blockchain as the transactions and once such types of the transactions are available, the QVs may be notified so that the QVs may decide whether to initiate a new quality evaluation. Similarly, the WTRU- 1 602 as the ARRC and the WTRU-2 604 as the ARRP may also subscribe to notifications related to new AR resource quality evaluation results. For example, if the results are also stored in the blockchain system as the transactions and once a new quality evaluation result becomes available, the corresponding and / or involved ARRP and / or ARRC may be notified. In addition, here it is assumed that the WTRU-1 602 has already discovered the WTRU-2 604 and has sent the ARRSR to the WTRU-2 604 (similar to 413), then the WTRU-2 604 has agreed to serve the WTRU-1 602. Alternatively, the WTRU-2 604 may proactively check the AR resources needed and / or the quality requirements indicated by the WTRU-1 602, which may be stored in the blockchain system or the any other shared off-chain storage medium. The WTRU-2 604 may notify the WTRU- 1 602 that the WTRU-2604 decides to serve the WTRU-1 602.
[0225] At 611, the WTRU-2 604 is in the physical environment X and generates the AR resources needed by the WTRU-1 602.
[0226] At 612, the WTRU-2 604 generates the ARRM for each AR resource created by the WTRU-2 604. The detailed information included in the ARRM may be found in the description of the AR resource quality and the ARRM described above.
[0227] At 613, the WTRU-2 604 may transmit the ARRMs as the blockchain transactions to the BCN-1 608 for recording. The detailed information of the ARRM may be included in the blockchain transaction.
[0228] At 614, the BCN-1 608 transmits an acknowledgment to the WTRU-2 604 that the ARRMs are recorded in the blockchain system.
[0229] At 615, due to notification subscriptions, the blockchain system may conduct a QV selection process and inform one or more qualified QVs (e g. the WTRU-3 606) about the new transactions representing the ARRM. A qualified QV is defined as the QV that is currently in the physical environment and meets the QV selection requirements defined by the ARRC (e.g. the QV has a high reputation etc.). The QV selection may be performed in multiple ways
[0230] The WTRU-1 602 (as the ARRC) may indicate one or more QV selection requirements in the blockchain transactions and record the blockchain transactions including the QV selection requirements in theblockchain system For example, the WTRU-1 602 may require that only the QVs having a high-reputation (e.g. exceeding a certain reputation threshold indicated by the QV selection requirements) are eligible to participate in the quality evaluation of the AR resources to be delivered to the WTRU-1 602. Accordingly, the blockchain system may utilize the QV selection requirements described in those transactions to decide which of the QVs meet the requirements.
[0231] In an example, the WTRU-1 602 as the ARRC may indicate the one or more QV selection requirements to the desired ARRP when building the connection with the ARRP during the ARRP discovery and selection process as described in 610. Then, once the ARRP produces the AR resources, the corresponding ARRMs as well as the one or more QV selection requirements of the ARRC may be included in the blockchain transactions and recorded in the blockchain system (as in 613).
[0232] In an example, each QV may check the one or more QV selection requirements indicated by the ARRC and evaluate by itself whether the QV is a qualified QV. If so, the QV may regard itself as a qualified QV and proceed with the quality evaluation.
[0233] At 616, after receiving the notification, one or more qualified QVs (e.g. the WTRU-3 606) retrieve the ARRM from the blockchain system. In other words, the ARRC will only consider and / or adopt the quality evaluation results reported by the qualified QVs as decided in 615 and / or 617.
[0234] At 617, each qualified QV (e.g. the WTRU-3606) determines and / or identifies the one or more pieces of the AR resources for quality evaluation For example, the QV may first determine whether the QV is close and / or near enough to the location and / or position where the corresponding AR resource was generated. If so, the QV may determine whether will not cost much trouble (e.g. having to make a big move) for the QV to conduct quality evaluation. Next, the WTRU-3 606 (as the QV) may evaluate whether the WTRU-3 606 has the required capabilities for the quality evaluation. For example, if the corresponding AR resource is the Type-1 AR resource (i.e the raw video frames, videos, and / or images etc., for example), the WTRU-3606 may retrieve and / or extract the technical specifications of the video frames, videos, and / or images (such as the resolution, size, and / or focus, etc., for example) stored in the ARRM and determine whether the camera of the WTRU-3 606 is capable of capturing similar images with the same specification (In some examples, the user of the WTRU-3 606 may be notified so that the user may maintain the WTRU-3 606 at the desired location and may maintain a good camera pose for validating the resource). If the corresponding AR resource is the Type-2 AR resource (i.e. the physical world knowledge), the WTRU-3 606 may evaluate whether the WTRU-3 606 possesses the required Al and / or ML capabilities (such as the video and / or image recognition and / or analysis capabilities) so that the WTRU-3606 may extract the physical world knowledge from the video frames, videos, and / or images captured by the camera (for validating the accuracy of the AR resources). In addition, multiple QVs may communicate with each other about the corresponding AR resource quality selection choice, so that for the given piece of the AR resource, there may be sufficient number of QVs to validate the quality of the given piece of the AR resource (e g.. at least two QVs), but not too many (e.g. no more than five) to avoid repetitive evaluation. In an example, one way for the QVs to communicate with each other is to leverage theblockchain system. For example, once the QV (e.g. the WTRU-3 606) decides to validate the given piece of the AR resource, such a decision may be described as another blockchain transaction and the transaction may be submitted to the blockchain system (which may also be used for future traceability purposes).
[0235] At 618, each QV (e.g. including the WTRU-3606) may analyze the selected ARRMs to evaluate the quality of the corresponding AR resources.
[0236] For the Type-1 AR resources generated by the ARRP (which is usually in large size), the WTRU-3 606 may analyze the corresponding ARRM and the corresponding needs and / or requirements (defined by the ARRC) for quality evaluation. An example quality evaluation process is described below.
[0237] In the example quality evaluation process, a first approach is that the WTRU-3606 retrieves the AR resources from the ARRP for quality evaluation. In a second approach, the WTRU-3 606 may not need to retrieve the AR resources from the ARRP. This is especially useful when the AR resources are large in size. For example, for a piece of an image-1 generated by the ARRP, the WTRU-3 606 may first determine and / or identify the location and / or position where the image-1 is captured, the WTRU-3 606 may determine and / or identify the camera pose (including the position, direction, and / or orientation), and the technical specifications of the image, i e. the resolution, brightness, focus, and / or size etc., for example. The aforementioned information may be determined, retrieved and / or extracted based on the corresponding ARRM. Accordingly, WTRU-3 606 may move to the location and / or position where the image-1 is captured (assuming the WTRU-3 606 is close to the location) and then use the camera of the WTRU-3606 to take another image-2. In particular, the image-2 captured by WTRU-3 606 needs to be similar and / or same as the image-1 (e.g. at the same location, using the same camera pose, same resolution, same focus, and / or having the same size, etc.). In an example, the WTRU-3 606 may retrieve the quality requirements associated with the image-1 (as defined by the WTRU-1 602) from the blockchain system. Finally, the WTRU-3606 may evaluate the quality of the image-2 by comparing the image-2 to the quality requirements and infer and / or estimate whether the image-1 meets the quality requirements desired by the ARRC or not (by assuming that the image-1 and the image-2 have the same and / or similar quality) In this way, the quality of the image-1 (captured and stored by the WTRU-2 604) may be evaluated by the WTRU-3 606 without incurring any AR resource communication cost (i.e. the WTRU-3 606 does not have to retrieve the image-1 from the WTRU-2 604 for quality evaluation).
[0238] For the Type-2 AR resources generated by the ARRP, i.e. the physical world knowledge (which could be in a relatively small size), it may be affordable for the WTRU-3 606 to retrieve the AR resources from the ARRP (i.e the WTRU-2 604). Accordingly, the WTRU-3 606 may determine what the knowledge is about, and how the knowledge was extracted based on the associated ARRM. After that, the WTRU-3 606 may use the camera and the Al and / or ML capabilities (such as image analysis and / or computer vision, etc., for example) of the WTRU-3 606 to validate the accuracy of the physical world knowledge generated by the WTRU-2 604.
[0239] At 619, for the given piece of the AR resource, once the WTRU-3 606 (as well as the other QVs) completes the quality evaluation, the WTRU-3 606 may provide a corresponding endorsement. The endorsement may indicate whether the piece of the AR resource meets the desired quality requirementsindicated by the ARRC or not. Accordingly, multiple QVs may submit corresponding endorsements about the AR resources to the blockchain system.
[0240] At 620, the BCN-1 608 may transmit an acknowledgment to the WTRU-3 606 indicating that the quality endorsement has been recorded.
[0241] At 621, for each piece of the AR resource, the BCN-1 608 may collect and / or integrate endorsement decisions from multiple QVs A final quality evaluation result may be created as the blockchain transaction. For example, a voting-based approach may be adopted for forming the final quality evaluation result if different QVs have varying opinions, evaluations, and / or endorsements for the quality of the given piece of the AR resource. It may be possible that some of the AR resources may not have sufficient QVs (e.g. at least two) for quality evaluation, then the blockchain system may indicate to the corresponding ARRC that the quality of those AR resources is unclear and / or undecided, and it may be up to the ARRC to decide whether to retrieve the resources from the ARRP or not.
[0242] At 622, the BCN-1 608 (or any other shared offline storage medium) may notify the WTRU-1 602 about the AR resource quality evaluation result. Alternatively, the BCN-1 608 may notify the WTRU-1 602 about one or more endorsement decisions reported by multiple qualified QVs and the WTRU-1 602 itself may integrate all of the endorsement decisions for generating an aggregated AR resource quality evaluation result.
[0243] At 623, the WTRU-1 602 (as the ARRC) may transmit an acknowledgment to the BCN-1 608. The same process of 622 and / or 623 may also be performed between the BCN-1 608 and the ARRP (i.e. WTRU-2 604) due to notification subscription.
[0244] At 624, the WTRU-1 602 may obtain the quality evaluation result from the blockchain system and / or the other shared off-chain storage, and the result may indicate whether the given piece of the AR resource has the desired quality or not.
[0245] At 625, the WTRU-1 602 sends an AR resource retrieval request to the WTRU-2 604 for the AR resources that have desired quality.
[0246] At 626, the WTRU-2 604 may transmit and / or share the required AR resources to the WTRU-1 602.
[0247] At 627, the WTRU-1 602 may have promised (e.g. using the smart contract and / or the blockchain transaction) that if the WTRU-2604 provides the desired AR resources that meet the quality requirements, the WTRU-1 602 may pay the corresponding service fee to the WTRU-2 604. Accordingly, based on how many of the AR resources were retrieved from the ARRP (i.e. the WTRU-2604) in 626, the WTRU-1 602 may calculate an amount of service fee that needs to be paid to the WTRU-2 604. As a result, the WTRU-1 602 may submit a blockchain transaction to transmit and / or transfer the service fee payment to the WTRU-2 604
[0248] At 628, the BCN-1 608 acknowledges to the WTRU-1 602 that the transaction is recorded in the blockchain system, i.e. the service fee is paid to the blockchain account of the WTRU-2 604 In an example, 627 and / or 628 may be performed before 624.
[0249] At 629, the blockchain system (such as the BCN-1 608 ) maintains a reputation system for the ARRCs, the ARRPs and the QVs. The reputations of the ARRCs, the ARRPs, and the QVs may be updatedperiodically and / or dynamically based on the corresponding behaviors and / or performances. For example, the ARD functioning as the QV may obtain one or more reputation scores indicative of the reputation of the ARD as the QV if the ARD correctly and / or truthfully performs the AR resource quality evaluation (it may be assumed that most of the QVs are rational and not malicious). Similarly, the ARD acting as the ARRC may lose one or more reputation scores and / or reputation points for the reputation of the ARD as the ARRC if the ARD refuses to pay the promised service fee to the corresponding ARRPs even when the AR resources delivered to the ARD meet the quality requirements. In another example, the ARRP may also lose one or more scores for the reputation of the ARD as the ARRP if multiple QVs report that the AR resources generated by the ARRP do not meet the quality requirements, i.e., the AR resources generated by the ARRP have an undesired quality. In an example, for the ARD functioning as the ARRC, if the ARD refuses to pay the promised service fee more than twice during a given time period (e.g. a week), then the ARD may lose one or more reputation scores and / or reputation points for the reputation of the ARD as the ARRC.
[0250] Although the procedure illustrated in FIGS. 6A-6B is based on the scenario where the QVs are the ARDs in the physical environment X while the ARRC is a different ARD that is still not yet in the physical environment , the proposed solution may also be applicable to the scenario where the QV and the ARRC are the same ARD, i.e one ARD may act as the ARRC and the QV at the same time. In an example, one ARRP may provide the same AR resources to multiple ARRCs As such, each ARRC may evaluate the quality of the resources as the QV and share the corresponding quality evaluation results between the ARRCs Another example, when there is no ARD functioning as the QV in the system, then the ARRCs themselves may decide to also function as the QVs.
[0251] FIGS. 7A-7B illustrate an example process 700 using a 3GPP SA2 embodiment for collaboration relationship establishment between the ARRP and the ARRC, according to one or more embodiments. In an embodiment, a ProSe function in 4G and / or the 5G direct discovery name management function (DDNMF) may be used to enable cross-user AR collaboration relationships between the ARRP and the ARRC. The ARDs may be embodied as the UEs and / or the WTRUs. There may be two models for a D2D direct discovery defined by 3GPP including the model shown in the FIGS. 7A-7B (which may follow Model B direct discovery as in Figure 6.3.1.3-1 in 3GPP TS 23 304 V18.0.0). The process 700 may be performed between a WTRU-1 702 (functioning as the ARRC), a WTRU-2 704 (functioning as the ARRP), a ProSe function i.e. a 5G DDNMF 706, and a blockchain node i.e. a BCN-1 708.
[0252] At 710, the WTRU-1 702 and / or the WTRU-2 704 complete the service authorization for using the ProSe function. For example, an existing ProSe service authorization and provisioning procedure may also be used by the WTRU-1 702 and / or the WTRU-2 704.
[0253] At 711, the WTRU-2 704 transmits a 3GPP D2D discovery request to the 5G DDNMF 706 for monitoring on PC5 and functioning as the ARRP. In an example, based on the Model B direct discovery defined in 3GPP, the WTRU-2 704 may be in a role of a discoverer WTRU.
[0254] At 712, the 5G DDNMF 706 may check related policies and may authorize the WTRU-2 704 to function as the ARRP. In an example, the 5G DDNMF 706 may further check with an application server and / or policy control function (PCF) for one or more related policies.
[0255] At 713, the 5G DDNMF 706 may allocate an AR resource solicitation filter and / or an AR resource provisioning response code to the WTRU-2 704.
[0256] At 714, the 5G DDNMF 706 may transmit the created AR resource solicitation filter and / or the AR resource provisioning response code to the WTRU-2 704.
[0257] At 715, the WTRU-2704 may monitor on PC5 for the AR resource solicitation code (to be announced by the ARRC).
[0258] At 716, the WTRU-1 702 may transmit a discovery request to the 5G DDNMF 706 for announcing the AR resource solicitation request on the PC5 and functioning as the ARRC.
[0259] At 717, the 5G DDNMF 706 may check related policies and authorize the WTRU-1 702 to function as the ARRC. In an example, the 5G DDNMF 706 may further check with the application server and / or the PCF for the related policies.
[0260] At 718, the 5G DDNMF 706 may allocate the AR resource provisioning response filter and the AR resource solicitation code to the WTRU-1 702.
[0261] At 719, the 5G DDNMF 706 may transmit the created AR resource provisioning response filter and the AR resource solicitation code to UE-1.
[0262] At 720, the WTRU-1 702 may travel through the physical environment X soon and may intend to dynamically design the customized AR scene design for the physical environment X. In an example, this may be performed as in 411 of FIGS. 4A-4B.
[0263] At 721 , to design the customized AR scene for the physical environment X, the WTRU-1 702 may need the AR resources captured from the physical environment X In an example, this may be performed as in 412 of FIGS. 4A-4B.
[0264] At 722, the WTRU-1 702 may announce the received AR resource solicitation code on the PC5, along with the ARRSR-1 that indicates the one or more quality requirements for the AR resources The ARRSR- 1 may include one or more parameters as described at least in 413 of FIGS. 4A-4B.
[0265] At 723, the WTRU-2 704 may monitor the PC5 and may receive the AR resource solicitation code received from the WTRU-1 702 that may match the AR resource solicitation filter.
[0266] At 724, the WTRU-2 704 may further evaluate the ARRSR-1 and decide to serve the ARRSR-1. In an example, this may be performed as in 414 of FIGS. 4A-4B.
[0267] At 725, the WTRU-2 704 may create the SC-1 for describing a detailed AR resource sharing agreement, which may be deployed into the blockchain system. In an example, this may be performed as in 415 and / or 416 of FIGS. 4A-4B.
[0268] At 726, the WTRU-2 704 may announce the AR resource provisioning code on the PC5, along with the proposed SC-1. In an example, this may be performed as in 417 of FIGS. 4A-4B.
[0269] At 727, the WTRU-1 702 may also monitor the PC5 and the WTRU-1 702 may find the AR resource provisioning code received from the WTRU-2704 that may match the AR resource provisioning response filter.
[0270] At 728, the WTRU-1 702 may checks the SC-1 and agree with the contents of the SC-1. In an example, this may be performed as in 418 and / or 416 of FIGS. 4A-4B.
[0271] At 729, the WTRU-1 702 may transmit a match report to the 5G DDNMF 706, along with the SC-1. Here, it may be assumed that the SC-1 may be submitted to the blockchain system by the 5G DDNMF 706. In an example, if there is a blockchain NF in the 3GPP core network (CN), the 5G DDNMF 706 may transmit the SC-1 to the blockchain NF so that the blockchain NF may write the SC-1 into the blockchain system. Alternatively, the WTRU-1 702 and / or the WTRU-2 704 may also directly submit the SC-1 to the blockchain system.
[0272] At 730, the 5G DDNMF 706 may deploy the SC-1 onto the blockchain system via the BCN-1 708 if the 5G DDNMF 706 has the capability to directly interact with the blockchain system In an example, the blockchain NF may be defined in the CN so that the 5G DDNMF 706 (or any other NFs) may interact with the blockchain system via the blockchain NF.
[0273] At 731 , in case the 5G DDNMF has the capability to directly interact with the blockchain system, the BCN-1 708 may transmit the acknowledgment to the 5G DDNMF 706 that the SC-1 has already been deployed on the blockchain. In that case, the blockchain NF may be defined in the CN, and the blockchain NF may notify the 5G DDNMF 706 that the SC-1 has already been deployed on the blockchain.
[0274] At 732, the 5G DDNMF 706 may transmit the acknowledgment to the WTRU-1 702 for the match report about the WTRU-1 702 and the WTRU-2 704.
[0275] At 733, the WTRU-1 702 and the WTRU-2 704 may establish a link using a ProSe direct communication for an AR resource delivery.
[0276] In the process 700, after the ARRP and the APPC are discovered, the SC-1 is created between the ARRP and the ARRC. In particular, the ARRSR-1 may be transmitted by the ARRC to the ARRP so that the ARRP may decide whether to serve the request. In an example, the 5G DDNMF 706 may be responsible for matching the needs of the ARRC with a desired ARRP and creating the SC for the ARRP and the ARRC. In an example, in 701, the WTRU-2704 may also indicate that the WTRU-2 704 may rely on the 5G DDNMF 706 for matching the ARRP and / or the ARRC and creating the smart contracts for the ARRP and / or the ARRC. At 714, the 5G DDNMF 706 may transmit the acknowledgement to the WTRU-1 702 by approving the WTRU-2 704 as the ARRP. In 716, the WTRU-1 702 may also include the ARRSR-1 when registering the WTRU-1 702 to the 5G DDNMF 706 and may indicate that the WTRU-1 702 may rely on the 5G DDNMF 706 for matching the ARRP and / or the ARRC and creating the smart contracts. Accordingly, the 5G DDNMF 706 may analyze the ARRSR-1 and determine that the WTRU-1 702 may be served by the WTRU-2 704. The 5G DDNMF 706 may create the smart contract (e.g the SC-1) for the WTRU-1 702 and / or the WTRU-2 704. The 5G DDNMF 706 may also allocate the AR resource provisioning response filter and the AR resource solicitation code to the WTRU-1 702 and allocate the AR resource solicitation filter and the AR resource provisioning response codeto the WTRU-2 704. The codes and / or filters may be recorded in the SC-1 and the 5G DDNMF 706 may deploy the SC-1 into the blockchain system The 5GDDNMF 706 may transmit the notification to the WTRU-1 702 and / or the WTRU-2704 about the blockchain addresses of the SC-1. Accordingly, the WTRU-1 702 and / or the WTRU-2 704 may access the SC-1 and obtain the respective codes and / or filters. Finally, the WTRU-1 702 and / or the WTRU-2 704 may use the codes and / or filters on the PC-5 to discover each other using the 3GPP D2D direct discovery
[0277] In an embodiment, in the 3GPP SA4, an AR WTRU may have several modules, such as but not limited to an AR runtime, a scene manager, an AR application, and / or a media access function etc. For example, the AR application may include a general manager that determines the business logic related to providing the AR experience to the user of the AR WTRU. The AR runtime may connect to one or more onboard sensors and / or cameras, as well as a user input interface and / or a display. The AR scene manager may retrieve one or more desired VOs. In an example, the VO may usually be represented using a scene description file in a standard gITF 2.0 (Graphics Library Transmission Format) format, which is a data structure that describes what the VO looks like, such as but not limited to an appearance, a shape, and / or a texture etc., and the corresponding media contents (e.g. one or more texture images, appearance images, and / or animations, etc.) The AR WTRU may retrieve the VOs from the VO provider and the spatial information from a spatial compute server. The spatial information and / or description may describe the physical world (e.g. the physical map), one or more semantic perceptions (e.g. what objects are in the physical world) and one or more spatial anchors and / or trackables (indicating where the VOs may be superimposed). The AR WTRU may retrieve the information (i.e. the retrieved VOs and / or the spatial information) using a standard media access function defined in 3GPP. One or more parts of the information may be integrated by the scene manager so that the AR scene may be rendered The rendered AR scene may be transmitted to the AR runtime for displaying the user
[0278] In an embodiment, certain extensions to the AR WTRU as defined in 3GPP SA4 are proposed. In an example, the AR application in the AR WTRU (as the ARRP) may use the AR runtime to collect raw information from the physical world of the physical environment X using the onboard sensors and / or cameras. The spatial compute server may also be available on the AR WTRU (as the ARRP) so that if needed (e.g. for the Type-2 resource sharing), the spatial compute server may process the raw information and extract the useful physical world knowledge The AR resources may be delivered from the ARRP to the ARRC using the standard media access function. On the ARRC side, the AR application in the AR WTRU (as the ARRC) may perform the customized AR scene design, i.e. may dynamically determine, retrieve and / or select the VOs that are needed (including downloading the VOs if not locally available) and determine how the VOs may be superimposed onto the POs and / or the PTOs in the physical environment X (e.g. deciding the spatial anchors and / or trackables) based on the AR resources shared by the ARRP. The customized AR scene (i.e. including the VOs and the decided spatial anchors and / or trackables) may be delivered to the AR scene manager of the ARRC for rendering in order to bring the customized and / or dynamic AR experience to the user of the ARRC.
[0279] A 3GPP SA4 embodiment of the procedure of AR resource sharing execution with quality management enforcement (as shown in FIGS. 5A-5B) is illustrated in FIGS. 8A-8D.
[0280] FIGS. 8A-8D illustrate an example process 800 of the 3GPP SA4 embodiment of the AR resource sharing execution with quality management enforcement in accordance with one or more embodiments. The process 800 may be performed by a WTRU-1 802 (as the ARRC) and a WTRU-2 804 (as the ARRP) and a blockchain node such as a BCN-1 806 The WTRU-1 802 may include an AR application, an AR runtime, a scene manager, and a media access function. The WTRU-2 804 may include another AR application, another AR runtime, another media access function, and a spatial compute server.
[0281] At 810, the WTRU-1 802 (as the ARRC) has established the AR resource sharing relationship with the WTRU-2 804 (as the ARRP), including the smart contract (i e. the SC-1) signed between the WTRU-1 802 and the WTRU-2 804) recorded in the blockchain system, for example using the procedures described in the AR resource sharing collaboration establishment using smart contract for AR resource quality management. The SC-1 signed between the WTRU-1 802 and the WTRU-2 804 may be recorded in the blockchain system.
[0282] At 811 , the WTRU-2 804 may travel in the physical environment X The AR runtime may collect the raw information about the physical environment X using the on-board sensors and / or cameras.
[0283] At 812, the AR runtime of the WTRU-2 804 may deliver the raw information to the spatial computing server.
[0284] At 813, the spatial compute server of the WTRU-2 804 generates the AR resources required by the WTRU-1 802. In an example, if the Type-1 AR resources are to be shared, the WTRU-2 804 may provide the original AR resources to the WTRU-1 802. If the Type-2 AR resources are needed, the spatial compute server of the WTRU-2 804 may extract the useful physical world knowledge for the WTRU-1 802. In an example, 812 and 813 may be performed as in 511 in FIGS. 5A-5B.
[0285] At 814, the spatial compute server of the WTRU-2 804 transmits the generated AR resources to the media access function of the WTRU-2 804, which further delivers the AR resources to the media access function of the WTRU-1 802.
[0286] At 815, the media access function of the WTRU-1 802 may transmit an acknowledgement to the media access function of the WTRU-2 804 upon receiving the AR resources.
[0287] At 816, the media access function of the WTRU-1 802 may forward the received AR resources to the AR application of the WTRU-1 802. In an example, 814-816 may be performed as in 512-513 in FIGS 5A- 5B.
[0288] At 817, the AR application of the WTRU-1 802 evaluates the technical specifications of the AR resources. In an example, this may be performed as in 514 of FIGS 5A-5B.
[0289] At 818, if the technical specification satisfies the requirements described in the SC-1, the AR application of the WTRU-1 802 may use the AR resource for the AR scene design.
[0290] At 819, the designed AR scene (i.e. the needed VOs and / or the decided spatial anchors and / or trackables) may be delivered to the AR scene manager.
[0291] At 820, the AR scene manager of the WTRU-1 802 may transmit the acknowledgement to the AR application of the WTRU-1 802 for receiving the AR scene. In an example, 818-820 may be performed as in 515 in FIGS. 5A-5B.
[0292] At 821, at a later time, the WTRU-1 802 may enter and / or arrive at the physical environment X, and the AR engine may use the on-board sensors and / or cameras to sense the physical world and then the AR application of the WTRU-1 802 may validate the freshness of the Type-1 AR resources, and / or the accuracy of the Type-2 AR resources shared by the WTRU-2 804.
[0293] At 822, in an example, the AR scene may be calibrated if the AR resources shared by the WTRU-2 804 are not accurate and / or are not fresh. For example, based on a comparison between what has just been sensed by the onboard sensors and / or cameras of the WTRU-1 802 and the AR resources previously used for scene design in 818, the AR application of the WTRU-1 802 may determine that the AR scene delivered to the AR scene manager may need one or more modifications and / or calibrations. If so, the AR application of the WTRU-1 802 may notify the AR scene manager of the WTRU-1 802 to perform the one or more modifications and / or adjustments and / or calibrations on the AR scene. Then, the scene manager may render the final AR scene to generate the AR effect for the user of the WTRU-1 802.
[0294] At 823, the final AR scene may be rendered by the scene manager of the WTRU-1 802
[0295] At 824, the rendered AR scene may be delivered to the AR runtime for displaying to the user of theWTRU-1 802. In an example, 821-824 may be performed as in 516 in FIGS. 5A-5B.
[0296] At 825, based on the overall quality evaluation of the technical specification and / or the resource freshness, the WTRU-1 802 may create the AR resource quality evaluation result. In an example, this may be performed as in 517 in FIGS. 5A-5B.
[0297] At 826, the WTRU-1 802 may report the AR resource quality evaluation result to the SC-1 stored in the blockchain system via the BCN-1 806. In an embodiment, this may be performed as in 518 in FIGS. 5A- 5B.
[0298] At 827, the SC-1 in the blockchain system may determine that the quality evaluation result may need to also be confirmed by the WTRU-2 804. The SC-1 may generate the notification, which may be delivered to the WTRU-2 804 via the BCN-1 806. In an example, this may be performed as in 519 in FIGS. 5A-5B.
[0299] At 828, the BCN-1 506 may transmit the notification to the WTRU-2 804 about the quality evaluation result reported by the WTRU-1 802. In an example, this may be performed as in 520 in FIGS. 5A-5B
[0300] At 829, the AR application of the WTRU-2 804 may be responsible for reviewing the AR quality evaluation result as reported by the ARRC, and the WTRU-2804 may determine whether to agree with the AR quality evaluation result or not In an example, this may be performed as in 521 in FIGS. 5A-5B.
[0301] In 830, if the AR quality evaluation result is not agreed by the WTRU-2 804, an additional arbitration process may be needed between the WTRU-1 802 and the WTRU-2804. In an example, this may be performed as in 522 in FIGS. 5A-5B.
[0302] In 831 , the AR application of the WTRU-2 804 may transmit the acknowledgement for agreeing with the AR quality evaluation result. In an example, this may be performed as in 523 in FIGS. 5A-5B.
[0303] At 832, based on the AR quality evaluation result, the SC-1 may automatically execute the service fee payment settlement and clearing between the WTRU-1 802 and the WTRU-2 804. In an example, this may be performed as in 524 in FIGS. 5A-5B.
[0304] At 833, the BCN-1 806 may transmit the acknowledgment to the WTRU-1 802 associated with the AR quality evaluation result reporting. In an example, this may be performed as in 525 in FIGS. 5A-5B.
[0305] This embodiment may also be aligned with an existing 3GPP SA6 architecture, which adopts a client-server model. In that, for a given WTRU (i.e. the WTRU-1 and / or the WTRU-2), the WTRU could be either in the role of the ARRC or in the role of the ARRP, or both at the same time. The WTRU functioning as the ARRC or as the ARRP may have a client function, which may be defined as an AR resource sharing client (ARRSC). Further, an edge entity (e.g. Edge Entity-1, which could be a WTRU, a gNB base station, a gateway, an edge application server, a NF, an AF, etc.) may host an AR resource sharing server (ARRSS) In an example, in this embodiment, the ARRSC on the ARRC or on the ARRP may register to the ARRSS. Accordingly, the ARRSS may provide the ARRC and / or the ARRP discovery function so that the ARRC could discover potential ARRP candidates in the desired area and / or location. Similarly, the blockchain-related function may also be realized using a client-server model. In an example, the Edge Entity-1 not only hosts the ARRSS but also hosts a blockchain function client (BFC). Another edge entity (e.g. Edge Entity-2, which could also be a WTRU, a gNB or a gateway, an edge application server, a NF, and / or an AF, etc.) could host a blockchain function server (BFS). In this embodiment, it may be assumed that the Edge Entity-1 provides the blockchain interaction capabilities to the ARRC and / or the ARRP such that the ARRSCs on the ARRC and / or the ARRP may interact with the blockchain system via the Edge Entity-1, which may use the corresponding BFC to interact with the BFS hosted on the Edge Entity-2. The BFS may be a main function to manage and / or interoperate one or more BCNs underlying blockchain system. In an example, it may also be possible that the ARRC or the ARRP may host the BFC by itself. If that is the case, the ARRC and / or the ARRP may interact with the BFS directly.
[0306] The 3GPP SA6 embodiment of the AR resource sharing collaboration establishment is illustrated in FIGS. 9A-9B (compared to FIGS 4A-4B illustrating the AR resource sharing collaboration establishment using smart contract for the AR resource quality management).
[0307] FIGS. 9A-9B illustrate an example process 900 of the 3GPP SA6 embodiment of the AR resource sharing collaboration establishment using the smart contract for quality management in accordance with one or more embodiments. The process 900 may be performed between a WTRU-1 902 (as the ARRC), a WTRU- 2 904 (as the ARRP), an Edge Entity-1 906 and an Edge Entity-2 908. The WTRU-1 902 may include an ARRSC and the WTRU-2 904 may include another ARRSC The Edge Entity-1 906 may include an ARRSS and a BFC. The Edge Entity-2908 may include another BFC.
[0308] At 910, the WTRU-1 902 and the WTRU-2 904 may host the ARRSCs and both the WTRU-1 902 and the WTRU-2 904 may have already discovered the Edge Entity-1 906, which hosts the ARRSS. In an example, this may be implemented using a well-known and / or preconfigured contact address provisioned to the ARRC and / or the ARRP, or the DNS discovery, or the WTRU-1 902 and the WTRU-2 904 may discover the available ARRSS via an EAS Discovery, etc.
[0309] At 911 , the WTRU-2 904 is in the physical environment X and can act as the ARRP to provide the AR resources to one or more WTRUs (as the ARRCs).
[0310] At 912, the ARRSC of the WTRU-2 904 may transmit the registration request-1 to the ARRSS on the Edge Entity-1 906. In this request, the WTRU-2904 may indicate that the WTRU-2 904 intends to register as the ARRP. The WTRU-2904 may also indicate a current location and / or planned travel path. In an example, the WTRU-2 904 may also indicate the AR-related capabilities of the WTRU-2 904. For example, the WTRU-2 904 may indicate what types of the AR resources can be provided by the WTRU-2 904.
[0311] At 913, the ARRSS on the Edge Entity-1 906 registers the WTRU-2 904 by creating a registration record, the current location, the planned travel and / or mobility path, and / or the AR capabilities. The WTRU-2 904 may update the location and / or mobility information periodically and / or dynamically and / or if needed at a later time.
[0312] At 914, the ARRSS on Edge Entity-1 906 may transmit the acknowledgement to the ARRSC on theWTRU-2 904 indicating that the ARRP registration is completed.
[0313] At 915, the WTRU-1 902 may travel through the new and / or unknown physical environment X soon and the WTRU-1 902 may need to design the customized AR scene for the physical environment X.
[0314] At 916, the WTRU-1 902 may request some AR resources captured from the physical environment X. Therefore, the WTRU-1 902 may discover some desired ARRPs in the physical environment X.
[0315] At 917, the ARRSC of the WTRU-1 902 may transmit a registration request to the ARRSS on the Edge Entity-1 906. For example, the WTRU-1 902 may indicate that the WTRU-1 902 intends to be the ARRC. If the ARRSC of the WTRU-1 902 has already registered to the ARRSS at an earlier time, this process may be used for ARRP discovery In an example, the WTRU-1 902 may indicate one or more ARRP selection criteria based on one or more needs, requirements, and / or location, etc. In an example, the WTRU-1 902 may indicate that the WTRU-1 902 needs to find some candidate ARRPs in the physical environment X that could provide the Type-2 AR resources.
[0316] At 918, the ARRSS on the Edge Entity-1 906 registers the WTRU-1 902 as the ARRC. The ARRSS may also check the ARRP registration records and decide that one or more candidate ARRPs (e.g. the WTRU- 2 904) may serve the AR resource needs of the WTRU-1 902.
[0317] At 919, the ARRSS on the Edge Entity-1 906 may return one or more candidate ARRPs (such as the WTRU-2904) to the ARRSC on the WTRU-1 902, including information regarding how to contact those the ARRPs.
[0318] At 920, the WTRU-1 902 may contact the WTRU-2 904. Accordingly, the ARRSC on the WTRU-1 902 may transmit the ARRSR-1 to the ARRSC of the WTRU-1 902. The parameters included in the ARRSR-1 may be the same as the parameters included in 413 in FIGS. 4A-4B. In an example, 921-924 may be performed as in 414-417 in FIGS. 4A-4B.
[0319] At 925, the WTRU-1 902 and the WTRU-2 904 may not host the BFC. Therefore, the WTRU-1 902 and the WTRU-2 904 may rely on the Edge Entity-1 906 to interact with the blockchain system. In this step, the ARRSC of the WTRU-1 902 may transmit the SC-1 to the ARRSS of the Edge Entity-1 906 for the smart contract deployment. In an example, the ARRSC of the WTRU-2 904 may transmit the SC-1 to the ARRSS of the Edge Entity-1 906 for the smart contract deployment. In addition, in case the ARRC and / or ARRP also hosts the BFC, then the ARRC and / or the ARRP could directly interact with the BFS on the Edge Entity-2908.
[0320] At 926, the Edge Enitty-1 906 not only hosts the ARRSS but also hosts the BFC. Accordingly, the ARRSS forwards the SC-1 to the BFC on the Edge Entity-1 906 In an example, the Edge Entity-1 906 may directly host the BFS In this case, the Edge Entity-1 906 may directly interact with the BCNs in the underlying blockchain system. Then, the ARRSS may directly forward the SC-1 to the BFS hosted on the Edge Entity-1 906 for deploying the SC-1 into the underlying blockchain system
[0321] At 927, the BFC on the Edge Entity-1 906 may transmit the SC-1 to the BFS on the Edge Entity-2 908 and the BFS may interact with the one or more BCNs in the underlying blockchain system.
[0322] At 928, the BFS on the Edge Entity-2 908 may interact with the underlying blockchain system to record the SC-1 in the blockchain system.
[0323] At 929, the BFS on the Edge Entity-2 908 may transmit an acknowledgement to the BFC on the Edge Entity-1 906 indicating that the SC-1 was already recorded in the blockchain system.
[0324] At 930, the BFC on the Edge Entity-1 906 forwards the acknowledgement to the ARRSS on the Edge Entity-1 906.
[0325] At 931, the ARRSS on the Edge Entity-1 906 may returns the acknowledgement to the ARRSC on the WTRU-1 902 indicating that the SC-1 was already recorded in the blockchain system.
[0326] At 932, the ARRSC of the WTRU-1 902 (or the ARRSS of the Edge Entity-1 906) may inform the ARRSC of the WTRU-2 904 about the address of the SC-1 in the blockchain system. The WTRU-1 902 generates a corresponding job for serving the ARRSR-1.
[0327] In an embodiment, an ETSI Industry Specification Group (ISG) on permissioned distributed ledger (PDL) specifies standards related to a PDL architecture, one or more PDL services, and a PDL for vertical domains such as telecom. ETSI GR PDL-0024 proposes one or more functions, which may be used in the 3GPP networks for provisioning a PDL service. A ledger anchor function (LAF) may handle a control plane related processing For example, the LAF may receive a request from a 3GPP entity (such as a WTRU or an NF) for the PDL service provisioning. The LAF may also monitor the status of one or more deployed PDL services and conduct management and / or maintenance of the one or more PDL services when needed. Another function BC enabler function (BCEF), may receive and / or execute the commands sent from the LAF.The BCEF may be a full blockchain node, a light blockchain node and / or a simple client implementing a real interaction interface to aspecific underlying distributed ledger orthe blockchain system. For example, the BCEF may function as a blockchain and / or a ledger client so that the BCEF may be a part of an underlying ledger system. The BCEF may handle a transaction composition and submit the transaction to the underlying PDL system. The node functioning as the BCEF may also be a full operating node in the underlying PDL system and may run consensus protocols required by the underlying PDL system.
[0328] In an example, an ETSI PDL implementation of the AR resource sharing collaboration establishment is illustrated in FIGS 10A-10B (compared to FIGS. 4A-4B illustrating the AR resource sharing collaboration establishment using smart contract for AR resource quality management).
[0329] FIGS. 10A-1 OB illustrates an example process 1000 of an ETSI PDL embodimentofthe AR resource sharing collaboration establishment using the smart contract for quality management according to one or more embodiments. The process 1000 may be performed between a WTRU-1 1002 (as the ARRC), a WTRU-2 1004 (as the ARRP), an LAF 1006, and one or more PDL networks.
[0330] In an example, 1011-1017 in FIGS. 10A-10B may be the same and / or similar to 411-417 in FIGS. 4A-4B.
[0331] At 1018, the WTRU-1 1002 generates and / or transmits a PDL service request to the LAF 1006. In this request, the WTRU-1 1002 may indicate what kinds of the PDL service and / or underlying blockchain system is needed. The WTRU-1 1002 may generate, provide, and / or indicate a plurality of parameters including but not limited to one or more desired PDL service types, one or more desired characteristics of the underlying PDL system, one or more desired PDL network deployments, one or more desired PDL node entities, a desired scale of the underlying PDL network, a need for creating a new underlying PDL system, one or more desired performance requirements, one or more autoscaling indicators for the PDL service performance, one or more autoscaling schedules, one or more autoscaling rules, and / or one or more autoscaling indicators for one or more PDL service capabilities etc., for example.
[0332] The desired PDL service type may indicate a desired type of the underlying PDL system and / or a name of a PDL system. Other public, private and / or consortium distributed ledgers may also be used.
[0333] The desired characteristics of the underlying PDL system may indicate a PDL system operating a desired type of consensus protocol preferred by the WTRU-1 1002.
[0334] The desired PDL network deployment may indicate that the needed underlying PDL system may be deployed on public Internet, and / or within a data center, and / or within a communication infrastructure of a mobile operator etc., for example.
[0335] The desired PDL node entities may indicate that the needed underlying PDL system may be deployed on one or more types of nodes. For example, if the needed PDL system is to be deployed within the communication infrastructure of the mobile operator, the PDL nodes may be deployed on mobile WTRUs, base stations, edge nodes, and / or in the CN.
[0336] The desired scale of the underlying PDL network may indicate a number of nodes that may be required for the desired underlying PDL network. For example, the WTRU-1 1002 may indicate that the WTRU- 1 1002 needs the PDL service that may be provisioned by the underlying PDL network having at least 25 nodes.
[0337] The need for creating the new underlying PDL system may indicate if there is no available underlying PDL system that can provision the desired PDL service for the WTRU-1 1002, whether a new underlying PDL network and / or system may be created in an on-demand manner. For example, this may also happen when the WTRU-1 1002 may have certain special needs such that a customized PDL service provisioning may be needed by creating the new PDL system For example, the LAF may initiate a new PDL node on a certain 3GPP entity such as a WTRU and / or a base station. The 3GPP entity may not have the required software for functioning as a PDL node with e.g. a specific consensus protocol. Then, the LAF may retrieve the desired consensus protocol software from a blockchain repository function (BCR), which may provide the required software components for the realization of the PDL node for provisioning the desired PDL service.
[0338] The desired performance requirements may indicate the desired performance requirements of the needed PDL system that provision the desired PDL service, such as the minimum and / or maximum transaction throughput per hour, the transaction latency incurred by the consensus protocol, etc
[0339] The autoscaling indicator for the PDL service performance may indicate whether the WTRU-1 1002 needs the PDL service with an autoscaling capability. For example, the WTRU-1 1002 may not be able to specify the desired performance requirements as listed above since the needs of the WTRU-1 1002 may be not predictable. Therefore, the WTRU-1 1002 may require the PDL service to have autoscaling capabilities, so that the PDL service may automatically scale up and / or scale down depending on e.g. how many transactions are being submitted by the WTRU-1 1002.
[0340] The autoscaling schedule may indicate when the WTRU-1 1002 requires the autoscaling capability for the desired PDL service. For example, the WTRU-1 1002 may only need autoscaling capability during a certain time period.
[0341] In the autoscaling rules, the WTRU-1 1002 may suggest a list of rules, based on which the required PDL service may make decision on when to scale up and / or scale down the performance.
[0342] In the autoscaling indicator for the PDL service capabilities, in addition to scaling up and / or scaling down the service performance, the WTRU-1 1002 may also indicate whether the service capability can be autoscaled. For example, the WTRU-1 1002 may be provisioned with a PDL service by a particular PDL Node-1 1008. Accordingly, the WTRU-1 1002 may require the PDL Node-1 1008 to have dynamic blockchain capabilities. For example, initially, the PDL Node-1 1008 may not store a full copy of blockchain data locally due to limited storage, i e. the PDL Node-1 1008 may not be a full PDL node. However, in a certain time period, based on the needs of the WTRU-1 1002, the PDL Node-1 1008 may automatically scale up the capability of the PDL Node-1 1008 (e.g. by allocating more storage space etc.) so that the PDL Node-1 1008 may host a complete copy of the blockchain data (which e.g. could reduce the blockchain retrieval delay for the WTRU-11002). In an example, initially, the PDL Node-1 1008 may not function as a mining node in the underlying PDL network. However, in a certain time period, based on the needs of the WTRU-1 1002, the PDL Node-1 1008 may automatically scale up the capabilities (e.g. by allocating a significant amount of computing resources) so that the PDL Node-1 1008 may upgrade itself e.g. from a full node to a mining node in order to participate in a PoW-based consensus process, which e.g. could increase the chances for the transactions of the WTRU-1 1002 to be recorded in the blockchain system in a timely manner.
[0343] At 1019, the LAF 1006 discovers the available underlying PDL networks and / or systems. For example, the LAF 1006 may conduct a PDL system and / or network discovery for identifying the available underlying PDL systems. In an example, the underlying PDL systems may register themselves to the LAF 1006 so that the LAF 1006 may create a profile for each registered PDL network and / or system and / or create a PDL system repository so that the LAF 1006 may discover the needed PDL system from the PDL system repository. In an example, the LAF 1006 may conduct the BCEF discovery since each BCEF is a direct interface and / or contact-of-address of one or more underlying PDL systems.
[0344] At 1020, based on the discovery, the LAF 1006 selects an appropriate PDL network (e.g. PDL Network-A) for serving the request of the WTRU-1 1002 since the PDL Network-A may provision the desired PDL service for the WTRU-1 1002.
[0345] At 1021, if there is already the BCEF deployed to serve as the direct interface and / or contact-of- address of the PDL Network-A, then 1021-1024 may be skipped. In an example, the LAF 1006 may decide to instantiate a new BCEF (e.g. BCEF-1) so that the BCEF-1 may interoperate with the underlying PDL Network- A. For example, the PDL Node-1 1008 is one of the nodes in the PDL Network-A and accordingly the LAF 1006 may decide to instantiate the BCEF-1 on the PDL Node-1 1008. Even if there is already the BCEF deployed as the direct interface and / or contact-of-address of the PDL Network-A, the LAF 1006 may still perform 1022- 1024 to adjust the existing BCEF to make the BCEF meet the requirements of the WTRU-1 1002 as indicated in 1018. For example, if the WTRU-1 1002 has indicated that a certain autoscaling capability is needed, then the BCEF may need to make sure that such a feature may be realized via collaboration between the BCEF-1 and the PDL Node-1 1008. For example, if the WTRU-1 1002 requires autoscaling for the performance, then the BCEF-1 may check the PDL Node-1 1008 and configure the PDL Node-1 1008 accordingly. For example, one of the approaches for realizing performance scaling is that the BCEF-1 may ensure that the PDL Node-1 1008 has the sufficient resources (computing, memory, and / or storage) for performance autoscaling. Then, the PDL Node-1 1008 may also expose resource allocation APIs to the BCEF-1 so that the BCEF-1 may dynamically generate and / or transmit commands to the PDL Node-1 1008 to allocate more resources for scaling up the performance and / or recycling more resources for scaling down the performance. In an example, one of the approaches for realizing blockchain capability scaling may be that the BCEF-1 ensures that the PDL Node-1 1008 has sufficient resources (computing, memory, and / or storage) as well as a required software (e g. the software for running a consensus protocol so that the PDL Node-1 1008 may upgrade itself from the full PDL node to the mining node). In the meantime, the PDL Node-1 1008 may also expose related APIs to theBCEF-1 so that the BCEF-1 may dynamically transmit the commands to the PDL Node-1 1008 to dynamically scale up and / or scale down the blockchain capabilities, e.g. initially act as a light PDL node, and then switch to the full PDL node or the mining node based on needs
[0346] At 1022, the LAF 1006 transmits a BCEF deployment request to the PDL Node-1 1008 In an example, the LAF 1006 may transmit a configuration request to an existing BCEF-1 (which is interfacing with the PDL Node-1 1008) For example, if the WTRU-1 1002 needs the BCEF-1 for realizing the performance and / or capabilities autoscaling feature for the PDL Node-1 1008, then such a request may include all the autoscaling-related parameters as listed in 1018 so that the existing BCEF-1 may conduct certain configurations to the PDL Node-1 1008.
[0347] At 1023, the new BCEF-1 is instantiated and deployed on the PDL Node-1 1008. In an example, this may also be used for an existing BCEF-1 to indicate whether certain configurations are complete. For example, if the LAF-1 indicates a list of autoscaling-related parameters to an existing BCEF-1 in 1022, then the BCEF-1 may confirm whether the performance and / or blockchain capability autoscaling feature has been successfully realized and correctly configured.
[0348] At 1024, the PDL Node-1 1008 may transmit an acknowledgment to the LAF 1006 indicating that the BCEF-1 is deployed. The LAF 1006 may further conduct other control-related configurations on the BCEF- 1 if needed. For example, the LAF 1006 may configure the BCEF-1 so that the LAF 1006 may monitor the status and the real-time performance of the BCEF-1 and the underlying PDL Network-A In addition, the LAF 1006 may also create and maintain a profile for each BCEF so that the LAF 1006 may conduct profile management for the BCEFs and / or other management tasks, such as but not limited to enabling BCEF discovery, BCEF creation, instantiation, activation, deactivation, and / or de-installation, etc.
[0349] At 1025, the LAF 1006 indicates to the WTRU-1 1002 about the access address of the BCEF-1 and the corresponding instructions and / or policy or other useful information (e.g. available schedule, access right information, messaging protocols, etc.) regarding how to interact with the BCEF-1
[0350] At 1026, the BCEF-1 is instantiated on the PDL Node-1 1008.
[0351] At 1027, the WTRU-1 1002 deploys the SC-1 to PDL Network-A via BCEF-1. In an example .this may be performed as in 418 in FIGS. 4A-4B.
[0352] At 1028, the BCEF-1 records the SC-1 in the PDL Network-A. One or more deposits may be deducted from one or more PDL accounts of the WTRU-1 1002 and / or the WTRU-2 1004. In an example, this may be performed as in 419 in FIGS. 4A-4B.
[0353] At 1029, the BCEF-1 may transmit the acknowledgement to the WTRU-1 1002 indicating that the SC-1 is deployed. In an example, this may be performed as in 420 in FIGS 4A-4B.
[0354] At 1030, the WTRU-1 1002 informs the WTRU-2 1004 about the access address of the BCEF-1 and how to access the SC-1 deployed in the PDL Network-A. The WTRU-2 1004 may access the PDL Network-A via the BCEF-1 to check the existence and / or correctness of the SC-1. In an example, this may be performed as in 421 in FIGS. 4A-4B.
[0355] At 1031, the WTRU-2 1004 generates a corresponding job for serving the ARRSR-1. In an example, this may be performed as in 422 in FIGS. 4A-4B.
[0356] FIG. 11 is an example system diagram of an AR resource sharing system 1100, according to one or more embodiments. The AR resource sharing system 1100 includes a WTRU-1 1110 (functioning as the ARRC), a WTRU-2 1130 (functioning as the ARRP), and a blockchain network 1150. The WTRU-1 1110 includes one or more processors 1112, one or more sensors 1114, a network interface 1116, and a memory 1118. The memory 1118 includes a plurality of AR modules 1120. The AR modules 1120 include but are not limited to a media access module 1122, an application module 1124, a runtime module 1126, and a scene management module 1128. The WTRU-2 1130 includes one or more processors 1132, one or more sensors 1134, a network interface 1136, and a memory 1138. The memory 1138 includes a plurality of AR modules 1140. The AR modules 1140 include but are not limited to a media access module 1142, an application module 1144, a runtime module 1146, and a spatial computing module 1148. The blockchain network 1150 includes a plurality of blockchain nodes including BCN-1 1160 through BCN-N 1170. The BCN-1 1160 stores a plurality of blocks including Block-1 1162 though Block-N 1164. Each block may include a header, one or more transactions, and one or more timestamps. In an embodiment, the WTRU-1 1110 may function as the consumer device, i.e. the ARRC, and the WTRU-2 1130 may function as the provider device, i.e. the ARRP. The WTRU- 1 1110 and / or the WTRU-2 1130 may be AR WTRUs.
[0357] In the WTRU-1 1110, the application module 1124 may store and / or determine an AR logic and / or the business logic related to providing the AR experience to the user of the WTRU-1 1110 The runtime module 1126 may connect to the sensors 1114 such as cameras and / or microphones etc., for example, as well as a user input interface and / or a display. The scene management module 1128 may retrieve the desired VOs. The WTRU-1 1110 may retrieve the information (i.e. the desired VOs and / or the corresponding spatial information) using the media access module 1122. The scene management module 1128 may integrate one or more parts of received information to generate and / or render the AR scene. The rendered AR scene may be transmitted to the runtime module 1126 for displaying the user.
[0358] In the WTRU-2 1130, the application module 1144 in the WTRU-2 1130 may use the runtime module 1146 to collect raw information from the physical world of the physical environmentX using the onboard sensors 1134 such as the cameras and / or the microphones etc., for example. The spatial computing module 1148 may process the raw information and extract the useful physical world knowledge. The AR resources may be delivered from the WTRU-2 1130 to the WTRU-1 1110 using the media access module 1142.
[0359] ln the WTRU-1 1110, the application module 1124 in the WTRU-1 1110 may perform the customized AR scene design, i.e. may dynamically determine, retrieve and / or select the VOs that are needed (including downloading the VOs if not locally available) and determine how the VOs may be superimposed onto the POs and / or the PTOs in the physical environment X (e.g. deciding the spatial anchors and / or trackables) based on the AR resources shared by the ARRP. The customized AR scene (i.e. including the VOs and the decided spatial anchors and / or trackables) may be delivered to the scene management module 1128 of the WTRU-11110 for rendering in order to provide the customized and / or dynamic AR experience to the user of the WTRU- 1 1110.
[0360] Fig 12 is an example flowchart of a process 1200 performed by the provider device, according to one or more embodiments. The process 1200 may be performed by a WTRU (not shown in FIG 12) that functions as the provider device, i.e., the ARRP.
[0361] At 1210, the ARRP receives the ARRSR from the ARRC The ARRSR may include multiple quality requirements and multiple smart contract terms The quality requirements may be associated with the AR resources. The smart contract terms may be used for generating the smart contract.
[0362] At 1220, the ARRP compares the quality requirements with one or more device capabilities of the ARRP. In one example, the ARRP may determine whether focal length, resolution, shutter speed, contrast, definitions, and / or color profile etc. of the camera of the ARRP is capable of generating and / or capturing the images as per the resolution, focus, and / or size etc. indicated by the quality requirements in the ARRSR. The ARRP may not accept the ARRSR if the camera of the ARRP is not capable of producing the images that meet the quality requirements indicated by the ARRSR. In another example, the ARRP may compare the geolocation of the ARRP with a desired geolocation indicated by the ARRSR. For instance, the ARRP may accept the ARRSR only if the ARRP is in the desired geolocation and / or will visit the desired geolocation within a time indicated by the ARRSR.
[0363] At 1230, the ARRP decides whether the ARRP will accept the ARRSR or reject the ARRSR. If the capabilities of the ARRP do not meet the quality requirements indicated in the ARRSR, at 1240, the ARRP may reject the ARRSR. If the capabilities of the ARRP meet the quality requirements indicated in the ARRSR, the ARRP may accept the ARRSR.
[0364] At 1250, after accepting the ARRSR, the ARRP generates the smart contract when the ARRP decides to accept the ARRSR. The smart contract may be transmitted to the ARRC. The ARRP may negotiate the smart contract with the ARRC till the ARRP and the ARRC reach the consensus. The ARRP and the ARRC may finalize the smart contract and deploy the final smart contract on the blockchain.
[0365] At 1260, the ARRP generates and transmits the AR resources based on the final smart contract. The AR resources may be of a first AR resource type (i.e. the Type-1 AR resources such as the raw sensory data) and / or may be of a second AR resource type (i.e. the Type-2 AR resources such as the processed and / or pre-processed sensory data). The ARRP may also generate and transmit the physical world knowledge associated with the desired location. The ARRP may also generate and transmit the metadata (i.e. the ARRM) associated with the AR resources. The ARRP may further generate and transmit a timestamp indicative of a time of generation of the AR resources. The ARRP may also transmit the information indicative of technical settings of the sensors (such as the cameras) used to generate and / or capture the AR resources. The ARRP may also determine and transmit a confidence value and / or a confidence parameter indicative of an estimated accuracy of the AR resources. The ARRP may also generate and transmit a freshness dynamics parameter indicative of how frequently the POs and / or PTOs reflected in the AR resources may change and / or indicativeof how stable are the POs and / or PTOs reflected in the AR resources (e.g. the POs like the street buildings are more stable than the POs like the road intersection).
[0366] FIG. 13 an example flowchart of a process 1300 performed by the consumer device, according to one or more embodiments. The process 1300 may be performed by a WTRU (not shown in FIG 13) that functions as the consumer device, i.e , the ARRC.
[0367] At 1310, the ARRC discovers multiple ARRPs.
[0368] At 1320, the ARRC transmits the ARRSR to the discovered ARRPs.
[0369] At 1330, the ARRC receives one or more smart contracts from one or more ARRPs. The ARRC may select a smart contract from the one or more smart contract. The ARRC may determines whether the smart contract is acceptable. The ARRC may identify one or more parts of the smart contract for negotiation if the smart contract is not acceptable. The ARRC may transmit a negotiation indication to theARRP. The negotiation indication may be indicative of the one or more parts of the smart contract that are not acceptable to the ARRC and / or that are to be negotiated. The ARRC may negotiate the smart contract with the ARRP until the consensus is reached between the ARRP and the ARRC and / or until the smart contract is finalized. The ARRP and the ARRC may agree to perform as per the final smart contract. The final smart contract may establish the collaboration agreement and / or the resource sharing agreement between the ARRP and the ARRC.
[0370] At 1340, the ARRC deploys the final smart contract to the blockchain network.
[0371] At 1350, the ARRC receives the AR resources based on the deployed smart contract. The ARRC may generate the AR scene based on the received AR resources and provide the AR scene to the user of the ARRC. The ARRC evaluates the received AR resources based on the quality requirements indicated in the ARRSR. The ARRC generates one or more quality evaluation results based on the evaluation. The one or more quality evaluation results are indicative of one or more quality requirements that are met by the AR resources, and / or one or more quality requirements that are not met by the one or more AR resources. The one or more quality evaluation results may also include measured accuracy and / or measured freshness of the AR resources. The one or more quality evaluation results may be transmitted to the blockchain network and / or the ARRP. The smart contract may be executed based on the one or more quality evaluation results. That is, the ARRP may be paid and / or rewarded for the one or more quality requirements that are met by the AR resources and / or the ARRP may be fined and / or penalized for the one or more quality requirements that are not met by the one or more AR resources.
[0372] In an example, for evaluating the accuracy of the AR resources, the ARRC generates the sensory data using the sensors of the ARRC in real-time or in near-real time. The ARRC compares the generated sensory data with the AR resources received from the ARRP. Based on the comparison, the ARRC evaluates the accuracy of the AR resources. The ARRC may recalibrate the AR scene based on the determined accuracy.
[0373] In an example, for evaluating the AR resources, the ARRC may determine whether the AR resources capture the desired POs and / or PTOs indicated by the ARRSR
[0374] Although features and elements are provided above in particular combinations, one of ordinary skill in the art will appreciate that each feature or element can be used alone or in any combination with the other features and elements. The present disclosure is not to be limited in terms of the particular embodiments described in this application, which are intended as illustrations of various aspects. Many modifications and variations may be made without departing from its spirit and scope, as will be apparent to those skilled in the art. No element, act, or instruction used in the description of the present application should be construed as critical or essential to the invention unless explicitly provided as such. Functionally equivalent methods and apparatuses within the scope of the disclosure, in addition to those enumerated herein, will be apparent to those skilled in the art from the foregoing descriptions. Such modifications and variations are intended to fall within the scope of the appended claims. The present disclosure is to be limited only by the terms of the appended claims, along with the full scope of equivalents to which such claims are entitled. It is to be understood that this disclosure is not limited to particular methods or systems.
[0375] The foregoing embodiments are discussed, for simplicity, with regard to the terminology and structure of infrared capable devices, i.e., infrared emitters and receivers. However, the embodiments discussed are not limited to these systems but may be applied to other systems that use other forms of electromagnetic waves or non-electromagnetic waves such as acoustic waves.
[0376] It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting. As used herein, the term "video" or the term "imagery" may mean any of a snapshot, single image and / or multiple images displayed over a time basis. As another example, when referred to herein, the terms "user equipment" and its abbreviation "UE", the term "remote" and / or the terms "head mounted display" or its abbreviation "HMD" may mean or include (i) a wireless transmit and / or receive unit (WTRU); (ii) any of a number of embodiments of a WTRU; (iii) a wireless-capable and / or wired-capable (e.g., tetherable) device configured with, inter alia, some or all structures and functionality of a WTRU; (iii) a wireless-capable and / or wired-capable device configured with less than all structures and functionality of a WTRU; or (iv) the like. Details of an example WTRU, which may be representative of any WTRU recited herein, are provided herein with respect to FIGs. 1 A-1 D As another example, various disclosed embodiments herein supra and infra are described as utilizing a head mounted display. Those skilled in the art will recognize that a device other than the head mounted display may be utilized and some or all of the disclosure and various disclosed embodiments can be modified accordingly without undue experimentation. Examples of such other device may include a drone or other device configured to stream information for providing the adapted reality experience.
[0377] In addition, the methods provided herein may be implemented in a computer program, software, or firmware incorporated in a computer-readable medium for execution by a computer or processor. Examples of computer-readable media include electronic signals (transmitted over wired or wireless connections) and computer-readable storage media. Examples of computer-readable storage media include, but are not limited to, a read only memory (ROM), a random access memory (RAM), a register, cache memory, semiconductormemory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks, and digital versatile disks (DVDs). A processor in association with software may be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or any host computer.
[0378] Variations of the method, apparatus and system provided above are possible without departing from the scope of the invention. In view of the wide variety of embodiments that can be applied, it should be understood that the illustrated embodiments are examples only, and should not be taken as limiting the scope of the following claims. For instance, the embodiments provided herein include handheld devices, which may include or be utilized with any appropriate voltage source, such as a battery and the like, providing any appropriate voltage.
[0379] Moreover, in the embodiments provided above, processing platforms, computing systems, controllers, and other devices that include processors are noted. These devices may include at least one Central Processing Unit ("CPU") and memory. In accordance with the practices of persons skilled in the art of computer programming, reference to acts and symbolic representations of operations or instructions may be performed by the various CPUs and memories. Such acts and operations or instructions may be referred to as being "executed," "computer executed" or "CPU executed."
[0380] One of ordinary skill in the art will appreciate that the acts and symbolically represented operations or instructions include the manipulation of electrical signals by the CPU. An electrical system represents data bits that can cause a resulting transformation or reduction of the electrical signals and the maintenance of data bits at memory locations in a memory system to thereby reconfigure or otherwise alter the CPU's operation, as well as other processing of signals. The memory locations where data bits are maintained are physical locations that have particular electrical, magnetic, optical, or organic properties corresponding to or representative of the data bits. It should be understood that the embodiments are not limited to the above-mentioned platforms or CPUs and that other platforms and CPUs may support the provided methods
[0381] The data bits may also be maintained on a computer readable medium including magnetic disks, optical disks, and any other volatile (e.g., Random Access Memory (RAM)) or non-volatile (e.g., Read-Only Memory (ROM)) mass storage system readable by the CPU. The computer readable medium may include cooperating or interconnected computer readable medium, which exist exclusively on the processing system or are distributed among multiple interconnected processing systems that may be local or remote to the processing system. It should be understood that the embodiments are not limited to the above-mentioned memories and that other platforms and memories may support the provided methods.
[0382] In an illustrative embodiment, any of the operations, processes, etc. described herein may be implemented as computer-readable instructions stored on a computer-readable medium. The computer- readable instructions may be executed by a processor of a mobile unit, a network element, and / or any other computing device.
[0383] There is little distinction left between hardware and software implementations of aspects of systems. The use of hardware or software is generally (but not always, in that in certain contexts the choice between hardware and software may become significant) a design choice representing cost versus efficiency tradeoffs. There may be various vehicles by which processes and / or systems and / or other technologies described herein may be affected (e.g., hardware, software, and / or firmware), and the preferred vehicle may vary with the context in which the processes and / or systems and / or other technologies are deployed. For example, if an implementer determines that speed and accuracy are paramount, the implementer may opt for a mainly hardware and / or firmware vehicle. If flexibility is paramount, the implementer may opt for a mainly software implementation. Alternatively, the implementer may opt for some combination of hardware, software, and / or firmware
[0384] The foregoing detailed description has set forth various embodiments of the devices and / or processes via the use of block diagrams, flowcharts, and / or examples. Insofar as such block diagrams, flowcharts, and / or examples include one or more functions and / or operations, it will be understood by those within the art that each function and / or operation within such block diagrams, flowcharts, or examples may be implemented, individually and / or collectively, by a wide range of hardware, software, firmware, or virtually any combination thereof. In an embodiment, several portions of the subject matter described herein may be implemented via Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), digital signal processors (DSPs), and / or other integrated formats. However, those skilled in the art will recognize that some aspects of the embodiments disclosed herein, in whole or in part, may be equivalently implemented in integrated circuits, as one or more computer programs running on one or more computers (e.g., as one or more programs running on one or more computer systems), as one or more programs running on one or more processors (e.g., as one or more programs running on one or more microprocessors), as firmware, or as virtually any combination thereof, and that designing the circuitry and / or writing the code for the software and or firmware would be well within the skill of one of skill in the art in light of this disclosure. In addition, those skilled in the art will appreciate that the mechanisms of the subject matter described herein may be distributed as a program product in a variety of forms, and that an illustrative embodiment of the subject matter described herein applies regardless of the particular type of signal bearing medium used to actually carry out the distribution. Examples of a signal bearing medium include, but are not limited to, the following: a recordable type medium such as a floppy disk, a hard disk drive, a CD, a DVD, a digital tape, a computer memory, etc., and a transmission type medium such as a digital and / or an analog communication medium (e.g., a fiber optic cable, a waveguide, a wired communications link, a wireless communication link, etc.).
[0385] Those skilled in the art will recognize that it is common within the art to describe devices and / or processes in the fashion set forth herein, and thereafter use engineering practices to integrate such described devices and / or processes into data processing systems. That is, at least a portion of the devices and / or processes described herein may be integrated into a data processing system via a reasonable amount of experimentation Those having skill in the art will recognize that a typical data processing system may generally include one or more of a system unit housing, a video display device, a memory such as volatile and non-volatile memory, processors such as microprocessors and digital signal processors, computational entities such as operating systems, drivers, graphical user interfaces, and applications programs, one or more interaction devices, such as a touch pad or screen, and / or control systems including feedback loops and control motors (e.g., feedback for sensing position and / or velocity, control motors for moving and / or adjusting components and / or quantities). A typical data processing system may be implemented utilizing any suitable commercially available components, such as those typically found in data computing / communication and / or network computing / communication systems.
[0386] The herein described subject matter sometimes illustrates different components included within, or connected with, different other components. It is to be understood that such depicted architectures are merely examples, and that in fact many other architectures may be implemented which achieve the same functionality. In a conceptual sense, any arrangement of components to achieve the same functionality is effectively "associated" such that the desired functionality may be achieved. Hence, any two components herein combined to achieve a particular functionality may be seen as "associated with" each other such that the desired functionality is achieved, irrespective of architectures or intermedial components. Likewise, any two components so associated may also be viewed as being "operably connected", or "operably coupled", to each other to achieve the desired functionality, and any two components capable of being so associated may also be viewed as being "operably couplable" to each other to achieve the desired functionality. Specific examples of operably couplable include but are not limited to physically mateable and / or physically interacting components and / or wirelessly interactable and / or wirelessly interacting components and / or logically interacting and / or logically interactable components.
[0387] With respect to the use of substantially any plural and / or singular terms herein, those having skill in the art can translate from the plural to the singular and / or from the singular to the plural as is appropriate to the context and / or application. The various singular / pl ural permutations may be expressly set forth herein for sake of clarity.
[0388] It will be understood by those within the art that, in general, terms used herein, and especially in the appended claims (e.g., bodies of the appended claims) are generally intended as "open" terms (e.g., the term "including" should be interpreted as "including but not limited to," the term "having" should be interpreted as "having at least," the term "includes" should be interpreted as "includes but is not limited to," etc.). It will be further understood by those within the art that if a specific number of an introduced claim recitation is intended, such an intent will be explicitly recited in the claim, and in the absence of such recitation no such intent is present. For example, where only one item is intended, the term "single" or similar language may be used. As an aid to understanding, the following appended claims and / or the descriptions herein may include usage of the introductory phrases "at least one" and "one or more" to introduce claim recitations. However, the use of such phrases should not be construed to imply that the introduction of a claim recitation by the indefinite articles "a" or "an" limits any particular claim including such introduced claim recitation to embodiments including only one such recitation, even when the same claim includes the introductory phrases "one or more" or "at leastone" and indefinite articles such as "a" or "an" (e.g., "a" and / or "an" should be interpreted to mean "at least one" or "one or more"). The same holds true for the use of definite articles used to introduce claim recitations. In addition, even if a specific number of an introduced claim recitation is explicitly recited, those skilled in the art will recognize that such recitation should be interpreted to mean at least the recited number (e.g., the bare recitation of "two recitations," without other modifiers, means at least two recitations, or two or more recitations). Furthermore, in those instances where a convention analogous to "at least one of A, B, and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B, and C" would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together, etc.) In those instances where a convention analogous to "at least one of A, B, or C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B, or C" would include but not be limited to systems that have A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together, etc.). It will be further understood by those within the art that virtually any disjunctive word and / or phrase presenting two or more alternative terms, whether in the description, claims, or drawings, should be understood to contemplate the possibilities of including one of the terms, either of the terms, or both terms. For example, the phrase "A or B" will be understood to include the possibilities of "A" or "B" or "A and B." Further, the terms "any of" followed by a listing of a plurality of items and / or a plurality of categories of items, as used herein, are intended to include "any of," "any combination of," "any multiple of," and / or "any combination of multiples of" the items and / or the categories of items, individually or in conjunction with other items and / or other categories of items. Moreover, as used herein, the term "set" is intended to include any number of items, including zero. Additionally, as used herein, the term "number" is intended to include any number, including zero. And the term "multiple", as used herein, is intended to be synonymous with "a plurality".
[0389] In addition, where features or aspects of the disclosure are described in terms of Markush groups, those skilled in the art will recognize that the disclosure is also thereby described in terms of any individual member or subgroup of members of the Markush group.
[0390] As will be understood by one skilled in the art, for any and all purposes, such as in terms of providing a written description, all ranges disclosed herein also encompass any and all possible subranges and combinations of subranges thereof. Any listed range can be easily recognized as sufficiently describing and enabling the same range being broken down into at least equal halves, thirds, quarters, fifths, tenths, etc. As a non-limiting example, each range discussed herein may be readily broken down into a lower third, middle third and upper third, etc. As will also be understood by one skilled in the art all language such as "up to," "at least," "greater than," "less than," and the like includes the number recited and refers to ranges which can be subsequently broken down into subranges as discussed above. Finally, as will be understood by one skilled in the art, a range includes each individual member. Thus, for example, a group having 1-3 cells refers to groupshaving 1, 2, or 3 cells. Similarly, a group having 1-5 cells refers to groups having 1 , 2, 3, 4, or 5 cells, and so forth
[0391] Moreover, the claims should not be read as limited to the provided order or elements unless stated to that effect. In addition, use of the terms "means for" in any claim is intended to invoke 35 U.S.C. §112, If 6 or means-plus-function claim format, and any claim without the terms "means for" is not so intended.
Claims
CLAIMSWhat is claimed is:1 . A method performed by a wireless transmit / receive unit (WTRU), the method comprising: receiving, from a consumer device, an augmented reality (AR) resource solicitation request comprising one or more smart contract terms and one or more quality requirements associated with one or more AR resources; determining that one or more device capabilities of the WTRU meet the one or more quality requirements; generating a smart contract based on the one or more smart contract terms; generating the one or more AR resources based on the smart contract; and transmitting the one or more AR resources to the consumer device.
2. The method of claim 1, wherein the one or more quality requirements comprise a resource type indicative of at least one of: a first AR resource type indicative of one or more original AR resources, or a second AR resource type indicative of one or more processed AR resources.
3. The method of claim 2, wherein generating one or more AR resources includes generating sensory data using one or more sensors of the WTRU.
4. The method of claim 3, the method further comprising: transmitting the sensory data as the one or more AR resources when the resource type is indicative of the first AR resource type5. The method of claim 4, the method further comprising: processing the sensory data when the resource type is indicative of the second AR resource type; and transmitting processed sensory data as the one or more AR resources.
6. The method of claim 1, wherein the one or more quality requirements comprise a freshness requirement associated with a time of generation of the one or more AR resources.
7. The method of claim 6, the method further comprising: transmitting a timestamp indicative of the time of generation of the one or more AR resources.
8. The method of claim 1, wherein the one or more smart contract terms comprise at least one of: a service fee associated with the smart contract or a penalty fee associated with the smart contract.
9. The method of claim 1 , wherein the AR resource solicitation request further comprises at least one of: one or more physical target objects to be captured, one or more geographical locations associated with generation of the one or more AR resources, or one or more requirements on technical specifications associated with the one or more AR resources to be generated.
10. The method of claim 9, wherein the one or more requirements on technical specifications are indicative of at least one of: a camera orientation, a camera pose, or one or more camera settings11. The method of claim 1 , the method further comprising: negotiating the smart contract with the consumer device; and generating a final smart contract based on the negotiation.
12. The method of claim 11 , wherein the one or more AR resources are generated based on the final smart contract, and wherein the final smart contract is stored in a blockchain network.
13. A method performed by a wireless transmit / receive unit (WTRU), the method comprising: discovering a plurality of provider devices; transmitting, to the plurality of provider devices, an augmented reality (AR) resource solicitation request indicative of a plurality of quality requirements; receiving a smart contract from at least one provider device of the plurality of provider devices; transmitting the smart contract to a blockchain network; and receiving one or more AR resources based on the smart contract.
14. The method of claim 13, the method further comprising: determining whether the smart contract is acceptable; identifying one or more parts of the smart contract for negotiation if the smart contract is not acceptable; and transmitting, to the at least one provider device, a negotiation indication associated with the one or more parts of the smart contract.
15. The method of claim 13, the method further comprising: generating an AR scene based on the one or more AR resources.
16. The method of claim 15, the method further comprising: evaluating the one or more AR resources based on the plurality of quality requirements; generating, based on the evaluation, a quality evaluation result indicative of at least one of: one or more quality requirements of the plurality of quality requirements that are met by the one or more AR resources, or one or more quality requirements of the plurality of quality requirements that are not met by the one or more AR resources; and transmitting the quality evaluation result to the smart contract stored in the blockchain network.
17. The method of claim 16, wherein the smart contract is executed based on the quality evaluation result.
18. The method of claim 16, wherein evaluating the one or more AR resources comprises: generating sensory data using one or more sensors of the WTRU;comparing the sensory data with the one or more AR resources received from the at least one provider device; and determining an accuracy of the one or more AR resources based on the comparison, wherein the quality evaluation result is indicative of the accuracy of the one or more AR resources.
19. The method of claim 18, the method further comprising: re-calibrating the AR scene based on the accuracy of the one or more AR resources.
20. The method of claim 16, wherein evaluating the one or more AR resources comprises determining whether the one or more AR resources are indicative of one or more physical target objects indicated by the AR resource solicitation request.
Citation Information
Patent Citations
An electronic device for providing a virtual environment for generating synthetic data, a method of operating the electronic device, and a system including the electronic device
KR102519094B1
Smart city commodity exchange with smart contracts
US20190102850A1
Augmented reality system
US20190122443A1
Real-time quality control during manufacturing using augmented reality
US20190279433A1
Environmentally adaptive extended reality display system (as amended)
US20220254120A1