Virtual, augmented and augmented reality system

Abstract

The invention relates to a virtual, augmented and augmented reality system. Systems, methods, and apparatus for a multi-domain, computer-generated reality system are disclosed. A method for managing multi-domain, computer-generated reality includes determining one or more differences between each activity of a first participant and a corresponding baseline activity of each of a plurality of activities associated with traversal of a managed reality system during a session, and quantizing the one or more differences to obtain a performance metric. The method includes combining at least one performance metric for each activity of the first participant to obtain a session performance measurement for the first participant.

Description

[0001] This application is a divisional application of the application with application date of April 22, 2020, application number 202080045461.4, and invention name “Virtual, augmented and extended reality system”. priority

[0002] This application claims priority to and the benefit of U.S. Provisional Patent Application No. 16 / 854,743, filed with the U.S. Patent Office on April 21, 2020, which claims priority to and the benefit of U.S. Provisional Patent Application No. 62 / 837,141, filed with the U.S. Patent Office on April 22, 2019, U.S. Provisional Patent Application No. 62 / 867,224, filed with the U.S. Patent Office on June 26, 2019, U.S. Provisional Patent Application Serial No. 62 / 903,710, filed with the U.S. Patent Office on September 20, 2019, and U.S. Provisional Patent Application Serial No. 62 / 986,621, filed with the U.S. Patent Office on March 6, 2020, the entire contents of which are incorporated herein by reference as if fully set forth herein and for all applicable purposes. Technical Field

[0003] The present invention relates generally to virtual reality, augmented reality, and cross reality systems, and more particularly to a system that provides a universal portal to multiple virtual reality, augmented reality, and cross reality systems. background

[0004] Virtual reality (VR), augmented reality (AR), and cross or extended reality (XR) systems are being deployed to provide specific services, including gaming and control systems. VR provides a computer-simulated reality in which participants can be immersed in gaming environments, simulations for training, and other emerging uses. AR provides computer-generated information that can be overlaid on a participant's direct or captured view of their physical environment. AR can provide contextually relevant information to users and / or support gaming that takes place in the physical world. XR can be used to describe applications that use a combination of VR and AR, including in human-computer interaction systems involving closed-loop control through a combination of sensors that provide tactile, auditory, and visual feedback.

[0005] Conventional VR, AR, and XR systems have been implemented using proprietary and / or standards-based protocols that provide a limited architecture with limited interoperability. Devices used to provide input and / or output (I / O) are also not designed with a strong emphasis on interoperability. For example, a VR headset designed for a first gaming environment may not be directly usable in a second gaming environment. Users of multiple conventional VR and AR systems must typically establish a profile and be present in each VR and AR system in order to participate in activities involving computer-simulated reality. The configuration and operation of peripheral devices used in conventional VR and AR systems can also be complex, and it can be difficult to use peripheral devices across multiple VR and AR systems.

[0006] Improved interoperability and reduced configuration complexity are needed to enable VR, AR, and XR systems to realize their full potential. Overview

[0007] Certain aspects disclosed herein relate to improved methods and systems for virtual and augmented reality systems.

[0008] In one aspect of the present disclosure, a method for managing a multi-domain, computer-generated reality includes: receiving information generated by one or more sensors in a device worn by a participant in a session conducted on a managed reality system, wherein the information generated by the one or more sensors indicates a focus or viewpoint of the participant; providing focus measurements for the participant based on the information generated by the one or more sensors, each focus measurement characterizing a duration that the participant focuses on a leader of the session or one or more objects indicated by the leader of the session while participating in multiple activities associated with the session; calculating a difference between the focus measurements of the participant and a corresponding baseline focus measurement of the session; and generating a session performance metric for the participant by quantifying the difference.

[0009] In some examples, the method includes: authenticating a user of a reality situation at a controller that manages or controls the multi-domain, computer-generated reality; configuring a logical connection between the reality situation and the multi-domain, computer-generated reality using one or more physical communication channels; and establishing the user's presence as a participant in the multi-domain, computer-generated reality through the logical connection while maintaining the user's presence in the reality situation.

[0010] In one aspect of the present disclosure, a method for managing a multi-domain, computer-generated reality includes determining one or more differences between each activity of a first participant during a session and a corresponding baseline activity for each of a plurality of activities associated with traversal of a managed reality system, and quantifying the one or more differences to obtain a performance metric. The method includes combining at least one performance metric for each activity of the first participant to obtain a session performance measure for the first participant. In some cases, the performance metric can be used for future persistence and retention predictions.

[0011] In one aspect, the baseline activity is associated with the leader of the session. The baseline activity can be obtained from a collection of previous sessions. One or more differences can include a difference in location of the first participant's avatar and the session leader's avatar. One or more differences can include a difference in time when the first participant's avatar arrives at a location and a corresponding time when the session leader's avatar arrives at the location. One or more differences can include a difference in time when the first participant's avatar leaves a location and a corresponding time when the session leader's avatar leaves the location. One or more differences can include a difference in time the first participant's avatar spends at a location and a corresponding time the session leader's avatar spends at the location.

[0012] In one aspect, the method includes determining a focus parameter for a first participant based on input received from one or more sensors managed by a device operated by the first participant while participating in a session, and combining the focus parameter with at least one performance metric for each activity of the first participant when obtaining session performance measurements for the first participant. The one or more sensors may include a motion sensor. The one or more sensors may include a position sensor. The one or more sensors may include an audio sensor.

[0013] In one aspect, determining the attention parameter includes monitoring chat activity of the first participant.Determining the attention parameter may include monitoring access to system information by the first participant.

[0014] Computer-generated reality may include simulations. Computer-generated reality can enable experiential teaching reality. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1

[0014] An example of a system enabling coexistence and collaboration among multiple instances of computer-generated or computer-assisted realities according to certain aspects disclosed herein is shown.

[0016] Figure 2 Shown are examples of participant profiles and configuration information according to certain aspects disclosed herein.

[0017] Figure 3is an illustration of participant access according to certain aspects of the present disclosure Figure 1 state diagrams of the system and certain aspects related to configuration.

[0018] Figure 4 A unique user ID configured in accordance with certain aspects of the present disclosure is shown.

[0019] Figure 5 An environment is shown that allows a participant to engage in one or more VR, AR, and / or XR realities according to certain aspects disclosed herein.

[0020] Figure 6

[00145] Illustrated is an architecture for an integrated VR, AR, and / or XR system that may be adapted according to certain aspects disclosed herein.

[0021] Figure 7

[0066] Shown are examples of managed system architectures that may be provided in accordance with certain aspects disclosed herein.

[0022] Figure 8 Shown is an example of a relationship established between two participants by a common XR manager implemented according to certain aspects disclosed herein.

[0023] Figure 9 Shown is an example of a logical system architecture including a common XR manager provided according to certain aspects disclosed herein.

[0024] Figure 10

[00145] A first example of a descriptive structure that may be used to manage virtual, augmented, and extended reality systems implemented according to certain aspects disclosed herein is shown.

[0025] Figure 11

[0066] A second example of a descriptive structure that may be used to manage virtual, augmented, and extended reality systems implemented according to certain aspects disclosed herein is shown.

[0026] Figure 12 is a message flow diagram illustrating certain aspects of operations related to a multi-device VR, AR, and / or XR system.

[0027] Figure 13 is a logical representation of a system that can be implemented according to certain aspects disclosed herein.

[0028] Figure 14 is a functional representation of a system operable in accordance with certain aspects disclosed herein.

[0029] Figure 15 is a logical representation of a VR / AR / XR session configuration flow according to certain aspects disclosed herein.

[0030] Figure 16is a flow chart illustrating an example of a VR / AR / XR session conducted by a leader according to certain aspects disclosed herein.

[0031] Figure 17 is a flow diagram illustrating a participant perspective of a VR / AR / XR session conducted according to certain aspects disclosed herein.

[0032] Figure 18 Illustrated is a managed VR / AR / XR system from which performance data may be collected, aggregated, and analyzed according to certain aspects disclosed herein.

[0033] Figure 19 Showing some aspects of the present disclosure Figure 18 A first example of engagement with the spatial aspects of traversal of a first reality situation.

[0034] Figure 20 Showing some aspects of the present disclosure Figure 18 A second engagement example of the spatial aspects of the traversal of the first reality situation.

[0035] Figure 21 An example of engagement involving temporal aspects of traversing a portion of a managed VR / AR / XR system is shown, in accordance with certain aspects disclosed herein.

[0036] Figure 22 Examples of participation reports that may be generated for sessions in a multi-participant environment are shown in accordance with certain aspects of the present disclosure.

[0037] Figure 23 One example of a graphical representation of engagement information presented in accordance with certain aspects of the present disclosure is shown.

[0038] Figure 24 Certain other parameters according to certain aspects disclosed herein that may be helpful in evaluating performance within a managed VR / AR / XR system are shown.

[0039] Figure 25 Shown are examples of data flows related to parameters that may facilitate performance evaluation within a managed VR / AR / XR system.

[0040] Figure 26 An example of an experiential teaching scenario provided according to certain aspects disclosed herein is shown.

[0041] Figure 27 Examples of scenarios provided to a docent, lecturer, guide, escort, or other leader within a VR timeline, reality scenario, and / or reality type are shown.

[0042] Figure 28Examples of scenarios presented to participants within a VR timeline, reality context, and / or reality type are shown.

[0043] Figure 29 Scenes including or interacting with multiple virtual reality situations according to certain aspects of the present disclosure are shown.

[0044] Figure 30 A process for creating realistic scenarios from instructional course materials, according to certain aspects disclosed herein, is shown.

[0045] Figure 31 An example of a system for converting instructional material into one or more real-life scenarios constructed according to certain aspects disclosed herein is shown.

[0046] Figure 32 Shown are visualization tools provided according to certain aspects disclosed herein.

[0047] Figure 33 One example of an apparatus employing processing circuitry that may be adapted according to certain aspects disclosed herein is shown.

[0048] Figure 34 is a flow chart illustrating a method for creating a multi-domain, computer-generated reality according to certain aspects disclosed herein.

[0049] Figure 35 is a flow chart illustrating a first example of a method for managing multi-domain, computer-generated reality according to certain aspects disclosed herein.

[0050] Figure 36 is a flow chart illustrating a method for operating a multi-domain, computer-generated reality according to certain aspects disclosed herein.

[0051] Figure 37 is a flow chart illustrating a second example of a method for managing multi-domain, computer-generated reality according to certain aspects disclosed herein.

[0052] Figure 38 is a flowchart 3800 illustrating a method for creating a virtual reality system according to certain aspects of the present disclosure. Detailed description

[0053] The detailed description set forth below in conjunction with the accompanying drawings is intended as a description of various configurations and is not intended to represent the only configuration in which the concepts described herein may be practiced. The detailed description includes specific details in order to provide a thorough understanding of the various concepts. However, it will be apparent to those skilled in the art that these concepts may be implemented without these specific details. In some instances, well-known structures and components are shown in block diagram form to avoid obscuring these concepts.

[0054] Several aspects of VR, AR, and XR systems will now be presented with reference to various apparatuses and methods. These apparatuses and methods will be described in the detailed description below and illustrated in the accompanying drawings by various blocks, modules, components, circuits, steps, processes, algorithms, etc. (collectively, "elements"). These elements can be implemented using electronic hardware, computer software, or any combination thereof. Whether such elements are implemented as hardware or software depends on the specific application and design constraints imposed on the overall system. In addition, the term "headset" will be used synonymously for any type of head-mounted device, head-mounted device (HMD), goggles, or glasses that provide VR and / or AR functionality, whether alone or together.

[0055] For example, an element or any part of an element or any combination of elements can be implemented with a "processing system" that includes one or more processors. Examples of processors include microprocessors, microcontrollers, digital signal processors (DSPs), field programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gating logic, discrete hardware circuits, and other suitable hardware configured to perform the various functions described throughout this disclosure. One or more processors in a processing system can execute software. Software should be broadly understood to mean instructions, instruction sets, codes, code segments, program codes, programs, subroutines, software modules, applications, software applications, software packages, routines, subroutines, objects, executable programs, execution threads, processes, functions, etc., whether referring to software, firmware, middleware, microcode, hardware description languages, or other. The software may reside on a processor-readable storage medium. Processor-readable storage media, which may also be referred to herein as computer-readable media, may include, for example, magnetic storage devices (e.g., hard disk, floppy disk, magnetic stripe), optical disks (e.g., compact disk (CD), digital versatile disk (DVD)), smart cards, flash memory devices (e.g., card, stick, key drive), near field communication (NFC) tokens, random access memory (RAM), read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), registers, removable disks, carrier waves, transmission lines, and any other suitable medium for storing and transmitting software. The computer-readable medium may reside in the processing system, be external to the processing system, or be distributed across multiple entities including the processing system. The computer-readable medium may be embodied in a computer program product. For example, a computer program product may include the computer-readable medium in packaging materials. Those skilled in the art will recognize how to best implement the described functionality presented throughout this disclosure based on the specific application and the overall design constraints imposed on the overall system. Overview

[0056] Certain aspects of the present disclosure relate to systems, apparatus, and methods for use in VR, AR, and XR environments. A controller for interconnecting multiple computer-generated realities may authenticate a participant of a first device and a second device coupled to the controller, receive a message from the first device including a request to participate in a multi-domain application, configure a first communication path between the controller and the second device, configure a second communication path between the first device and the second device, and send at least one message related to the multi-domain application via the first communication path.

[0057] A communication path provides a method for providing, in a multi-domain, computer-generated reality, the ability to determine one or more differences between each activity of a first participant during a session and a corresponding baseline activity for each of a plurality of activities associated with traversal of a managed reality system and quantifying the one or more differences to obtain a performance metric. Aspects of VR, AR, and XR Systems

[0058] According to certain aspects disclosed herein, VR, AR, and XR systems may be provided using a combination of processing devices and systems that are capable of communicating with each other to some extent. VR, AR, and XR systems are typically developed and operated according to one or more proprietary standards and protocols. In one example, a VR game may be developed by a first provider according to a set of programming interfaces and / or protocols that define the structure of systems and software that can interoperate with products and services deployed by other providers. Certain aspects disclosed herein provide or enable a platform that can provide interoperability between different VR, AR, and XR systems and experiences, regardless of the nature of the systems and software used to develop and deploy the different VR, AR, and XR systems and experiences.

[0059] Figure 1An example of a system 100 architecture is shown, which can support or enable coexistence and collaboration among multiple participants accessing multiple VR, AR, and / or XR systems and / or multiple instances of computer-generated or computer-assisted realities (experiences). System 100 can be built on one or more processing systems and / or processing circuits 102. In one example, system 100 can implement a closed environment for simulating complex machinery, where the virtual environment is generated using one or more processing circuits operating under unified control and interacting with a processing system that manages feedback sensors, transducers, and other electromechanical simulation devices. In another example, system 100 can implement a multi-player online educational environment, where each participant or a small group of participants participates via a console (such as a game console), personal computer, mobile phone, or other communication device. In another example, system 100 can implement a multiplayer online role-playing game, where each player or a small group of players participates via a game console, personal computer, mobile phone, or other communication device.

[0060] In the example shown, system 100 may include a common communication infrastructure 104 that interconnects components of the processing system and / or processing circuitry 102 and provides a communication channel between VR, AR, and XR instances, which may be referred to herein as reality context 106. Common communication infrastructure 104 includes some combination of electronic circuitry and software configured to implement a communication protocol. Common communication infrastructure 104 may be distributed across different physical devices. Common communication infrastructure 104 implemented independently in different devices may collaborate via one or more network technologies.

[0061] The common communication infrastructure 104 can provide a transport mechanism for carrying controls and data between input / output devices 108 and / or between various reality environments 106 and certain input / output devices 108. The communication channels can be implemented using any available and / or suitable network interface and associated network protocol. The input / output devices 108 can include a keyboard, mouse, joystick, position sensor, microphone, camera, motion detector, as well as audio playback devices, displays and other video outputs (including display systems, smart glasses, and gaming headsets).

[0062] An adaptation subsystem 110 is provided to interface each reality context 106 and input / output device 108 with the common communication infrastructure 104 and, through the common communication infrastructure 104, with other reality contexts 106 and / or other input / output devices 108. In the example shown, a first set of reality contexts 112 (C1 to C X-1) can be developed according to a first set of programming interfaces and / or protocols, and a first adapter 114 can be implemented to convert the inputs, outputs, definitions, descriptors, and behaviors of the first set of reality scenarios 112 into a common set of inputs, outputs, definitions, descriptors, and behaviors. Other sets of reality scenarios 106 can be developed according to other sets of programming interfaces and / or protocols, and other adapters 116, 118 can be implemented to convert the inputs, outputs, definitions, descriptors, and behaviors into this common set of inputs, outputs, definitions, descriptors, and behaviors.

[0063] Adapters 124, 126, 128 may be provided for each type of input / output device 108 or group 122 of input / output devices 108. Adapters 124, 126, 128 may be configured to convert device-specific resolutions, capabilities, sensitivities, and definitions of input / output devices 108 into a common set of resolutions, capabilities, sensitivities, and definitions. System 100 may maintain participant-specific configuration information 120, which may be accessed after a participant has been authenticated by the system. Configuration information 120 may include participant profile information, subscriptions, credits, metadata, avatars, licenses, and / or other parameters that affect a participant's presence, appearance, personality, and / or image in different reality scenarios 106.

[0064] Figure 2 Certain aspects related to an example of participant profile and configuration information 200 are shown, which may correspond to Figure 1 Configuration information 120 is shown. Participant profile and configuration information 200 can be accessed through authentication functionality 202, which can use or include any number of hardware and software security features that can be operated to limit access to the system 100 to authorized participants. Security features can range from biometric information to password controls. Once authenticated, a participant can access a presence management utility 204, which allows the participant to access or modify features and / or enter one or more reality contexts 106. These features can include subscriptions 206, which identify the reality contexts 106 available to the participant and define the rights, benefits, status, and participation level in each accessible reality context 106. Subscriptions can define the rights, benefits, status, and participation level of the participant in the corresponding reality context 106. These features can include relationships 208, which identify certain other participants and the nature of the relationship between the associated participant and the participant.

[0065] These features may include presence information 210. Presence information may include parameters defining skill levels, input / output capabilities of the system, avatars, historical performance, and other information. These features may include information 212, 214 describing one or more system environments that a participant may acquire to access one or more VR, AR, and / or XR systems. These features may include credit management information that defines benefits and / or payment methods earned or accumulated for use in extending or maintaining a participant's presence in one or more reality scenarios 106, as well as licensing credits or fees that may be associated with any described features of the participant profile and configuration information 200. Certain features may be stacked based on different sets of environments involving or associated with a participant, and may include overlapping or separate reality scenarios 106 that intersect with certain input / output devices 108, which may include sensor input devices associated with the participant.

[0066] These features may include message 218. Message 218 may define different levels of communication. For example, a participant may define limits on audio, text, and in-game communication for communicating with other participants. In another example, a participant may configure network parameters that allow personal computing devices to connect to other processors in the public communication infrastructure 104.

[0067] Figure 3 3 is a state diagram 300 illustrating a user entering system 100 and certain aspects related to configuration. The state diagram is provided as an example, showing some (but not all) possible states and transitions. A user can have a presence represented by a user profile. Initially, the user profile is in an idle state 302 and typically remains in the idle state 302 until the user attempts to enter system 100. The user profile can enter an authenticated state 304, where the user is asked to prove their identity. If the user fails to provide sufficient authentication information to prove their identity, the user profile can return to the idle state 302. If the user provides sufficient authentication information to prove their identity, the user can become a participant, and the user profile can enter either the active state 320 or the configured state 306. The term "participant" can apply to users who have been authenticated and authorized to access the TCXR system. In some cases, a participant can select between the active state 320 and the configured state 306. A participant can move between the active state 320 and the configured state 306 as needed or desired.

[0068] In some implementations, a participant can begin navigating from the configuration state 306 to configure a relationship 308, presence 310, or subscription 312. Updates to the user profile can be captured and propagated in state 314 before the user profile returns to the top-level configuration state 306. The user profile can be propagated to related participants and / or one or more reality scenarios 106 or managers of reality scenarios 106 that have an association or affiliation with the participant. In some cases, a credit profile 316 can be configured or accessed from the configuration state 306. In one example, the credit profile 316 can be accessed in association with the configuration of a subscription 312.

[0069] In some implementations, the active state 320 can be used to enter one or more realities 3241-324 N , in order to participate in services or events provided by one or more realities, and participants can be from one or more realities 3261-326 N Exit. In some implementations, a participant can participate in multiple realities simultaneously. While entering or exiting one or more realities and / or in the participating state 328, the user profile can be updated and / or read from the active state 320. In some cases, the participant can access and update subscriptions 312 and / or credit profiles 316.

[0070] refer to Figure 4 At the time of authentication, and before a participant is allowed to participate in a real-world scenario, a plurality of unique user identifiers (UUIDs) 418 may be selected and / or combined from available types of UUIDs 400 to identify, authorize, and / or enable the authenticated user to access and use features of the TCXR system. Types of UUIDs 400 include, but are not limited to, participant UUIDs 402 ("P-UUIDs"), institution UUIDs 406 ("I-UUIDs"), leader UUIDs 404 ("L-UUIDs") (including UUIDs for instructors, mentors, guides, chaperones, motivators, and other leaders), group UUIDs 410 ("G-UUIDs") associated with a class, instruction, and / or course, real-world scenario UUIDs 408 ("R-UUIDs"), guardian UUIDs 412 ("X-UUIDs"), TA UUIDs 414 ("T-UUIDs"), and observer UUIDs 416 ("O-UUIDs"). One or more UUIDs 400 can be grouped and matched in a variety of ways to authorize participants to enter one or more reality scenarios. It also allows parties to monitor or help lead through the reality scenarios. Figure 4Other example UUIDs are provided in a partial list 420 of UUIDs in

[0045] . This list includes: UUIDs 420 that can be used to identify or describe objects or concepts in real-world scenarios (Object UUID 422 or "Oo-UUID"; Presentation UUID 424 or "Op-UUID"; Theater UUID 426 or "Ot-UUID"; and Location UUID 428 or "Ol-UUID"). These may also be considered for authorization, depending on the participant. Certain other types of UUIDs not shown in this list but disclosed herein include CL-UUIDs, CT-UUIDs, CM-UUIDs, and others, which may include UUIDs that can be used in real-world scenarios to better describe TCXR characteristics of objects.

[0071] Figure 5 A participant environment 500 is shown that allows a participant to participate in one or more VR, AR, and / or XR realities in accordance with certain aspects disclosed herein. The illustrated participant environment 500 is merely one example, and other combinations of peripheral devices, systems, and services may be combined to provide a VR, AR, and / or XR experience in accordance with certain aspects disclosed herein. A participant may interact with the VR, AR, and / or XR reality through video, audio, and tactile input and output.

[0072] Haptic input and output can be provided in wearable device 502 and can include sensors and / or transducers that can detect or sense motion, pressure, temperature, impact, direction, shape, or other perceptible parameters. Wearable device 502 can include gloves, helmets, head-mounted devices, and / or other items that a participant can wear or carry. In the example shown, the haptic input and output can be coupled to a local controller 512 associated with the participant.

[0073] Video input and output can be provided using displays and cameras associated with the participants. In some cases, a head-mounted device 504 worn by the participant can display images and can include one or more cameras 506 that can be used to capture still and / or video images. Video input and output can also be provided using displays and cameras present at locations visited or occupied by the participant, and video equipment at such locations can be independently coupled to a centralized system or a local controller 512 associated with the participant.

[0074] Audio input and output may be provided using speakers 510 and microphone 508, which may be incorporated into headset 504. Audio input and output may also be provided using speakers and microphones present at locations visited or occupied by participants, and audio equipment at such locations may be independently coupled to a centralized system or to a local controller 512 associated with the participant.

[0075] Participants can also use a keyboard 514, a mouse, and / or other input devices to interact with one or more VR, AR, and / or XR realities. Such input devices can be coupled to a local controller 512 or another device operated by the participant.

[0076] Participants can also interact with mechatronic devices 516 that are controllable through one or more VR, AR, and / or XR realities. Mechatronic devices 516 can include flight simulators, gaming devices, and props associated with the game. Such mechatronic devices 516 can be coupled to a local controller 512 or another device operated by the participant. Interoperability

[0077] Certain aspects disclosed herein provide a platform that can support and / or interconnect multiple types of VR, AR, and / or XR realities. The platform enables coexistence and collaboration between different active VR, AR, and / or XR systems and can allow participants to participate in multiple instances of computer-generated or computer-assisted realities or experiences simultaneously. In one example, participants can Figure 5 The participant environment 500 is shown interacting with one or more VR, AR, and / or XR realities simultaneously.

[0078] The platform can convert inputs received from the participant environment 500 into a set of common descriptors and functions that can map physical devices to model devices or combinations of devices and / or sensors, input sources, or other information sinks generated by VR, AR, and / or XR systems. The platform can provide information output for transmission to the participant environment 500 from information encoded as a set of common descriptors representing model converters or destinations in the participant environment 500. In various examples, the platform provides services 520 that can include converters or machine learning features that enable the inputs and outputs of the participant environment 500 to coordinate and interact with machine controllers, different types of tactile information converters, and different formats of video, audio, images, and / or three-dimensional (3D) representations of objects.

[0079] Different types of headsets can be used in participant environment 500. For example, headset 522, depicted from a side view, includes a forward-facing camera 524 that can be used for AR and motion detection capabilities. Services 520 can be used to interpret images and / or motion seen through forward-facing camera 524. An internal, inward-facing camera 528 can be used to provide eye tracking capabilities or for individual identification using biometric markers (e.g., iris scan markers). Headset 522 can have both a forward-facing camera 524 and an internal, inward-facing camera 528 to facilitate both services. In some cases, headset 522 can include a single forward-facing camera 524 and a single inward-facing camera 526. Other types of headsets can include straps for charging and connectors for attaching wired or wireless controllers 532. Headset 522 can also incorporate IoT sensors 530 and / or function-specific sensors to detect motion, whether a participant is wearing headset 522, or audio input. For example, a microphone can be included to help identify participants using voice recognition.

[0080] According to certain aspects disclosed herein, the multiple participant environment 500 may be integrated with different proprietary VR, AR, and / or XR systems as well as common sources of VR, AR, and / or XR related information. Figure 6 An architecture 600 is shown that represents one implementation of an example of an integrated VR, AR, and / or XR system. The architecture 600 is represented in three dimensions with multiple functional layers 616 and multiple participant contexts 614. A participant context 614 may refer to a collection of all objects, rights, permissions, preferences, and metadata that may be used by a participant in one or more reality contexts to instantiate the participant's presence, provide the correct permissions to the participant, link the participant to other reality contexts, and present links to other participants in the current reality context where the participant contexts overlap. Each participant context 614 may use Figure 1 The system 100 shown is implemented as shown. Each participant context 614 is represented as a slice of the architecture 600, such that the third dimension of the architecture 600 is composed of multiple instances of the system 100, each of which can be implemented using a different combination of hardware, software, and peripherals. In many cases, it may be better to represent the architecture 600 in more than three dimensions to reveal certain relationships that cannot be adequately represented in three dimensions.

[0081] In some implementations, the participant context 614 may incorporate at least a portion of one or more reality contexts 106 at any time, and the reality context 106 may be modified by being associated with one or more participant contexts 614. The participant context 614 may also be considered an instance of a reality context for an individual participant or a group of participants that includes the individual participant. In one example, the reality context 106 may be modified when a participant enters an associated VR, AR, and / or XR space. The participant may enter a VR, AR, and / or XR space with specific objects, images, audio, and / or video feedback. In another example, the reality context 106 may be modified to absorb information, where the information becomes available when one or more sensors associated with the participant entering the VR, AR, and / or XR space become accessible within the reality context 106. The modified reality context may also be an instance of a reality context for the entering participant or a group of entering participants.

[0082] Processing layer 602 may include various processing circuits, systems, and operating software that support individual participant contexts 614 or portions of participant contexts 614. For example, a participant context 614 may be supported by a game console or a personal computer. A single participant context 614 may be supported by multiple processing circuits, including remote processing circuits operating as servers, conversion systems, and the like. Some processing circuits and systems in processing layer 602 may support multiple participant contexts 614 or portions of multiple participant contexts 614.

[0083] The common communication layer 604 may include devices, circuitry, communication networks, and interfaces that enable individual participant contexts 614 to communicate with each other. In one example, the common communication layer 604 may implement, include, or be coupled to a wireless local area network, a wide area network, and / or one or more cellular networks. The interface layer 606 includes circuitry and / or modules that can translate between individual VR, AR, and / or XR systems and components to corresponding VR, AR, and / or XR systems and components in the common reality context. In some implementations, the common reality context is implemented as a virtual reality operating system (VROS). In some examples, the VROS enables VR systems to seamlessly communicate with XR systems, including when the VR and XR systems use different technologies and models for representing the actions of objects, sensory information received from sensory I / O 612, video, audio, images, and / or 3D representations.

[0084] Each participant context 614 can support one or more reality contexts 608 based on the participant's subscription. When active, a reality context 608 can reside in the participant's environment. In some cases, when a participant is not actively participating in an idle reality context, the idle reality context can reside, allowing the idle reality context to generate alerts, messages, and other information to the participant and / or the active reality context. Subscriptions, reality context activities, avatars, and other configuration and system management information can be maintained as a participant profile 620 accessible through authentication functionality 610. Authentication functionality 610 can identify a participant and provide access to corresponding participant contexts 614 available to the authenticated participant. In some cases, an authenticated participant can be provided access to more than one participant context 614. In some examples, a participant can establish a relationship with another participant that defines restricted access to another participant context 614.

[0085] Figure 7 An example of a managed system architecture 700 that may be provided in accordance with certain aspects disclosed herein is shown. The architecture 700 supports the use of Figure 1 The illustrated system 100 implements multiple participant contexts 714, where each participant context 714 is represented as a slice in the third dimension of the managed system architecture 700. The processing layer 702 may include various processing circuitry, systems, and operating software that support individual participant contexts 714. For example, a participant context 714 may be supported by a gaming console or a personal computer. A single participant context 714 may be supported by multiple processing circuitry, including remote processing circuitry operating as a server, conversion system, etc. Some processing circuitry and systems in the processing layer 702 may support multiple participant contexts 714. The common communication layer 704 may include devices, circuitry, communication networks, and interfaces that enable components of the managed system architecture 700 to communicate with each other. The interface layer 706 includes circuitry and / or modules that can translate between individual VR, AR, and / or XR systems and components to corresponding VROS systems and components. Each participant context 714 may support one or more reality contexts 708 based on the participant's subscription, including certain reality contexts that may reside within the participant context 714 when active.

[0086] Subscriptions, reality context activities, avatars, and other configuration and system management information may be maintained as a participant profile 720 accessible through authentication functionality 710. Authentication functionality 710 may identify a participant and provide access to corresponding participant contexts 714 available to the authenticated participant. In some cases, an authenticated participant may be provided access to more than one participant context 714. In some examples, a participant may establish a relationship with another participant that defines limited access to another participant context 714.

[0087] Relationship-based access and participation in multiple reality contexts 708 can be managed by a common management and / or control layer, which may be referred to herein as a common XR manager 716. In one example, a participant can register multiple reality contexts 708 with the common XR manager 716. In another example, two or more participants can register and / or configure relationships with the common XR manager 716 so that the common XR manager 716 can control access to and participation in one or more reality contexts 708 that are available to participant contexts 714 associated with the corresponding or related participants. Multi-domain applications can be supported, managed, and operated by the common XR manager 716. Multi-domain applications can operate across multiple reality contexts 708 and / or multiple participant environments 714.

[0088] The public XR manager 716 can represent authenticated participants and corresponding participant profiles 720 in multiple contexts. In one example, the public XR manager 716 can interact with participant profiles maintained for participants to propagate aspects of the participant profiles to reality contexts 708 and / or profiles used by other applications. In one example, a participant can define an avatar to represent the participant in multiple realities, and the public XR manager 716 can format and transmit the avatar descriptor to each reality. In another example, the public XR manager 716 can obtain permission to use items purchased in one reality context 708 in other unrelated reality contexts 708.

[0089] The public XR manager 716 can establish and / or maintain connections between multiple reality instances accessible and / or occupied by authenticated participants. The public XR manager 716 can configure channels through the public communication layer 704 that enable information related to authenticated participants to be exchanged between two or more reality instances 708, two or more sensory input / output devices 712, and between one or more reality instances 708, one or more sensory input / output devices 712. The public XR manager 716 can configure VROS translators to support participant-specific input / output devices 712. In some cases, the public XR manager 716 can configure VROS reality instances to provide compatibility between different types of reality instances 708.

[0090] Figure 8An example of a relationship 800 established between two participants by the common XR manager 716 according to certain aspects disclosed herein is shown. The relationship is shown in a logical domain 824 and in an underlying physical domain 826. A first participant context 820 can be implemented using a first processing circuitry and system 802, while a second participant context 822 can be implemented using a second processing circuitry and system 852. In some cases, the first and second participant contexts 822, 822 can share at least some portions of the processing circuitry and systems 802, 852. Respective common communication layers 804, 854 enable communication between the participant contexts 820, 822. Each participant context 820, 822 supports multiple reality contexts, including certain reality contexts 806, 808, 856, 858 that are supported or accessible by both participant contexts 820, 822.

[0091] A common XR manager 828 present in both participant contexts 820, 822 can establish a logical connection 834 between one or more reality contexts 806, 808, 856, 858 in each participant context 820, 822. The logical connection 834 can be requested by a participant and / or one or more multidomain applications 830. For example, a multidomain application 830 can request the establishment of a logical connection 834 between instances of the same reality context 808, 858 maintained on two participant contexts 820, 822 so that the avatars of the two participants can interact in a manner not provided or otherwise supported by the reality contexts 808, 858. In response to the request, the common XR manager 828 can cause one or more physical communication paths 832 to be established between the participant contexts 820, 822 to implement and support the logical connection 834. In some implementations, information generated at one of the participant contexts 820, 822 related to the logical connection 834 can be routed through the common XR manager 828 and / or the physical communication paths 832. In this regard, the common XR manager 828 can operate as a router and / or translator. In some cases, the common XR manager 828 can configure the physical communication path 832 to establish the logical connection 834 and can have limited interaction with communications associated with the logical connection 834, including in implementing the multi-domain application 830's response at the end-to-end communication between the real-world scenarios 808, 858 after the physical communication path 832 is established.

[0092] In some implementations, the multi-domain application 830 can execute independently in two participant contexts 820, 822 and can support interaction between two or more shared reality contexts 806, 808, 856, 858. In one example, the multi-domain application 830 can establish a logical connection 834 between the reality contexts 806, 808 maintained in the first context and the reality context 858 maintained in the second context. The common XR manager 828 can configure the corresponding physical communication path 832 through the common communication layer 804, 854. The common XR manager 828 can manage the nature of the communication between the reality contexts 806, 808 according to the authorization functions 814, 864 in the respective participant contexts 820, 822.

[0093] According to certain aspects of the present disclosure, participant contexts 820 and 822 may include both virtual reality and augmented reality contexts. A public XR manager 828 may configure a participant experience that allows participants to seamlessly transition between virtual reality and augmented reality contexts (individually or as a group). In one example, a group may gather at a physical location, where each member of the group enters an augmented reality context that enables a leader to provide information and / or manage certain initial activities. Activities may include briefing group participants, taking roll calls, assigning tasks and responsibilities, establishing rules and conditions, and performing equipment checks.

[0094] In some cases, the augmented reality scenario can enable the assembled group members to visualize and interact with remote participants operating in a virtual reality scenario associated with the group. The group can then transition from the augmented reality scenario to the virtual reality scenario under the guidance of the leader.

[0095] Figure 9An example of a logical system architecture is shown, which includes a common XR manager 900 provided in accordance with certain aspects disclosed herein. Common XR manager 900 includes a participant manager 902 and a system manager 904, and may include or collaborate with VROS 908. Participant manager 902, system manager 904, and VROS 908 may interact with one or more external services 910. External services 910 may provide access to VR, AR, and / or XR spaces. External services 910 may interact with participant manager 902 and system manager 904 via respective interfaces 912, 914 and / or via VROS 908. In some implementations, interfaces 912, 914 comprise application programming interfaces. Interfaces 912, 914 and / or interfaces associated with VROS 908 may convert descriptions of VR, AR, and / or XR features into and / or conform them to corresponding features of VROS 908. These features may include video formats, audio formats, image formats, formats for 3D representations of objects, avatars, and tactile input and output. Participant system 906 may interact with VROS 908, and one or more interfaces associated with VROS 908 may convert and / or conform descriptions of VR, AR, and / or XR features to corresponding features of participant system 906.

[0096] Participant manager 902 interacts with the participant profile and authentication system. In the illustrated public XR manager 900, participant manager 902 may include an access manager 916 that manages and controls the access rights of each participant. Access manager 916 may determine the access rights of each participant and communicate them to various external services 910.

[0097] In the illustrated public XR manager 900, the participant manager 902 may include a profile subsystem 918 that maintains participant-specific configurations and preferences, including, for example, avatars, system capabilities, including display resolution. The profile subsystem 918 may manage and facilitate the use of subscriptions.

[0098] In the illustrated public XR manager 900, the participant manager 902 may include a group subsystem 920 that identifies each participant's group membership and affiliation. This may include using a UUID associated with user authorization, as necessary. In some cases, group membership and affiliation may be associated with one or more subscriptions, invitations, and / or access rights. In one example, an educational application may support affiliations representing learning groups that may be pre-assigned by a group leader or assigned by an on-the-fly instantiation of a group meeting without a group leader.

[0099] In the illustrated public XR manager 900, the system manager 904 may include an avatar subsystem 922. The avatar subsystem 922 may include a library to help identify the origin of an avatar. In one example, an avatar associated with each reality context 106 accessible to a participant may be obtained through the access manager 916. The avatar subsystem 922 may provide or maintain configuration information in the profile subsystem 918, including information related to avatar type, pseudonym, clothing, skills, and / or decorations. The avatar subsystem 922 may enable conversion of an avatar obtained from one reality context 106 for use in various other reality contexts 106 indicated as accessible to the participant in the profile subsystem 918, where permitted by agreement with the licensee, where applicable.

[0100] In the illustrated common XR manager 900, the system manager 904 may include an object subsystem 924. The object subsystem 924 may be configured to participate in converting objects known in one reality context 106 into corresponding objects in other reality contexts 106 that are indicated as accessible to participants in the profile subsystem 918. The object subsystem 924 may provide or maintain configuration information in the profile subsystem 918 as well as any licensing considerations in the interservice purchase subsystem 928 or the DRM module 930.

[0101] In the illustrated public XR manager 900, the system manager 904 may include an application subsystem 926. The application subsystem 926 may be configured to participate in converting known application entities in one reality context 106 into corresponding application entities in other reality contexts 106 that are indicated as accessible to participants in the profile subsystem 918. The application subsystem 926 may provide or maintain configuration information in the profile subsystem 918.

[0102] In the illustrated public XR manager 900, the system manager 904 may include an inter-service purchasing subsystem 928. The inter-service purchasing subsystem 928 may be configured to participate in acquiring commercially available objects known in one reality context 106 and converting them into corresponding objects in other reality contexts 106 that are indicated as accessible to participants in the profiling subsystem 918. The inter-service purchasing subsystem 928 may provide or maintain configuration information in the profiling subsystem 918. The inter-service purchasing subsystem 928 may facilitate a marketplace for objects between reality contexts 106. In one example, items with a market value in one reality context 106 may be traded or purchased in another reality context 106. A universal currency may be used and / or created for converting values ​​between reality contexts 106.

[0103] The avatar subsystem 922, object subsystem 924, application subsystem 926, and inter-service subsystem 928 may include, interact with, or exchange objects protected as intellectual property of third-party systems or participants. The system manager 904 may include or be integrated with a digital rights management (DRM) module 930, which may negotiate, purchase, and / or sell rights in certain objects. The DRM module 930 may control or gate transmissions via one or more logical connections between participant contexts.

[0104] In the illustrated common XR manager 900, the system manager 904 may include a display subsystem 932 for exchanging images and video between reality situations 106. The system manager 904 may include an input subsystem 934 for exchanging sensory information between reality situations 106.

[0105] Figure 10 A first example of an architecture 1000 that can be used to describe and manage a virtual, augmented, and extended reality system implemented according to certain aspects disclosed herein is shown. Architecture 1000 relates to an example describing certain portions of an input subsystem 934. Input subsystem 934 can be expected to correlate information generated by various types of sensory devices and / or other input information related to context within a virtual, augmented, and extended reality system, and can convert and / or transmit output information related to one or more devices in a participant system.

[0106] This example illustrates entries in structure 1000 related to one or more XR imaging devices. Other examples may involve other types of input devices. For example, an entry may relate to a streaming source that provides recordings, streaming information, chat information, program information, and the like. Entries are maintained and accessed under input subsystem 934. The entry may include an input / output device descriptor 1002 identifying the device type, which in the illustrated example is an imaging device. The input / output device descriptor 1002 may be associated with additional device-specific information 1004, which defines capabilities, status, and other information including manufacturer, model, and software version. The entry may include location information 1008 identifying the physical and / or virtual location of the imaging sensor. The physical location may be maintained as Global Positioning System (GPS) coordinates or another type of coordinate system. In one example, location information 1008 describes the location of the imaging device that contributes to the scene or setting of a virtual reality scenario. In one example, location information 1008 is used to determine the movement of a participant that can be mapped to the virtual reality scenario.

[0107] The entry may include a descriptor of sensory input data 1006 generated by or relating to the corresponding device. In the illustrated example, metadata 1012 corresponding to the video sensory input data 1006 may be maintained with virtual or augmentation information. Metadata 1012 is used to identify, map, index, categorize, and / or otherwise describe such information. Virtual, augmentation, or other information and / or metadata 1012 may include imaging device input / output characteristics 1014 describing the format, resolution, encoding, and other characteristics of image data 1016, which may be associated with or maintained within structure 1000. Image data 1016 may be formatted and / or encoded according to a standard or protocol identified by imaging device output characteristics information 1014 or metadata 1012. In some cases, image data 1016 is included in or comprises a video stream, and the video information may include change vectors 1018 that may be used for video compression. This information may include perspective information 1020, which may be used to position the image within a scene, virtual environment, or physical volume.

[0108] Figure 11 A second example of a structure 1100 that can be used to describe and manage a virtual, augmented, and extended reality system implemented according to certain aspects disclosed herein is shown. Structure 1100 relates to an example describing certain portions of input subsystem 934. Input subsystem 934 can be expected to correlate information generated by various types of sensory devices and / or other input information related to context in the virtual, augmented, and extended reality system, and can convert and / or transmit output information related to one or more devices in a participant system.

[0109] This example shows entries in structure 1100 related to one or more tactile sensors used in an XR system. Other examples may involve other types of input devices. Entries are maintained and accessed under input subsystem 934. The entry may include an input / output device descriptor 1102 identifying the device type, which in the example shown is a tactile sensor. The input / output device descriptor 1102 may be associated with additional device-specific information 1104, which defines capabilities, status, and other information including manufacturer, model, and software version. The entry may include location information 1108 identifying the physical and / or virtual location of the tactile sensor. The physical location may be maintained as Global Positioning System (GPS) coordinates or another type of coordinate system. In one example, location information 1108 describes the location of the tactile sensor that contributes to the scene or setting of a virtual reality environment. In one example, location information 1108 is used to determine the movements made by a participant while interacting with the virtual reality environment.

[0110] The entry may include a descriptor of the sensory input data 1106 generated by or relating to the corresponding device. In the example shown, metadata 1112 corresponding to the tactile sensor input data 1106 may be maintained with virtual or augmented information. Metadata 1112 is used to identify, map, index, categorize, and / or otherwise describe such information. Virtual, augmented, or other information and / or metadata 1112 may include sensor input / output characteristics information 1114 describing the format, resolution, encoding, and other characteristics of the tactile sensor data 1116, which may be associated with or maintained in structure 1100. Tactile sensor data 1116 may be formatted and / or encoded according to a standard or protocol identified by sensor output characteristics information 1114, metadata 1112, and / or sensitivity parameters 1118.

[0111] Figure 12 1200 is a message flow diagram illustrating certain aspects of operations associated with a multi-device VR, AR, and / or XR system. Two participant systems 1202 and 1206 may participate in an integrated VR, AR, and / or XR system via a server or other controller 1204. The illustrated operations may be employed when more than two participant systems 1202 and 1206 are to be used in a multi-device VR, AR, and / or XR system. In the illustrated example, a first participant system 1202 activates a relationship between the two participant systems 1202 and 1206. A participant of a second participant system 1206 may be authenticated 1210 by the second system 1206. The second participant system 1206 may send an authentication request 1212 to the controller 1204 and may become active in the integrated VR, AR, and / or XR system when the controller 1204 responds with an authentication confirmation message 1214. At some point, the participant of the first participant system 1202 may be authenticated 1216 by the first system 1202. The first participant system 1202 may send an authentication request 1218 to the controller 1204 and may become active in the integrated VR, AR, and / or XR system when the controller 1204 responds with an authentication confirmation message 1220 .

[0112] Participants of the first participant system 1202 may cause the first participant system 1202 to send a multi-domain request 1222 to establish a multi-domain relationship and / or participate in a multi-domain application. In response to the multi-domain request 1222, the controller 1204 may exchange control and command information 1224 with the first participant system 1202 and control and command information 1226 with the second participant system 1206 to establish a communication path between the real-world instances in the participant systems 1202 and 1206. The controller 1204 may exchange control and command information 1228 with the first participant system 1202 and control and command information 1230 with the second participant system 1206 to configure the relationship between the participant systems 1202 and 1206 based on the multi-domain request 1222. Thereafter, the participant systems 1202 and 1206 may exchange information related to the multi-domain application in messages 1232 and 1234 transmitted by the controller 1204. These messages 1232, 1234 may include content, control, meta, location, perspective, focus, gesture, and / or notification information between the participants or a group of participants. The participant systems 1202, 1206 may exchange information related to the multi-domain relationship in direct messages 1236 transmitted between the participant systems 1202, 1206. The direct messages 1236 may include content, control, meta, location, perspective, focus, gesture, and / or notification information between the participants or a group of participants.

[0113] Figure 13 1300 is a logical representation of a provisioning process 1300 according to certain aspects disclosed herein. The provisioning process 1300 enables an individual participant and / or a group of participants to interact with or simultaneously participate in a VR / AR / XR experience. The VR / AR / XR experience may include multiple reality scenarios 106, where the reality scenarios 106 may be instantiated using one or more reality types, standards, or environments. Each participant gains access to the VR / AR / XR experience through an authentication and / or authorization process 1302.

[0114] If the user is authorized through the authorization process, the user becomes a TCXR participant ("Participant"). During the authorization process 1302, the TCXR system, through the user portal assistant, will help the user determine the type of participation role allowed and the authorized experience, which can be identified or queried through the participant's profile information.

[0115] At block 1304, the participant may be identified as a leader or a participant. In some cases, when the participant is a leader, a leader participant profile may be selected at block 1320. The participant may be identified as a leader based on information derived from the authentication and / or authorization process 1302. The leader participant profile may provide the necessary permissions for the leader to configure various aspects of the VR / AR / XR experience. The leader may configure the VR / AR / XR experience and participants iteratively and / or in a sequence defined for the VR / AR / XR experience.

[0116] In block 1322, the leader may identify and configure participants in one or more groups to participate in the VR / AR / XR experience. Groups may be created based on the level or type of participation using information received or retrieved from repositories or databases 1312, 1314, and 1316. Before configuring group participation in block 1322, the leader may determine the roles of the participants based on information retrieved from the participant profile repository 1312, the rights management system 1314, and other sources related to participant preferences and capabilities. The leader may update the participant profile repository 1312 and the rights management system 1314 as needed. In block 1328, the leader may select and configure a set of reality scenarios to be included in the VR / AR / XR experience. The set of reality scenarios may be selected from the list or database 1318. In block 1326, the leader configures a framework or structure for directing and controlling the VR / AR / XR experience. In one example, the leader may configure a default participation level for an AIVA. In another example, the leader may configure avatars for one or more sessions. In another example, the leader may define the avatar's initial positioning, perspective, focus, and posture.

[0117] Preconfiguration can involve interaction with the reality scenario database 1310 and / or one or more reality scenarios 106 through the reality scenario interface 1324. In one example, a leader can configure participant participation using information defining preferences, options, permissions, registration, and requirements for participating in a group activity. The leader can follow the preconfiguration process 1300 to configure various aspects of the VR, AR, and / or XR experience for each participant and / or each group before starting the VR, AR, and / or XR experience. At block 1330, and after completing preconfiguration, the leader can store the configuration in the configuration database 1332. In some implementations, the leader can create multiple configurations for one or more VR, AR, and / or XR experiences and can activate the stored configurations before starting the preconfiguration process 1300.

[0118] Participants may also perform pre-configurations and / or test configurations defined by the leader. Participants gain access to the VR / AR / XR experience through an authentication and / or authorization process 1302. At block 1306, the participant may load a configuration from a configuration database 1332. Participants may personalize the configuration to adjust it for the participant's preferences and / or capabilities of the system operated by the participant. Participants may personalize the configuration within limits specified by the profile repository 1312 and / or the rights management system 1314. In some cases, personalization may involve adjusting the configuration based on predefined relationships with other participants and / or the leader.

[0119] Figure 14 1400 is a functional representation of a system 1400 operable in accordance with certain aspects disclosed herein. The system 1400 can enable participants (individual participants or groups) to interact with or concurrently participate in multiple reality scenarios 106, where the reality scenarios 106 can be instantiated using one or more reality types, standards, or environments. Each participant gains access to the system 1400 through an authentication and / or authorization process 1402. The authentication and / or authorization process 1402 can identify the participant, the participant's system capabilities, and / or the reality scenarios 106 in which the participant is currently a participant. A participant can indicate whether they wish to participate in one or more reality scenarios 106, and whether they wish to participate autonomously, as an individual participant in a multi-player reality, or as a member of a group of participants in a shared VR, AR, and / or XR experience. In some cases, a participant can initially enter the system 1400 as an individual and can transition to group participation during or after initial configuration.

[0120] System 1400 may initiate a participant configuration process 1404. In some cases, AIVA participation may be facilitated for a session involving a selected reality scenario 106. In one example, participant configuration process 1404 configures participant participation using configuration information maintained in one or more repositories or databases 1412, 1414, 1416. The configuration information may include information associated with multiple reality scenarios 106, where the reality scenario information may be accessed from reality scenario database 1410. Participant configuration process 1404 may configure participant participation using information in participant profile database 1412, which may define system preference metadata, avatars, and / or other parameters that affect a participant's presence, appearance, personality, and / or image in different reality scenarios 106. Participant configuration process 1404 may configure participant participation using information in rights management database 1414, which may define access rights, credits, and payment information that a participant subscribes to, acquires, or provides to the participant. Participant configuration process 1404 may configure participant participation using information in relationship or affiliation database 1416, which may define preferences, options, permissions, registrations, and requirements that enable a participant to communicate and / or interact with other participants and / or reality scenarios 106. After completing participant configuration process 1404, a participant may participate as one or more reality scenarios 1410 (see also Figure 1 Individual or autonomous participants participate 1406 in a real-life situation 106).

[0121] When a participant is about to be admitted as a group participant, the participant configuration process 1404 may pass the participant configuration information to the group management function 1418, which manages and / or updates the VR, AR, and / or XR experience shared by the group. The group management function 1418 may coordinate with or control the rights management function 1420, which may delegate, assign, acquire, or otherwise provide the participant with the access rights to objects and real-world situations 106 necessary to participate in the shared AR and / or XR experience. In one example, the rights management function 1420 may configure the participant's participation using information defining preferences, options, permissions, registration, and requirements for participating in group activities received or retrieved from one or more repositories or databases 1412, 1414, 1416. The system 1400 may determine the participant's role as part of the leader configuration function 1422. For example, the leader configuration function 1422 may register the participant as a lead participant or a regular or subordinate participant. After configuration, participants may participate 1424 in the shared AR and / or XR experience and / or one or more corresponding or related real-world scenarios 106 .

[0122] Figure 15 1500 is a logical representation of a VR / AR / XR session configuration process according to certain aspects disclosed herein. After performing the authentication and / or authorization process 1502, each participant can participate in the VR / AR / XR session. Participants can be leaders or participants. The participant identified as a leader in block 1504 can select and initiate the initial configuration of the VR / AR / XR session in block 1508. The initial configuration can be received from a configuration database 1506 and may have been created during the pre-configuration process. In block 1510, the leader can adjust or configure the AIVA's participation level. In block 1512, the leader can update or modify the list of authorized participants and / or the participation level of one or more participants. The leader can provide or update the session configuration in a repository 1518 maintained for VR / AR / XR sessions. In block 1514, the leader can initiate the VR / AR / XR session and enter the VR / AR / XR arena. In one example, an instance of a specific reality scenario 106 can be created at this time to facilitate entry by the leader and one or more other participants.

[0123] A user identified as a participant in block 1504 may select a participant configuration for the VR / AR / XR session in block 1520. The participant configuration may be maintained locally or from a shared repository. In one example, a base configuration may be received from configuration database 1506. In block 1522, the participant may adjust or configure their personal system. In block 1524, the participant may retrieve the session configuration from repository 1518. The participant may modify the session configuration by adjusting their appearance, posture, and / or presence before entering the VR / AR / XR venue. In block 1526, the participant may join the VR / AR / XR session.

[0124] Figure 16Flowchart 1600 illustrates an example of a VR / AR / XR session conducted in accordance with certain aspects disclosed herein. Flowchart 1600 represents the experience of a leader of a VR / AR / XR session. In one example, the VR / AR / XR session may involve an educational course, tour, or training event provided on behalf of a commercial, educational, governmental, or non-profit entity ("sponsoring entity"). The sponsoring entity may establish guidelines, evaluation, and testing criteria, and may define reporting criteria for categorizing participants in the VR / AR / XR session. In some cases, the sponsoring entity may evaluate the VR / AR / XR session in advance to credit or reward participants for their attendance. The sponsoring entity may request evaluations of the VR / AR / XR session and its leader from participants of the VR / AR / XR session. The sponsoring entity may define policies related to the conduct of the VR / AR / XR session, its leader, and participants. In some cases, the sponsoring entity may provide VR equipment to the leader and / or some or all participants.

[0125] At block 1602, a leader may load policies, guidelines, assessment forms, and report cards for a VR / AR / XR session. In some implementations, policies may define or govern certain permissible behaviors within a VR / AR / XR session. In one example, policies may include regional policies, institutional policies, acceptable use policies, age policies and regulations, and / or policies related to the leader.

[0126] At block 1604, the leader can configure the session. In pre-configured mode, the leader can configure the venue, the duration of the session, individual and group participants, roles, goals, objectives, and engagement metrics, as well as other aspects of the VR / AR / XR session. When a VR / AR / XR session has been previously configured, the leader can load the existing configuration and modify or adjust certain aspects of the VR / AR / XR session as needed. The sponsoring entity may provide some configuration information, including, for example, profile information, subscriptions, and credits. Certain configuration metadata may be obtained from profiles generated or maintained in different virtual contacts, including appearance, avatars, personality poses, and other appearance characteristics. Imported configuration information related to appearance may be subject to policies defined for the VR / AR / XR session.

[0127] The leader can configure statuses and controls displayed as dials or other control mechanisms. In some examples, statuses and controls can be provided to track participant attendance, presence, and engagement status (e.g., logged in, active, passive, distracted). In some examples, statuses and controls can be provided for timing or other temporal considerations, including statuses and controls related to participant attention and / or the pace of the session. The leader can access identifying information for each participant, including one or more of attendee aliases, nicknames, full names, affiliations, and the like. In some examples, statuses and controls can be provided to manage and monitor scripts, protocols, lesson plans, and the like. The leader can control and / or access one or more active message boxes to communicate privately or publicly with participants. The leader can control participant access to one or more active message boxes to communicate privately or publicly with the leader or other participants. The leader can access a query window that provides session information, including attendance, activity, received queries, and AIVA configuration.

[0128] The leader can control Maestro configuration, including balancing control of the VR / AR / XR session between the leader and AIVA. Maestro can be configured to control presentations, lectures, materials, timing of the environment, classroom control, and monitoring of the VR / AR / XR session. Within a VR / AR / XR session, the leader can change, alter, or filter the parameters controlled by Maestro. For example, parameters can be changed to reflect changes in the number of participants, group performance, query volume, etc. Maestro can be configured to adjust the percentage of administrative control of AIVA from full control to no control. The configuration and / or metadata used to manage Maestro can be configured by the leader for each environment within the venue and can be adjusted from one environment to another based on participant variables, engagement, and actions.

[0129] When engagement levels drop, the leader can configure Maestro to provide stimulation (nudges) to participants. The leader can configure thresholds that determine when stimulation is provided. Maestro can exercise control of a participant's virtual position by controlling the position of their avatar within one or more real-world scenarios during a VR / AR / XR session. In one example, Maestro can move a participant's avatar between locations within a scenario or between scenarios. Avatar movement can be instantaneous or occur at a rate determined by Maestro and / or system parameters. Maestro can be configured to monitor each participant's session completion. Upon completing certain tasks or stages within a VR / AR / XR session, participants may receive deliverables ("Easter Eggs"). Maestro can be configured to manage environment timeouts, which can be adjusted in real time based on group performance within the current VR / AR / XR session or predetermined set criteria.

[0130] At block 1606, the leader may initiate and enter a VR / AR / XR session. In some cases, a VR / AR / XR session may be initiated when the leader or a participant first enters, logs in, or requests it. At block 1608, the leader may initiate an AIVA.

[0131] At block 1610, the leader may perform one or more tests. In one example, the leader may configure and calibrate the location and scene for use in the VR / AR / XR session. In another example, the leader may perform a walkthrough of the location defined for the VR / AR / XR session. In another example, the leader may rehearse a script for the VR / AR / XR session. In another example, the leader may record at least a portion of the content to be used in the VR / AR / XR session. If the leader determines at block 1610 that the VR / AR / XR session is not ready for operation, the leader may return to block 1604 to repeat the configuration process.

[0132] When the leader determines that the VR / AR / XR session is appropriately or sufficiently configured at block 1610, the leader may configure displayable controls and status fields that are presented in the leader's field of view within one or more places of the VR / AR / XR session at block 1612. The leader may also configure authorization and / or adjust the AIVA's participation level.

[0133] At block 1614, the leader may initiate the VR / AR / XR session and perform the objectives of the VR / AR / XR session. At block 1616, the leader may determine whether the goals and objectives of the VR / AR / XR session have been achieved. If one or more participants have not performed all tasks of the VR / AR / XR session and the goals and / or objectives have not been achieved, the leader may continue the VR / AR / XR session at block 1614. At block 1618, if the objectives have been achieved, the leader may terminate the VR / AR / XR session.

[0134] Figure 17 1700 is a flowchart illustrating an example of a VR / AR / XR session conducted in accordance with certain aspects disclosed herein. Flowchart 1700 represents a participant's experience. The participant may be a subscriber or otherwise registered with a sponsoring entity. At block 1702, the participant may load a policy that defines or governs certain permissible behaviors of the participant in the VR / AR / XR session.

[0135] At block 1704, the participant may configure their personal devices and systems to be used in the VR / AR / XR session. In pre-configured mode, the participant may load an existing configuration and, as needed, modify or adjust certain aspects of the VR / AR / XR session corresponding to the participant's environment.

[0136] At block 1706, the participant may enter a waiting room. In the waiting room, the participant may contact the AIVA and configure the level of interaction and assistance requested by the AIVA. The participant may configure, test, and inspect certain aspects of the participant's system. The participant may be prompted to perform system configuration and may receive configuration information from the sponsoring entity. Profile information, subscriptions, credits, and other configuration capabilities, such as appearance, personality, and appearance attribute metadata, may be loaded or determined from other VR systems.

[0137] At block 1708, the participant may determine that the participant system is not ready for operation, and the participant may return to block 1704 to repeat the configuration process. When the participant determines at block 1708 that the participant system is properly or sufficiently configured, then at block 1710, the participant may configure displayable controls and status fields that are presented in the participant's field of view within one or more places of the VR / AR / XR session.

[0138] At block 1712, the participant may join the VR / AR / XR session and participate in the purpose of the VR / AR / XR session. At block 1714, the participant may determine that all tasks, goals, and / or objectives of the VR / AR / XR session have not been completed and may continue to participate in the VR / AR / XR session at block 1712. At block 1716, when the objectives have been achieved, the leader may exit the VR / AR / XR session. Performance Evaluation in VR, AR, and XR Systems

[0139] VR, AR, and XR systems that may be adapted according to certain aspects disclosed herein provide multi-player environments that may span multiple real-world scenarios. Multi-player environments may be used for education, simulation-based training games, and other purposes. In some applications, participants may enter a multi-player environment through an initial virtual scenario and traverse links to one or more virtual scenarios. A multi-player environment may bridge two or more virtual scenarios that are inherently incompatible with one another. A multi-player environment may provide an interface between participant systems and virtual scenarios, regardless of the protocols and standards governing the operation of participant systems and virtual scenarios. In one example, a multi-player environment enables a group experience led by a docent, lecturer, guide, escort, or other leader, alone or in combination with an AIVA.

[0140] According to certain aspects of the present disclosure, performance measurements can be used to characterize activities and behaviors in a multi-player environment. Performance measurements can be used to develop, modify, optimize, and / or evaluate applications that can be executed in a multi-player environment and / or materials presented in a multi-player environment. Participants and leaders can be evaluated, ranked, and / or otherwise assessed based on performance measurements obtained in a multi-player environment.

[0141] In one aspect, the managed system architecture provided according to certain aspects disclosed herein can be configured to collect performance data in virtual, augmented, and / or extended reality systems across multiple reality scenarios. The performance data can be associated with leaders and participants and can characterize individual performance against a baseline or peer profile. In one example, the performance data can be stored in a public XR manager 716 (see Figure 7 ) is collected in a common communication layer 704 of the VR / AR / XR system, where the actions, activities, relationships, gestures and other aspects of the managed VR / AR / XR system have been characterized according to a common protocol or format. Figure 12 ) messages 1232, 1234 or direct messages 1236 transmitted between participant systems 1202, 1206 may include content, control, meta, positioning, perspective, focus, gesture and / or notification information between these participants or a group of participants.

[0142] Figure 18A managed VR / AR / XR system 1800 is shown from which performance data can be collected, aggregated, and analyzed. The managed VR / AR / XR system 1800 spans four virtual reality contexts 1802, 1804, 1806, 1808, which may have been developed using different and / or inconsistent programming interfaces or protocols, wherein each virtual reality context 1802, 1804, 1806, 1808 is coupled to the managed VR / AR / XR system 1800 via an adapter configured to convert the inputs, outputs, definitions, descriptors, and behaviors of the respective virtual reality context into a common set of inputs, outputs, definitions, descriptors, and behaviors. In one example, the adapter may include Figure 7 A processing layer 702, a common communication layer 704, an interface layer 706, and / or one or more participant contexts 714 are shown, or cooperate with them.

[0143] Although shown in two dimensions, the managed VR / AR / XR system 1800 can provide a three-dimensional experience. In one example, the managed VR / AR / XR system 1800 provides a multi-player virtual educational environment in which a leader can guide a group of participants through a lesson plan. In another example, the managed VR / AR / XR system 1800 provides a multiplayer online role-playing game. In another example, the managed VR / AR / XR system 1800 provides a flight training environment in which an air traffic controller or control function guides one or more pilots through a flight plan.

[0144] The managed VR / AR / XR system 1800 includes a set of features to be accessed, including waypoints 1814, 1816a, 1818a, 1820a, 1824a, 1824b, 1824c, 1826a, 1826b, 1828a, 1828b, 1828c. In some applications, entry points 1810 and exit points 1830 may be defined in one or more virtual reality environments 1802, 1804, 1806, 1808. In other applications, unrestricted entry and exit from the virtual reality environments 1802, 1804, 1806, 1808 may be allowed. In some implementations, transitions between virtual reality environments 1802, 1804, 1806, 1808 may be accomplished via links or portals 1822a, 1822b, 1822c. In one example, transitions between virtual reality scenarios 1802, 1804, 1806, 1808 are accomplished through a common portal 1822b.

[0145] In a virtual education environment, these features could be points of interest, subjects covered in the lesson plan, assembly points, challenges, rest stops, layover / catch-up points, or waypoints that serve as distribution points or information sources identified in the lesson plan. In a flight training environment, features could include simulated air traffic control waypoints.

[0146] Certain landmarks 1816a, 1818a, 1820a, 1824b, 1824c, 1826b, 1828b, 1828c may be viewed or accessed from a defined or desired perspective. In the example shown, landmarks 1816a, 1818a, 1820a in the first reality scenario 1802 may be viewed or accessed through an attendance area 1816b, 1818b, 1820b defined by the application.

[0147] A lesson plan, game storyline, flight plan, or other plan defined by an application may define a preferred, optimal, or prescribed path 1840 to be followed within the managed VR / AR / XR system 1800. The path 1840 may be initially defined by a designer and / or may be adjusted or optimized based on observed or measured behavior of participants as their avatars follow the path 1840. Adjustments and optimizations may be made based on performance measurements that characterize how well the participants adhered to the plan, including how well the participants' avatars followed the path 1840.

[0148] Figure 19 Shown with Figure 18 A first engagement example 1900 is shown, relating to spatial aspects of traversal of a first reality scenario 1802 in a real-world scenario. First engagement example 1900 involves a single participant traversing the first reality scenario 1802. The participant's performance can be measured, at least in part, by comparing the participant's performance to a baseline performance. In one example, the baseline performance can be based on the performance of peers or predecessors and can include adherence to a preferred, optimal, or prescribed path. The preferred, optimal, or prescribed path can include a segment 1902 leading from an entry point 1810 to a first waypoint 1814 serving as a direction or rendezvous point. In one example, the rendezvous point can be defined as allowing the participant to receive initial instructions. In this example, the rendezvous point can have no restrictions on the participant's proximity or viewing angle. A participant entering through entry point 1810 can proceed along a path including a first segment 1922 leading to a first point 1912. Before, during, or after delivering instructions, the participant can stop and / or move along a second segment 1924 to a second point 1914. The system may infer that the lag relative to baseline performance may be attributable to the participant focusing their attention on the object of interest at the second point 1914, which may be inferred from the dwell at the second point 1914 and / or from other sensors, activity, or inactivity during the dwell.

[0149] The preferred, optimal, or prescribed path includes second segment 1904 leading to second waypoint 1816a, which has a required attendance area 1816b. In the example shown, the participant proceeds via third indirect path segment 1926 to point 1916, which is located near but outside of required attendance area 1816b associated with second waypoint 1816a. The preferred, optimal, or prescribed path includes third segment 1906 leading to third waypoint 1818a, which has a required attendance area 1818b. In the example shown, the participant proceeds via fourth path segment 1928 (an indirect segment) to point 1918 within required attendance area 1818b associated with third waypoint 1818a. The preferred, optimal, or prescribed path includes fourth segment 1908 leading to fourth waypoint 1820a, which has a required attendance area 1820b. In the example shown, the participant proceeds to point 1920, which is located within the required attendance area 1820b associated with the third waypoint 1820a, via fifth path segment 1930. The preferred, optimal, or prescribed path includes a transition segment 1910 to the next reality scenario through portal 1822a, and the participant follows the preferred, optimal, or prescribed path along segment 1932 to the next reality scenario.

[0150] Participation example 1900 illustrates a single participant's two-dimensional deviation from a preferred, optimal, or prescribed path. These deviations can be characterized using a variety of measurements. In one example, adherence to the plan can be measured by path length, and the maximum or average distance from the preferred, optimal, or prescribed path can quantify certain aspects of the participant's performance. In another example, certain aspects of performance can be measured by the maximum or average linear and / or angular separation from the required attendance area. In some implementations, a single participant's three-dimensional deviation from the preferred, optimal, or prescribed path can be measured. Linear and angular separation can be measured in two or three dimensions.

[0151] Figure 20 Shows the Figure 18 1802. The second engagement example 2000 involves a single participant traversing the first reality situation 1802, wherein the participant's performance can be measured by comparing the participant's performance to a baseline performance and concurrent activities of other participants or leaders in the same first reality situation 1802 and / or other participants or leaders following the same lesson plan, game storyline, flight plan, or other plan defined by the application. In this example, the baseline performance can be adjusted based on a comparison with peers or predecessors' adherence to a preferred, optimal, or prescribed path.

[0152] Figure 20 A heat map 2020 is included that illustrates the aggregated or average paths associated with a group of common participants. In one example, the heat map 2020 may represent the boundaries of the paths followed by a certain percentage of participants, where the percentage may be defined by a distribution curve based on each participant's average distance from the average path. In another example, another type of heat map may include each participant's path, where darker areas represent areas of more travel than lighter areas.

[0153] A preferred, optimal, or prescribed path may include a segment 2002 leading from an entry point 1810 to a first waypoint 1814 serving as a direction or meeting point. In one example, the meeting point may be defined to allow participants to receive initial instructions. In this example, the meeting point may not restrict the proximity or viewing angle of the participants. A target participant entering through the entry point 1810 may proceed along a path including a first segment 2012 leading to a first point 2022. In one example, the target participant may be drawn to a group of common participants gathered at the first point 2022. Participants may receive instructions from a leader or from a controller.

[0154] The preferred, optimal, or prescribed path includes a second segment 2004 leading to a second waypoint 1816a, which has a required attendance area 1816b. In the example shown, the target participant proceeds to a point outside the required attendance area 1816b associated with the second waypoint 1816a via a second indirect path segment 2014. The preferred, optimal, or prescribed path includes a third segment 2006 leading to a third waypoint 1818a, which has a required attendance area 1818b. In the example shown, the target participant proceeds to the required attendance area 1818b associated with the third waypoint 1818a via a third indirect path segment 2016. The preferred, optimal, or prescribed path includes a fourth segment 2008 leading to a fourth waypoint 1820a, which has a required attendance area 1820b. In the example shown, the target participant proceeds via the fourth path segment 2018 to a point within the required attendance area 1820b associated with the third waypoint 1820a. In some cases, the participant may not be using a device capable of zooming in on a specified object for viewing, and the participant's avatar may be moved to a position suitable for viewing. The position suitable for viewing may be removed from the preferred, optimal, or specified path, but the participant's attention may be considered to be focused on the target. In other cases, the participant may be using a device with greater zoom capabilities, and such a participant's avatar need not be placed at a point on the preferred, optimal, or specified path. According to certain aspects disclosed herein, the system is able to combine positioning information, perspective (perspective in 3D space), and the zoom level of the viewing device to accurately determine the object of the participant's attention.

[0155] Participation example 2000 illustrates a single target participant's two-dimensional deviation from a preferred, optimal, or prescribed path, and / or deviation from an average path followed by co-participants (e.g., the centerline of heat map 2020). These deviations can be characterized using various metrics. In one example, adherence to the plan can be measured by path length, and the maximum or average distance from the preferred, optimal, or prescribed path can quantify certain aspects of the participant's performance. In another example, certain aspects of performance can be measured by the maximum or average linear and / or angular separation from the desired attendance area or the center of heat map 2020. Linear and angular separation can be measured in two or three dimensions.

[0156] Multiple types of variables can be measured to quantify participant performance. For example, time measurements can define the adequacy of the time spent by a participant, as well as the elapsed time and pace associated with a target participant's traversal of a managed VR / AR / XR system. In another example, sensor input received from a participant's device can be used to determine the quality of engagement as a function of attention at a landmark. Certain landmarks may include tasks, challenges, or tests that can be used to assess the quality of engagement. Each of these variables can be measured as an absolute amount or relative to a variable associated with a baseline, aggregated, or peer performance. In some implementations, the performance of each participant can be aggregated and / or integrated into a baseline metric. In some implementations, lesson plans, game storylines, flight plans, or other plans can be updated or optimized based on the measured performance of one or more participants or leaders.

[0157] Figure 21 An example 2100 of participation in the temporal aspects of traversing a portion of a managed VR / AR / XR system is shown. A first curve 2106 shows the path taken by a leader between waypoints 2104, while a second curve 2108 shows the paths taken by participants between waypoints 2104. The leader can navigate waypoints 2104 according to a defined or desired cadence. The cadence can define the time period 2101 between visits to each of a pair of waypoints 2104. For example, the leader can exhibit some variability in the time of arrival or departure from waypoints 2104 while maintaining a cadence by appearing at waypoints 2104 at specified time points. A visit can be defined as having both temporal and spatial components. In one example, the system can define a minimum duration for a visit at a waypoint 2104 to be recognized. In another example, the system can define a combination of positioning, viewing angle (perspective in 3D space), and zoom level of the viewing device that satisfies the participant focus requirement sufficient to recognize a visit at a waypoint 2104.

[0158] Participants may arrive at or depart landmark 2104 before or after the leader. In some examples, a participant's performance may be measured as a delay 2112 between the leader's and the participant's arrival at a point near landmark 2104. In the example shown, curves 2106 and 2108 include representations of the distance to landmark 2104. In some examples, a participant's performance may be measured as an angle or linear separation 2114 from landmark 2104. In some cases, a participant's performance may be measured based on the angle or linear separation 2114 from the leader at landmark 2104. The leader may stay at a landmark for a first time period 2116, while a participant stays at the same landmark for a second time period 2118. In some examples, a participant's performance may be measured as the difference between first time period 2116 and second time period 2118, and / or as the overlap between first time period 2116 and second time period 2118.

[0159] In some implementations, a participant may not participate in the leader's stops at waypoints 2104. In some examples, a participant's performance may be measured as the number of missed waypoints 2104, or the number of additional stops at locations or waypoints 2104 that differ from the leader's stops at locations or waypoints 2104. In some examples, the leader may make optional stops 2120 that may be included or excluded from the performance measurement. Adherence to a cadence may be a measure of the leader's performance, and additional stops 2120 or missed stops may affect the cadence.

[0160] Cadence may be represented as a frequency and / or may be analyzed using digital signal processing techniques to enable real-time performance assessment. For example, a traversal of a managed VR / AR / XR system may be characterized using a periodic function, such as a Fourier series that allows for rapid comparison of each traversal of the managed VR / AR / XR system to an optimal traversal of the managed VR / AR / XR system. Other signal processing techniques may be employed to measure performance in a managed VR / AR / XR system. For example, phase and amplitude relationships may be used to define stops at waypoints 2104 and distances from waypoints or leaders. In some cases, large amounts of performance data may be processed and assimilated in real-time, enabling artificial intelligence capabilities to quickly detect discrepancies and issues and provide assistance and prompts to participants and leaders.

[0161] According to certain aspects of the present disclosure, performance measures used to characterize engagement in activities and behaviors in a multi-player environment can be generated based on participant focus. Participant focus can be determined based on information received from one or more sensors in devices worn by participants in a session conducted on a managed reality system. This information can be generated by the sensors and can indicate the participant's gaze point or viewing direction. In one example, a rear-facing camera mounted on or within a head-mounted device or other wearable device can provide images that can be processed to determine gaze point based on knowledge of the participant's eyes and images displayed to the participant within or related to the multi-player environment. In another example, eye-tracking sensors mounted on or within the head-mounted device or other wearable device can be configured to directly report the participant's gaze point. Participant focus can also be determined based on information reported by one or more controllers managing the multi-player environment regarding the positioning of a participant's avatar within the multi-player environment. Participant focus can also be determined based on information received from a controller or local system indicating a participant's activity unrelated to the multi-player environment. In some cases, a controller or local system can instruct a participant to be inactive when activity is expected.

[0162] Figure 22 An example of a participation report that can be generated for a session in a multi-participant environment is shown. The participation report includes a list of session configuration information 2200. The session configuration information 2200 can set certain global parameters and constraints to be applied during the session. In the example shown, certain guidelines are defined that determine the expected virtual distance of participants from the leader of the session and objects of interest identified by the leader or a session plan, learning plan, etc.

[0163] The participation report includes a list of object configuration information 2210. Object configuration information 2210 may identify certain controls available to the leader when the leader focuses on an object, and / or certain controls available to participants when the participants focus on an object. Object configuration information 2210 may configure certain parameters that define the intended relationship between the leader and / or participants, including parameters and constraints that override corresponding global parameters and constraints set for the session. In some cases, the leader may redefine certain parameters and constraints set for the session or object.

[0164] The engagement report includes participant performance information 2220. In the example shown, the performance of M participants is presented for each of the N objects visited in the session. In some cases, performance information 2220 may include one or more objects that can be used to record a participant's performance (including focus) relative to a leader. Performance information 2220 may include information reporting the participant's total dwell time at each object, as well as their virtual distance from the object and from the leader. This information may be expressed as an average, mean, or statistically correlated representation of the distance corresponding to the dwell time. Performance information 2220 may also include information indicating the activity level for each object. Activity levels may be reported on a scale (here, 0-10), whereby a participant's activity relative to an object of interest may be recorded on a scale of 1-10, with complete inactivity or disengagement reported as a value of zero. The performance information 2220 shown includes participants 2222 with consistently high activity levels, participants 2224 with fluctuating activity levels, and participants 2226 with activity levels indicating a lack of focus.

[0165] Leaders can view configuration, participation, and control information during a session. This information is available in Figure 22 In some implementations, configuration, participation, and control information can be overlaid on the image presented to the leader, such as Figure 26 In some implementations, configuration, participation, and control information can be presented graphically during or after a session. Figure 23 One example of a graphical representation of engagement information 2300 presented in accordance with certain aspects of the present disclosure is shown.

[0166] The graphical representation of engagement information 2300 plots the focus of multiple participants 2304 over time 2302. A leader 2306 can be used as a baseline for determining the focus and engagement of other participants 2304. In the example shown, leader 2306 traverses the conversation between four objects 2312, 2314, 2316, and 2318, with each of the other participants 2304 expected to follow. The location and duration of each participant's dwell are represented by lines, such as lines 23101-23106. Certain lines 23101 and 23104 indicate brief dwell times, while other lines, including each of lines 23101-23106, indicate random focus on objects 2312, 2314, 2316, or 2318. Lines 23101-23106 can be color-coded to indicate gaze points and / or random focus.

[0167] Figure 24Certain other parameters that may be helpful in evaluating performance within a managed VR / AR / XR system are shown. These parameters may be measured and processed so that status can be observed and reported to a leader or system manager in real time. Parameters may be measured for each participant, a group of participants, and / or for all traversals or uses of a managed VR / AR / XR system. These parameters may be quantized, weighted, combined, and / or aggregated to obtain one or more metrics that may be provided to a leader or system manager in real time. In one example, parameters representing attention may be combined with measures of spatial and temporal performance (see Figure 19-20 ) to evaluate participants’ performance in real time.

[0168] Various performance metrics can be displayed to the leader in real time. A first graphical display element (e.g., any object type UUID) can be generated, displayed, or invoked to represent a transition activity that changes a participant's focus of attention 2400 from a first landmark (x) to a second landmark (y). According to certain aspects disclosed herein, the system can combine positioning information, perspective (perspective in 3D space), and the zoom level of the viewing device to accurately determine the object of a participant's attention. A delay between changes in focus of attention can be displayed for each of a group of participants 2402. Changes in focus of attention can be determined based on motion sensor data, data received from head, face, or eye tracking sensors, display information, zoom settings, and inactivity indicators received from participant devices. In some cases, data provided by head, face, or eye tracking sensors, in addition to display information, zoom settings, and inactivity indicators received from participant devices, can be used to determine the current focus of attention or to signal that there has been no change in focus of attention. A second graphical display element can be generated to represent each participant's distance 2410 in addition to each participant's transition activity. Distance information can be obtained from motion, position, and orientation sensor data (including zoom settings) received from the participant devices. A third graphical display element may be generated to represent each participant's attention level in addition to each participant's transition activity 2420. As the participant performs other tasks on the participant's device, the attention level may decrease as indicated by the participant's device when no motion is detected or the participant does not provide input or respond for a period of time.

[0169] One or more participants may be engaged in an activity supported by the application, but this may indicate certain issues with the participant's performance, including access to system-generated information. For example, a problem may be indicated when a participant engages in excessive in-application conversations, AIVA interactions, references to the guide, and / or playback streams or subtitles. A fourth graphical display element may be generated to represent chat activity 2430 between two or more participants and the AIVA, as indicated by the participant device and / or system device. A fifth graphical display element may be generated to identify interactions 2440 between a participant and the viewing guide, as indicated by the participant device and / or system device. A sixth graphical display element may be generated to identify interactions 2450 between a participant and stream or subtitle feedback.

[0170] Figure 25 An example of a data flow 2500 is shown relating to parameters that may facilitate performance evaluation within a managed VR / AR / XR system (see Figure 24 ). The database 2502 can be configured to receive information related to changes in focus 2400, the distance of each participant 2410, the attention level of each participant 2420, chat activity 2430 between two or more participants and the AIVA, interactions 2440 between participants and the viewing guide, and interactions 2450 between participants and stream or caption feedback. Other types of information 2504 can be provided to the database, including chats, stream data and other data captured by the system, scores calculated by one or more algorithms for processing performance data, and observations made by the leader. In one example, the leader can verbally or manually annotate the activity.

[0171] The information maintained by database 2502 can be managed and / or processed by one or more applications that generate performance measurements for participants. In some implementations, the performance measurements can be used to determine or predict future persistence and retention rates. In one example, the information maintained by database 2502 can be processed to generate reports and other analyses 2506. In another example, the information maintained by database 2502 can be analyzed to provide status information 2508, which can be provided to a stream received by the leader and / or one or more participants. Reports and status information can be provided in various fields of the scene presented to the leader or participants, including focus, status, and controls.

[0172] Figure 26 An example of an experiential teaching scenario 2600 according to certain aspects disclosed herein is shown, which can be implemented and / or managed by the public XR manager 900. In one example, the experiential teaching scenario 2600 is implemented and / or managed by the public XR manager 900. Figure 1The first adapter 114, 116, 118 is shown as being part of a group led by a lecturer, instructor 2602, guide, escort, or other leader within a VR timeline, reality scenario, and / or reality type accessed by the first adapter 114, 116, 118. In one example, the group leader can be considered instructor 2602, and VR timeline 2628 is managed by instructor 2602. Experiential teaching scene 2600 can represent a time slice that instructor 2602 can stop and / or navigate back or forward in time as needed using VR timeline 2628. In some cases, manipulating VR timeline 2628 can affect the VR timeline of the entire group. Instructor 2602 can control the presented material, including one or more objects 2604, 2606 that can be dynamically added by instructor 2602. Objects 2604, 2606 can be added to a background constructed for instructional purposes or to a virtual or augmented reality provided by one or more VR, AR, and / or XR systems. The objects 2604, 2606 may also contain embedded links and metadata that allow the group or members of the group (if so permitted or selected) to further inspect the object and / or provide more details about the object itself. This may include more information such as: a) part of the lesson plan, b) tips on what to do, c) How to obtain that specific object as an AR or VR object, d) How to obtain that specific object in the real world, and / or e) Testing.

[0173] The presence and participation level of students or other participants 2612, 2614, 2616, 2618 can be controlled by the instructor 2602. Participants 2612, 2614, 2616, 2618 can join the experiential teaching scene 2600 through different types of VR, AR, and / or XR systems, or through services provided by the common XR manager 900. Participation may require an invitation, subscription, and / or consent from the instructor 2602. Some participants 2612, 2614, 2616, 2618 can interact with the instructor 2602 and / or objects 2604, 2606 within the experiential teaching scene 2600. Some participants 2612, 2614, 2616, 2618 can be observers or auditors of the instructions provided through the experiential teaching scene 2600.

[0174] The instructor 2602 can access information about the participants 2612, 2614, 2616, 2618. For example, status information included in the control mechanism 2622 can list the participants 2612, 2614, 2616, 2618 participating in or reviewing the instruction. The status information can identify the participant's level of participation in the instruction, which can be provided by the corresponding participant context. In some implementations, the participant context can report the activities of the participants, and the public XR manager 900 can associate such activities with the experiential teaching scene 2600 and / or objects 2604, 2606 in the experiential teaching scene 2600. The instructor 2602 can, for example, warn about distractions, looking in the wrong direction (e.g., see Figure 23 23101 and 23102 in the ) and / or engaging in activities unrelated to the Instructions (e.g., see Figure 22 Participants 2612, 2614, 2616, 2618 of the group 2602 (participants 2226 in the group 2602). The mentor 2602 can send messages or speak directly to one or more participants through the private conversation as needed.

[0175] The instructor 2602 can dynamically control certain aspects of the experiential teaching scenario 2600. Control mechanisms 2622, 2624, 2626, 2628, and 2630 can be manipulated by the instructor 2602 within the experiential teaching scenario 2600. Certain control mechanisms 2624 can be related to the management of participants 2612, 2614, 2616, and 2618 and objects 2604 and 2606 within the experiential teaching scenario 2600. Certain control mechanisms 2626 can be related to the control of activities associated with the participants 2612, 2614, 2616, and 2618 and objects 2604 and 2606 within the experiential teaching scenario 2600. Certain control mechanisms 2628 and 2630 can be related to the management of the experiential teaching scenario 2600 itself and the timeline in which the experiential teaching scenario 2600 occurs. Some control mechanisms 2622, 2624, 2626, 2628, 2630 are used to generate, display, or call up objects in the contextual reality. The objects may appear at a specific location, or at a location specified by the leader by reference to a UUID or similar. Once all participants 2612, 2614, 2616, 2618 have viewed the object, some control mechanisms 2622, 2624, 2626, 2628, 2630 may be used to cause the participants 2612, 2614, 2616, 2618 to gather at that location, or to cause the participants 2612, 2614, 2616, 2618 to look at the object. Some control mechanisms 2622, 2624, 2626, 2628, 2630 may be used to hide one, any number, or all of the participants 2612, 2614, 2616, 2618 from one another so that they do not obstruct each other's view of the object. This may include hiding the leader. Some control mechanisms 2622, 2624, 2626, 2628, 2630 can also be used to indicate, for example, which objects the leader is discussing or wants the participants 2612, 2614, 2616, 2618 to look at. As previously mentioned, all actions performed in real time are assigned timestamps to indicate when they were completed and what experience was generated (see, for example, Figure 18-23 ).

[0176] Each control mechanism 2622, 2624, 2626, 2628, 2630 can have the potential to be an independent logical channel to the common XR manager 900. The common XR manager 900 can overlay or integrate input from the control mechanisms 2622, 2624, 2626, 2628, 2630 onto the VR or AR content provided in the real world. The common XR manager 900 can process and interpret the input from the control mechanisms 2622, 2624, 2626, 2628, 2630 and can generate requests and commands that can be transmitted over the logical connection, and in some cases, can establish additional logical connections in response to the input. In some cases, the input from the control mechanisms 2622, 2624, 2626, 2628, 2630 triggers certain pre-configured actions.

[0177] In one example, the control mechanism 2622 can alert the instructor 2602 that a student has left the group or is looking at something unrelated to the group or group activity. An automatic warning can be configured to alert the student that the student has been marked as non-participating. In some implementations, the control mechanism 2624 can be dialed to allow the removal of a student or object. In some implementations, the control mechanism 2624 can be dialed to summon a student or object, where summoning a student or object can have different effects. In one example, a student can be summoned and can optionally proceed to a gathering point indicated by the instructor 2602. In another example, when summoned by the instructor 2602, the student or object can automatically move to the gathering point. Based on the subscription, activity type, and policies defined by the VR / AR / XR experience, participants 2614 other than the leader can participate at one or more levels via the participant controls 2632 (for more details see Figure 28 ).

[0178] According to certain aspects disclosed herein, the public XR manager 900 can be operated by a third party. In one example, a sponsor or other content provider can provide product information related to the instruction, including video, audio, images, and / or 3D representations of objects. Information identifying the types of products to be discussed or highlighted in the experiential teaching scene 2600 can be provided to the content provider. The information provided to the content provider can identify aspects such as the display location, viewing angle, and size of the products to be displayed. The public XR manager 900 can configure a logical connection between the experiential teaching scene 2600 and the content provider, which allows the content provider to install images or 3D representations of products in the experiential teaching scene 2600. In some cases, the logical connection between the experiential teaching scene 2600 and the content provider is bidirectional and allows interaction between the content provider and the instructor 2602 and / or participants 2612, 2614, 2616, 2618. In one example, the instructor 2602 and / or participants 2612, 2614, 2616, 2618 can browse for products similar to the displayed product. In another example, the instructor 2602 and / or participants 2612, 2614, 2616, 2618 may choose to access real-life scenarios maintained by a content provider.

[0179] According to certain aspects disclosed herein, the public XR manager 900 can be configured to broker access rights, associated costs, and display transitions for objects within a shared environment (such as an experiential instructional scenario 2600), thereby allowing objects existing in one reality context 106 to be instantiated across two or more reality contexts 106 while preserving intellectual property rights and enforcing restrictions required by licenses or regulations. In some implementations, temporary access can be obtained on-site within a VR, AR, and / or XR experience. For example, access rights can be obtained, updated, or invoked based on a subscription or point-of-sale interface without leaving the current VR, AR, and / or XR context, space, or experience. Participant rights, object information, display requirements, and / or external context information can be maintained as metadata attached to participants 2612, 2614, 2616, 2618 and objects within the XR manager 900 system.

[0180] In some implementations, a "floating" object can overlay VR / AR content. Floating objects can be actionable. For example, a floating object might include a teaching management object that can float on a student or leader's screen. In some cases, instructional links can be provided as needed, including links that could be classified as advertising placement.

[0181] In some implementations, the "embedded" object can maintain its position relative to certain objects located within the content itself. In one example, an identified type of AR-based embedded object can be defined, where a radiator, chair, or staircase can be identified for instructional purposes. In some cases, the leader can define the embedded object dynamically or before the instruction is initiated. The action on the object can be to store the entire object by the participant 2612, 2614, 2616, 2618 for a subsequent more complete review. In some cases, the embedded object can be enhanced or more fully used in subsequent viewing within the participant's XR environment.

[0182] In some implementations, "highlight" objects can be used to enhance participant presence. In one example, a participant avatar can be adorned with wearable content, a wig, and / or tattoos. The avatar's character can be selected so that it can be recognized by other participants 2612, 2614, 2616, 2618 and stored for later use. Highlight objects can include objects that have been purchased or obtained through licensing. Highlight objects can enable certain identification or delineation features in VR, AR, and / or XR systems.

[0183] According to certain aspects disclosed herein, participants 2612, 2614, 2616, 2618 in a VR / AR / XR experience may include one or more leaders, subscribers, and / or invitees. The VR / AR / XR experience may be provided within a venue that includes one or more different VR reality scenarios and / or environments, which may be combined or used individually for events involving a single participant or a group of multiple participants. In some examples, the VR / AR / XR experience may be constructed as a tour, training, education, or other event or for other purposes.

[0184] Participants 2612, 2614, 2616, and 2618 in a group activity may include one or more leaders, who may control or manage the VR / AR / XR session and act as an instructor, guide, or mentor 2602 for the VR / AR / XR session. The leader may invoke, configure, and / or control an artificial intelligence virtual assistant (AIVA), which may be configured to monitor and respond to participant requests, actions, and needs. The leader may delegate responsibility and control to the AIVA for the VR / AR / XR session, portions of the VR / AR / XR session, and / or the VR / AR / XR experience. The leader and the AIVA may appear as a hybrid leader (referred to herein as a "Maestro") within the VR / AR / XR experience, where the relative contributions of the AIVA and leader are configurable by the leader. In one example, the AIVA may be assigned to monitor the participation levels of individual participants 2612, 2614, 2616, 2618, respond to inquiries from participants 2612, 2614, 2616, 2618, and provide playback of previous activities, guidance within the VR / AR / XR experience, and feedback to the leader regarding the participation. In another example, the AIVA may be assigned to present the core material of the VR / AR / XR experience or otherwise guide the VR / AR / XR experience.

[0185] Participants other than the leader can participate in group activities at one or more levels, depending on the subscription, activity type, and policies defined by the VR / AR / XR experience. Policies can be defined as courses of action or principles adopted or proposed by governments, institutions, businesses, or individuals. Policies can be applied to individual or group situations, including leader and institutional requirements, in real-world situations or VR / AR / XR experiences. For example, participants 2612, 2614, 2616, and 2618 in group activities include individual participants (attendees) in an event session, individual participants (trainees) in a training session, and individual participants (tourists) in a tour.

[0186] A leader can initiate a VR / AR / XR experience through preconfiguration. Preconfiguration can be used to configure one or more events, venues associated with the event, and identify participants 2612, 2614, 2616, 2618 in the VR / AR / XR experience. The leader can configure the experience, access, and rights of individual participants. In some cases, participants 2612, 2614, 2616, 2618 can be assigned obligations and / or tasks to perform in the VR / AR / XR experience. The leader can configure AIVA participation levels during preconfiguration. The leader can also configure time limits during preconfiguration. In one example, the leader can configure a start time, an end time, a minimum duration for the VR / AR / XR experience, and / or a maximum duration for the VR / AR / XR experience. The leader can also configure goals for one or more participants 2612, 2614, 2616, 2618 during preconfiguration, as well as metrics for judging performance against those goals.

[0187] Figure 27 An example of a scene 2700 provided to a presenter, instructor, guide, escort, or other leader within a VR timeline, reality scenario, and / or reality type is shown. Various elements are superimposed on the scene 2700. General status information may include a timing status 2702 indicating that the leader is adhering to the presentation schedule. Participant information 2704 may identify active, unfocused, and / or inactive participants. An information panel 2706 may be provided as a lesson plan, guide, or map for the leader. Subtitles 2708 may be displayed as a transcription of what the leader said, a prompt regarding current landmarks, or a summary of the information disclosed. In the example shown, elements of the transcribed material are highlighted. In some implementations, the highlighting may be automatically generated by the system by the leader directing or focusing their attention to specific elements identified in the situational reality. Other mechanisms may be used for self-identification, or for the leader to help identify elements that require attention, such as those detailed in the lesson plan.

[0188] System status information 2710 and an access point to the AIVA 2712 may be provided. A participant presence field 2714 may be displayed to allow the leader to identify and communicate with participants. In the example shown, participant-5 2724 is marked in participant presence field 2714 with a stored message to be sent to participant-5. This stored message to the participant may be automatically generated and may be initiated when the leader directs their attention to the corresponding participant presence field element for participant-5 or some other combination of actions. The leader may communicate privately with one or more participants via chat window 2716. The leader may activate a recorder to record notes and observations via icon 2718 and / or may provide text notes and observations or select checkboxes or predefined text scripts via user interface 2720.

[0189] Figure 28 An example of a scene 2800 presented to participants within a VR timeline, reality scenario, and / or reality type is shown. Various elements are superimposed on scene 2800. General status information 2802 may provide timed status and search capabilities. A help group 2806 may be provided, allowing participants to seek online help, including from AIVA, and / or flag a need for help to the leader. Subtitles 2804 may be displayed as a summary of the information disclosed regarding the current waypoint. In the example shown, subtitles 2804 display a transcript of the information disclosed by the leader regarding the current waypoint. Participants can privately communicate with the leader or one or more participants via chat window 2810. Participants can record the conversation and / or access playback via multimedia icon 2812. In some scenarios, the leader may be represented by an avatar 2814. In AR scenarios, an image of the leader may be displayed. Other participants may be represented by corresponding avatars 2816 or other entities configured for the participant viewer. In some cases, certain participants may not be represented in the scene. Participant representation may be determined by the organization, the leader, or the individual preferences of the participant. An organization, leader, or participant can determine that a participant's representation is not displayed.

[0190] Figure 29 Scene 2900 is shown, which can be constructed to include or interact with multiple virtual reality scenarios. In one example, scene 2900 can be used to initiate a classroom event. In some implementations, scene 2900 can be constructed in an augmented reality scenario superimposed on a physical conference room or classroom. Audiovisual and other equipment present in the physical conference room or classroom can be integrated into the scene. In some cases, display system 2902 can be provided via the augmented reality scenario. Materials can be presented to the group via display system 2902. In another example, scene 2900 can be constructed in a virtual reality scenario that can be designed to resemble or function as a conference room or classroom. Materials can be presented to the group via display system 2902, where display system 2902 is provided via the virtual reality scenario.

[0191] A group of participants can gather in scene 2900. Participants can be summoned by a leader (e.g., see Figure 26 2900 ). The user can also enter scene 2900 through their own actions (using control mechanisms 2624 in the example). Scene 2900 can be linked to other AR, VR, or XR scenes and / or elements. In one example, a presentation displayed on display system 2902 can include elements represented by virtualized objects, which can be introduced into scene 2900 in various ways. In one example, a presentation can include an image of object 2910 that is the subject of instruction or discussion.

[0192] The public XR manager can manage the relationship between the presentation and one or more virtualized objects. For example, these relationships can be defined by a script or lesson plan. In one example, the leader can bring one or more virtualized objects 2906 into the scene 2900 while discussing the relevant images of the object 2910 displayed on the display system 2902. In another example, the leader can cause a portal 2904 to be opened, and the group can transition to or from different real-life situations through the portal 2904. In the example shown, the portal 2904 leads to the virtual room depicted in the presentation displayed on the display system 2902, where the object 2910 that is the subject of instruction or discussion is present in the virtual room. The object 2910 can be represented in 3D in the virtual room. In some cases, the object 2910 can be viewed in a 360° view within the virtual space accessed through the portal. Editing process example

[0193] Figure 30 Included are diagrams 3000 and 3020, illustrating how an instructor, mentor 3002, guide 3006, trainer 3004, chaperone, or other leader can take a course outline, lesson plan 3010, or similar instructional materials, input them into the TCXR Editor 3022, and construct one or more real-world scenarios for later use. The TCXR Editor 3022 allows leaders to integrate their prepared course materials with real-world scenarios by creating various object UUIDs (as described above), allowing those identified objects to be displayed during the real-world VR experience. In this way, identified objects can be manipulated by the leader, seen by participants, and metrics derived based on views, groupings, discussions, or questions about the objects.

[0194] In certain implementations, the TCXR editor 3022 can construct a script that defines a four-dimensional session in a multi-participant environment. In one example, objects can be identified in one or more lesson plans 3010 and used to create scenes to be traversed in the session. The session configuration information 2200 and the object configuration information 2210 can be embedded in the lesson plan 3010 and / or provided separately. The timeline or pacing can be defined in the lesson plan 3010 and / or provided separately. The resulting realistic scenario can include a scene that unfolds according to a scene-specific timeline, with objects that become the focus of attention according to an object-specific timeline. The TCXR editor 3022 can define a session script that can be customized or modified by the leader and can be adapted to accommodate any number of participants who register to enter the experience.

[0195] In some implementations, the TCXR editor 3022 can define relative start and end times that can be mapped to the start and end times defined by the leader. In some cases, the timeline can be scaled to fit the start and end times defined by the leader.

[0196] Figure 31 An example of a system 3100 is shown in which an instructor, mentor, guide, coach, chaperone, or other leader can input an instructional course outline, lesson plan, or similar instructional material 3102 into a TCXR Contextual Reality Creator ("CRC") tool 3120 and construct one or more reality scenarios 3130 for later use.

[0197] The CRC tool 3120 is implemented by integrating an available 3D editor 3122 (such as UNITY ® The 3D Editor 3122 is used in conjunction with the TCXR CR Editor 3126 and the TCXR CR Visualizer 3124 to allow the lead or designated editor to integrate prepared course materials. The 3D Editor 3122 may include modules or circuitry 3132 that receive or provide instructional materials 3102 and / or modules or circuitry 3132 that assemble, edit, and / or merge instructional materials 3102 to obtain aggregated or combined input. The aggregated or combined input is provided to the TCXR CR Visualizer 3124, which may parse the instructional materials 3102 on a page-by-page, element-by-element, and / or object-by-object basis and, with input or control from the lead or designated editor, may identify not only significant objects, but also timelines, paths through various scenes and / or scenarios, locations, and expected participant activities. These will be referenced below. Figure 32 Detailed description is provided below. CRC tool 3120 utilizes the TCXR CR editor and / or UUID manager 3128 to retrieve, validate, and input UUIDs 3136, including the UUID 3136 corresponding to the intended owner of the output context reality (R-UUID). The types of UUIDs managed by UUID manager 3128 may include at least one or more institution UUIDs ("I-UUIDs"), one or more group (i.e., class, instruction, course) UUIDs ("G-UUIDs"), and / or one or more leader (i.e., instructor, mentor, guide, companion, motivator, leader) UUIDs ("L-UUIDs"). In some implementations, one or more context reality ("R-UUIDs") may be generated from a single I-UUID, or G-UUID, or L-UUID, or any combination of these or other UUIDs 3136, where the number of such UUIDs is greater than one.

[0198] The TCXR CR visualization tool 3124 can output the reality scenario in various formats. A leader or designated editor can view the reality scenario in one or more formats. For example, the original format can be used to provide a development visualization 3140, which includes one or more elements 3142, 3144 representing participant avatars, one or more objects 3146, 3148, and / or other features. The development visualization 3140 can help plan the initial placement of paths, transitions, gathering points, and control elements. In some implementations, the development visualization 3140 can help plan the variability associated with the placement of paths, transitions, gathering points, and control elements during an active session, where the leader can dynamically change pre-planned or pre-configured parameters.

[0199] Figure 32 Certain aspects associated with a system 3200 are shown, including a TCXR CR visualization tool 3206 provided in accordance with certain aspects of the present disclosure. The TCXR CR visualization tool 3206 may correspond to Figure 31 The TCXR CR visualizer 3124 is shown. Thus, the TCXR CR visualizer 3206 can output the reality scenario 3208, 3210 in various formats. The TCXR CR visualizer 3206 can allow the leader or designated editor to identify not only important objects, but also expected participant activities, timelines, pathways 3230, gathering points 3242, 3244, 3246, 3248, and other locations including entry points 3240 and portals 3236. The TCXR CR visualizer 3206 can provide the leader with the ability to identify objects in their course package 3202, such as tables 3232, seats 3234, homes, etc. These identified objects 3232, 3234 can be retrieved from a 2D or 3D object store 3204 (such as the Unity store), allowing the initiator to select objects to use in the reality scenario. A selected object, such as a 3D table 3232, can be assigned a target location within the real-world context, along with various controls 3222, 3224, 3226, and 3228. The object UUID ("Oo-UUID") and other metadata corresponding to the Oo-UUID can be provided to the TCXR CR visualizer 3206. In one example, the initiator can determine whether the time dial 3222 is displayed in response to a trigger or command from the session leader, if the object is visible for the entire duration of the session, or for a portion of the session. In another example, one or more objects 3232, 3234 may not be visible in the initial scene or context until called upon by the session leader. In another example, one or more objects 3232, 3234 may not be visible in the initial scene or context until called upon through portal 3236.

[0200] One or more object dials 3224 can be used to control certain features including the position, size, visibility, and other properties of one or more objects 3232, 3234. The object dials 3224 can have an appearance configured by the initiator, editor, and / or leader of the session. In one example, the leader of the session can configure one or more objects 3232, 3234 to be pop-up objects, associated with a gathering feature, and / or selected for a "view" feature. In another example, when the gathering or viewing feature is selected, the initiator can choose to provide and configure an invisible mode for participants, including the leader.

[0201] In another example, the initiator can assign a path 3230 (or multiple paths) within a real-world scenario. Path 3230 can be used to track participants' passage through the scene or scenario and / or deviations from path 3230 or a selected path. Participants can follow path 3230 when walking between objects 3232, 3234, or portals 3236. When the initiator sets these paths, they can assign alarms to be triggered based on the distance from the leader, path 3230, gathering points 3242, 3244, 3246, 3248, and / or objects 3232, 3234. Another ability for the initiator is to determine how the object behaves relative to time effects, which can be indicated or configured using a study dial 3228. The study dial 3228 provides controls that affect the time the object should initially be observed, the expected time of departure from the object, and a trigger 3256 that alerts the leader if the actual time is less than or greater than the configured or expected time.

[0202] In another example, the objects identified in the course package 3202 may correspond to homes, different scenes, or different reality scenarios that can be accessed through a portal 3236. The portal 3236 may be created for the purpose of transporting participants to a rendered 3D object of the home, different scenes, or different reality scenarios. In some cases, the home may be found or purchased through an object store or similar establishment.

[0203] Configurable controls 3222, 3224, 3226, 3228 may be initially set to default values ​​based on the originator's prior history. The values ​​associated with controls 3222, 3224, 3226, 3228 may be modified in real time by the leader, or even by the AIVA if permitted.

[0204] Other types of UUIDs may be used in real-world scenarios in conjunction with object UUIDs ("Oo-UUIDs"), such as presentation UUIDs (Op-UUIDs), theater UUIDs ("Ot-UUIDs"), portal UUIDs ("Or-UUIDs"), location UUIDs ("Ol-UUIDs"), and other types of UUIDs to identify or describe a temporal location ("CL-UUID"), a trigger point ("CT-UUID"), or a marker point ("CM-UUID").

[0205] These objects can be tagged with an identifier that indicates that the originator can use it in other contextual realities. In some cases, the public XR manager 900 can target not only that reality context, but also any other reality context that can be configured to use the same object (see also Figure 9 The object may have other characteristics in other real-world scenarios, depending on how the initiator determines the settings of, for example, the time dial 3222, the object dial 3224, the path object dial 3226, and the research dial 3228.

[0206] The initiator may also decide to include a visible AVIA in the real-world scenario. All object features are available on the AVIA, depending on the need or requirement. In some cases, object features may be disabled. Example of processing circuit

[0207] Figure 33 3300 is a diagram illustrating an example of a hardware implementation for an apparatus 3300. In some examples, apparatus 3300 can perform one or more functions disclosed herein. According to various aspects of the present disclosure, processing circuitry 3302 can be used to implement any element, portion of any element, or combination of any element disclosed herein. Processing circuitry 3302 can include one or more processors 3304 controlled by some combination of hardware and software modules. Examples of processors 3304 include microprocessors, microcontrollers, digital signal processors (DSPs), SoCs, ASICs, field programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, sequencers, gating logic, discrete hardware circuits, and other suitable hardware configured to perform the various functions described throughout this disclosure. One or more processors 3304 can include special-purpose processors that perform specific functions and can be configured, enhanced, or controlled by one of software modules 3316. One or more processors 3304 can be configured by a combination of software modules 3316 loaded during initialization and further configured by loading or unloading one or more software modules 3316 during operation.

[0208] In the example shown, processing circuit 3302 can be implemented using a bus architecture, which is generally represented by bus 3310. Bus 3310 can include any number of interconnecting buses and bridges, depending on the specific application and overall design constraints of processing circuit 3302. Bus 3310 links together various circuits including one or more processors 3304 and storage 3306. Storage 3306 can include memory devices and mass storage devices and may be referred to herein as computer-readable media and / or processor-readable media. Storage 3306 can include temporary storage media and / or non-temporary storage media.

[0209] The bus 3310 may also link various other circuits, such as timing sources, timers, peripheral devices, voltage regulators, and power management circuits. A bus interface 3308 may provide an interface between the bus 3310 and one or more transceivers 3312. Depending on the nature of the device 3300, a user interface 3318 (e.g., a keyboard, display, speaker, microphone, joystick) may also be provided and may be communicatively coupled to the bus 3310 directly or through the bus interface 3308.

[0210] The processor 3304 may be responsible for managing the bus 3310 and general processing, which may include executing software stored in computer-readable media including storage 3306. In this regard, the processing circuit 3302 including the processor 3304 may be used to implement any of the methods, functions, and techniques disclosed herein. The storage 3306 may be used to store data that is manipulated by the processor 3304 when executing the software, and the software may be configured to implement any of the methods disclosed herein.

[0211] One or more processors 3304 in processing circuit 3302 can execute software. Whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise, software should be broadly understood to include instructions, instruction sets, code, code segments, program code, programs, subroutines, software modules, applications, software applications, software packages, routines, subroutines, objects, executable programs, execution threads, procedures, functions, algorithms, and the like. The software may reside in computer-readable form in storage 3306 or in external computer-readable media. The external computer-readable media and / or storage 3306 may include non-transitory computer-readable media. For example, non-transitory computer-readable media include magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips), optical disks (e.g., compact disks (CDs) or digital versatile disks (DVDs)), smart cards, flash memory devices (e.g., "flash drives," cards, sticks, or key drives), RAM, ROM, programmable read-only memory (PROM), erasable PROM (EPROM) including EEPROM, registers, removable disks, and any other suitable medium for storing software and / or instructions that can be accessed and read by a computer. For example, computer-readable media and / or storage 3306 may also include carrier waves, transmission lines, and any other suitable medium for transmitting software and / or instructions that can be accessed and read by a computer. Computer-readable media and / or storage 3306 may reside in processing circuit 3302, in processor 3304, external to processing circuit 3302, or distributed across multiple entities including processing circuit 3302. Computer-readable media and / or storage 3306 may be embodied in a computer program product. For example, a computer program product may include the computer-readable medium in packaging materials. Those skilled in the art will recognize how best to implement the described functionality presented throughout this disclosure depending on the particular application and the overall design constraints imposed on the overall system.

[0212] Storage 3306 may maintain software maintained and / or organized in the form of loadable code segments, modules, applications, programs, etc., which may be referred to herein as software modules 3316. Each software module 3316 may include instructions and data that, when installed or loaded onto processing circuitry 3302 and executed by one or more processors 3304, facilitate a runtime image 3314 of controlling the operation of one or more processors 3304. When executed, certain instructions may cause processing circuitry 3302 to perform functions according to certain methods, algorithms, and processes described herein.

[0213] Some software modules 3316 may be loaded during initialization of the processing circuit 3302, and these software modules 3316 may configure the processing circuit 3302 to enable execution of the various functions disclosed herein. For example, some software modules 3316 may configure the internal devices and / or logic circuits 3322 of the processor 3304 and may manage access to external devices such as the transceiver 3312, the bus interface 3308, the user interface 3318, timers, a math coprocessor, and the like. The software modules 3316 may include a control program and / or an operating system that interacts with interrupt handlers and device drivers and controls access to various resources provided by the processing circuit 3302. Resources may include memory, processing time, access to the transceiver 3312, the user interface 3318, and the like.

[0214] One or more processors 3304 of the processing circuit 3302 can be multifunctional, whereby some software modules 3316 are loaded and configured to perform different functions or different instances of the same function. For example, one or more processors 3304 can be further adapted to manage background tasks initiated in response to input from the user interface 3318, the transceiver 3312, and the device driver. To support the execution of multiple functions, the one or more processors 3304 can be configured to provide a multitasking environment, whereby each of the multiple functions is implemented as a set of tasks serviced by the one or more processors 3304 as needed or desired. In one example, the multitasking environment can be implemented using a time-sharing program 3320 that transfers control of the processors 3304 between different tasks, whereby each task returns control of the one or more processors 3304 to the time-sharing program 3320 upon completion of any outstanding operations and / or in response to input such as an interrupt. When a task has control of one or more processors 3304, the processing circuit is effectively dedicated to the purpose addressed by the function associated with the control task. The time-sharing program 3320 may include an operating system, a main loop based on loop transfer control, a function that allocates control of one or more processors 3304 based on the priority of the function, and / or an interrupt-driven main loop that responds to external events by providing control of one or more processors 3304 to the processing function.

[0215] Figure 34is a flowchart 3400 illustrating a method for establishing a multi-domain, computer-generated reality. The method may be performed by the common XR manager 900 or another controller capable of managing or controlling a multi-device VR, AR, and / or XR system. In box 3402, the controller authenticates a user of a first device and a second device coupled to a controller that manages or controls the multi-domain, computer-generated reality. In box 3404, the controller receives a message from the first device, the message including a request to participate in a multi-domain application. In box 3406, the controller configures a first communication path between the controller and the second device. In box 3408, the controller configures a second communication path between the first device and the second device. In box 3410, the controller may send at least one message related to the multi-domain application via the first communication path.

[0216] In some implementations, the method may establish the presence of a second participant in a second reality situation in the computer-generated reality via a second logical connection while maintaining the presence of the second participant in the second reality situation. The method may include configuring a third logical connection between the computer-generated reality and the third reality situation, and integrating content provided by a content provider via the third logical connection into the computer-generated reality. The content may include video feedback, audio feedback, images, or 3D representations of objects. In some cases, the content includes access to another computer-generated reality maintained by the content provider.

[0217] In some examples, the method includes monitoring the activities of a first participant and a second participant in a computer-generated reality via respective logical connections, and communicating the status of the first participant and the second participant to a mentor or guide via a third logical connection established between the computer-generated reality and a fourth reality context associated with the mentor. The method may include receiving control information representing manipulation of one or more control mechanisms in the computer-generated reality, and modifying the computer-generated reality experienced by the first participant or the second participant in response to the control information. The control mechanism may be displayed in the fourth reality context and hidden in the first reality context.

[0218] In some examples, modifying the computer-generated reality includes terminating the presence of the first participant or the second participant. Modifying the computer-generated reality may include moving along a timeline of the computer-generated reality. Modifying the computer-generated reality may include adding or removing objects from the computer-generated reality.

[0219] In some implementations, the method includes importing a 3D representation of the object from a different computer-generated reality. Importing the 3D representation of the object can include obtaining permission from an owner of the 3D representation before importing and associating the 3D representation with the first participant.

[0220] In some implementations, establishing the presence of the first participant includes configuring haptic feedback between the first reality situation and the computer-generated reality. The computer-generated reality can include a simulation, or the computer-generated reality implements an experiential instructional reality.

[0221] Figure 35 3500 is a flowchart illustrating a method for managing a multi-domain, computer-generated reality. At block 3502, a manager may determine one or more differences between each activity of a first participant and a corresponding baseline activity for each of a plurality of activities associated with traversal of a managed reality system during a session. At block 3504, the manager may quantify the one or more differences to obtain a performance metric. At block 3506, the manager may combine at least one performance metric for each activity of the first participant to obtain a session performance measurement for the first participant.

[0222] In some implementations, the baseline activity is associated with the leader of the session. The baseline activity can be obtained from an aggregation of previous sessions. One or more differences can include a difference in the location of the first participant's avatar and the session leader's avatar. One or more differences can include a difference in the time the first participant's avatar arrives at a location and a corresponding time the session leader's avatar arrives at the location. One or more differences can include a difference in the time the first participant's avatar leaves a location and a corresponding time the session leader's avatar leaves the location. One or more differences can include a difference in the time the first participant's avatar spends at a location and a corresponding time the session leader's avatar spends at the location.

[0223] In some implementations, the method includes determining a focus parameter for the first participant based on input received from one or more sensors managed by a device operated by the first participant while participating in the session, and combining the focus parameter with at least one performance metric for each activity of the first participant when obtaining a session performance measurement for the first participant. The one or more sensors may include a motion sensor. The one or more sensors may include a position sensor. The one or more sensors may include an audio sensor.

[0224] In some implementations, determining the attention parameter includes monitoring chat activity of the first participant. Determining the attention parameter may include monitoring access to system information by the first participant. The computer-generated reality may include a simulation. The computer-generated reality may implement an experiential teaching reality.

[0225] Figure 363600 is a flowchart illustrating a method for operating a virtual reality system. The virtual reality system may be a multi-domain, computer-generated reality that can be operated using a VR / AR / XR manager. In block 3602, the manager may provide a first virtual or augmented reality situation including a first scene. In block 3604, the manager may gather one or more participants within the first scene. In block 3606, the manager may present an image of an object in the first virtual or augmented reality situation. In block 3608, the manager may provide a virtualized version of the object in the first virtual or augmented reality situation or in a second virtual or augmented reality situation.

[0226] In some cases, a virtualized version of an object may be provided by providing a three-dimensional representation of the object within a first scene. In some cases, a virtualized version of an object may be provided by providing a portal that provides a passage between a first virtual or augmented reality situation and a second virtual or augmented reality situation, and by providing a three-dimensional representation of an object within a second scene provided in the second virtual or augmented reality situation.

[0227] The manager can cause one or more participants to gather near a virtualized version of an object and suppress the visual representations of the multiple participants provided to a first participant within the virtual reality system, thereby placing the multiple participants into an invisible mode. When a second participant is separated from the first participant by a minimum distance configured for the virtual reality system, the manager can restore the visual representation of the second participant. The first virtual or augmented reality scenario can be generated by an editor that defines a plurality of scenes, one or more objects included in each scene, and a timeline for traversing the plurality of scenes. The editor can generate each scene from a lesson plan.

[0228] Figure 37Flowchart 3700 illustrates a method for managing a virtual reality system. The virtual reality system can be a multi-domain, computer-generated reality system operable using a VR / AR / XR manager. In block 3702, the manager may receive information generated by one or more sensors in devices worn by participants in a session conducted on the managed reality system. The information generated by the one or more sensors may indicate the participant's focus or viewpoint. In block 3704, the manager may provide focus measurements for the participants based on the information generated by the one or more sensors. Each focus measurement may characterize the duration of a participant's focus on the session leader or one or more objects indicated by the session leader while participating in multiple activities associated with the session. In some cases, the focus measurement may indicate the time when the participant first focused on the session leader or an object indicated by the session leader, the time when the participant last focused on the session leader or an object indicated by the session leader, or some other temporal aspect of the activity. In block 3706, the manager may calculate the difference between the participant's focus measurement and a corresponding baseline focus measurement for the session. In block 3708, the manager may generate a session performance metric for the participant by quantifying the difference.

[0229] In one example, a manager may authenticate a user of a reality scenario at a controller that manages or controls the multi-domain, computer-generated reality, configure a logical connection between the reality scenario and the multi-domain, computer-generated reality using one or more physical communication channels, and establish the user's presence as a participant in the multi-domain, computer-generated reality through the logical connection while maintaining the user's presence in the reality scenario.

[0230] In one example, a baseline focus measurement is obtained by performing a statistical analysis of focus measurements for multiple participants in a session. The baseline focus measurement may be based on the aggregate focus measurements for multiple participants in multiple sessions conducted on the managed reality system. The baseline focus measurement may include a measure of the relative difference between the position of a participant's avatar and the position of a session leader's avatar. The baseline focus measurement may include a measure of the relative difference between the position of a participant's avatar and an object indicated by the session leader. The baseline focus measurement may include the difference between the time a participant's avatar arrives at a location and the corresponding time the session leader's avatar arrives at the location. The baseline focus measurement may include the difference between the time a participant's avatar spends at or leaves a location and the corresponding time the session leader's avatar leaves the location.

[0231] In one example, the difference includes the time a participant's avatar spends at a location and the corresponding time the session leader's avatar spends at the location. The difference may also include the time a participant's avatar arrives at the location and the corresponding time the session leader's avatar arrives at the location. The difference may also include the time a participant's avatar leaves the location and the corresponding time the session leader's avatar leaves the location. The location may be a gathering point.

[0232] In one example, the one or more sensors include a motion sensor, a location sensor, or a camera. The discrepancy may include the distance between a participant's avatar and the conversation leader's avatar. The discrepancy may include the distance between a participant's avatar and an object pointed at by the conversation leader. The discrepancy may include the difference between the distance between a participant's avatar and a location and the distance between the conversation leader's avatar and the location. The location may be a gathering point.

[0233] In some examples, providing a measure of the participant's focus includes monitoring the participant's chat activity. Providing a measure of the participant's focus may include monitoring the participant's access to system information.

[0234] In one example, a manager can provide focus measurements for multiple participants to a session leader. The manager can display activity performance metrics for multiple participants to the session leader as corresponding session performance metrics are being generated. Computer-generated reality can include simulations. Computer-generated reality can enable experiential instructional reality.

[0235] Figure 38Flowchart 3800 illustrates a method for creating a virtual reality system according to certain aspects of the present disclosure. The virtual reality system can be a multi-domain, computer-generated reality system operable using a VR / AR / XR manager. In block 3802, a 3D representation of an object associated with a first activity can be generated. The 3D representation of the object can be assigned a unique identifier and configured for use in a first reality context. In block 3804, the position of the object in the reality context can be defined or calibrated. In block 3806, one or more time points associated with the object can be defined. The one or more time points can be represented relative to a start time of the first activity. In block 3808, multiple thresholds can be defined for the object. The multiple thresholds can include time-based thresholds and position-based thresholds. In block 3810, a determination can be made as to whether 3D representations should be generated for additional objects associated with the first activity. If 3D representations are to be generated for additional objects, the method continues at block 3802. If no additional 3D representations are to be generated for the first activity, in block 3812, a determination can be made as to whether additional activities have been defined or configured. Where additional activities have been defined or configured, the method continues with the next activity at block 3802. Where additional activities have not been defined or configured, the method continues at block 3814. At block 3814, the three-dimensional representation of the object may be provided in the multi-domain, computer-generated reality while one or more participants in the three-dimensional representation of the object engage in the first activity.

[0236] In some examples, one or more time points associated with an object identify when a three-dimensional representation of the object is visible in a multi-domain, computer-generated reality. The one or more time points associated with the object can identify a start time and an end time for viewing the three-dimensional representation of the object. The multiple thresholds can include a maximum time period between the start time and the end time. The multiple thresholds can include a minimum time period between the start time and the end time. The multiple thresholds can include an earliest start time for viewing the three-dimensional representation of the object. The multiple thresholds can include a latest time for viewing the three-dimensional representation of the object. The multiple thresholds can include a maximum distance for viewing the three-dimensional representation of the object, the maximum distance being calculated from the position of the object in the real-world scenario.

[0237] In some implementations, the three-dimensional representation of the object includes licensing information or information defining rights and permissions associated with the object when it is rendered in a multi-domain, computer-generated reality. In one example, the information defining rights and permissions associated with the object can be associated with a Figure 9 The DRM module 930 provided in the system manager 904 of the illustrated public XR manager 900 is related to the rights negotiated, purchased, and / or sold.

[0238] The foregoing description is provided to enable those skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other implementations. Accordingly, the claims are not intended to be limited to the aspects shown herein, but are intended to be consistent with the full scope of the claim language, wherein elements mentioned in the singular are not intended to mean "one and only one" unless explicitly stated as such, but rather "one or more." Unless otherwise specified, the term "some" refers to one or more. All structural and functional equivalents of the elements described in the various aspects throughout this disclosure that are known or subsequently known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be included by the claims. In addition, nothing disclosed herein is intended to be dedicated to the public, regardless of whether such disclosure is explicitly stated in the claims. No claim element should be interpreted under the provisions of 35 U.S.C. § 112, paragraph 6, unless the element is explicitly stated using the phrase "for," or in the case of a method claim, the element is stated using the phrase "step for."

[0239] Project 1): A method for managing multi-domain, computer-generated reality, comprising: receiving information generated by one or more sensors in a device worn by a participant in a session conducted on the managed reality system, wherein the information generated by the one or more sensors indicates a focus or viewpoint of the participant; providing focus measurements of participants based on information generated by the one or more sensors, each focus measurement characterizing a duration of focus of the participant on a leader of the session or on one or more objects indicated by the leader of the session while engaging in a plurality of activities associated with the session; calculating the difference between the focus measure for the participant and the corresponding baseline focus measure for the session; and By quantifying the differences, a measure of the participants' session performance was generated.

[0240] Item 2): The method according to item 1), further comprising: authenticating a user of a reality scenario at a controller that manages or controls a multi-domain, computer-generated reality; configuring a logical connection between the reality context and the multi-domain, computer-generated reality using one or more physical communication channels; and Through logical connections, the user's presence as a participant in a multi-domain, computer-generated reality is established, while maintaining the user's presence in the real situation.

[0241] Item 3): The method of item 1), wherein the baseline focus measure is obtained by statistically analyzing focus measures of multiple participants in a session.

[0242] Item 4): The method of item 1), wherein the baseline focus measure is based on aggregate focus measures of multiple participants in multiple sessions conducted on the managed reality system.

[0243] Item 5): The method of item 1), wherein the baseline focus measure comprises a measure of the relative difference between the positions of the participant's avatar and the avatar of the leader of the session.

[0244] Item 6): The method of item 1), wherein the baseline focus measure comprises a measure of the relative difference between the position of the participant's avatar and an object indicated by the leader of the session.

[0245] Item 7): The method of item 1), wherein the baseline focus measure comprises a difference between a time at which a participant's avatar arrives at a location and a corresponding time at which a leader's avatar arrives at the location.

[0246] Item 8): The method of item 1), wherein the baseline focus measure comprises the difference between the time a participant's avatar spends at or leaves a location and the corresponding time a session leader's avatar leaves the location.

[0247] Item 9): The method of item 1), wherein the difference comprises the difference between the dwell time of the participant's avatar at a location and the corresponding dwell time of the session leader's avatar at the location.

[0248] Item 10): The method according to item 1), wherein the one or more sensors include a motion sensor or a position sensor.

[0249] Item 11): According to the method of item 1), one or more sensors include a camera.

[0250] Item 12): The method of item 1), wherein providing a participant's focus measure comprises: Monitor participants' chat activity.

[0251] Item 13): The method of item 1), wherein providing a participant focus measure comprises: Monitor participant access to system information.

[0252] Item 14): The method according to item 1), further comprising: Provides focus measurements of multiple participants to the leader of a conversation.

[0253] Item 15): The method according to item 1), further comprising: Activity performance metrics for multiple participants are displayed to the leader of the session while corresponding session performance metrics are being generated.

[0254] Item 16): The method according to item 1), wherein the computer-generated reality comprises a simulation.

[0255] Item 17): The method according to Item 1), wherein the computer-generated reality implements an experiential teaching reality.

[0256] Item 18): A method for operating a virtual reality system, comprising: providing a first virtual or augmented reality scenario including a first scene; Gathering one or more participants within a first scene; presenting an image of the object in a first virtual or augmented reality context; and A virtualized version of the object is provided in the first virtual or augmented reality context or in the second virtual or augmented reality context.

[0257] Item 19): The method according to Item 18), wherein providing a virtualized version of the object comprises: A three-dimensional representation of an object is provided within a first scene.

[0258] Item 20): The method according to Item 18), wherein providing a virtualized version of the object comprises: providing a portal that provides a passage between a first virtual or augmented reality situation and a second virtual or augmented reality situation; and A three-dimensional representation of the object is provided within a second scene provided in a second virtual or augmented reality context.

[0259] Item 21): The method according to Item 18), further comprising: causing one or more participants to gather near the virtualized version of the object; and Visual representations of the plurality of participants provided to a first participant within the virtual reality system are suppressed, thereby placing the plurality of participants into an invisible mode.

[0260] Item 22): The method according to Item 21), further comprising: When the second participant is separated from the first participant by a minimum distance configured for the virtual reality system, the visual representation of the second participant is restored.

[0261] Item 23): A method for creating multi-domain, computer-generated reality, comprising: For each of multiple activities defined in one or more documents: generating a three-dimensional representation of an object associated with the first activity, wherein the three-dimensional representation of the object is assigned a unique identifier and is configured for use in a first real-world context; Calibrate the position of objects in real-world situations; defining one or more time points associated with the object, the one or more time points being expressed relative to a start time of the first activity; and defining a plurality of thresholds for the object, the plurality of thresholds comprising a time-based threshold and a location-based threshold; and When one or more participants in the three-dimensional representation of the object participate in a first activity, the three-dimensional representation of the object is provided in a multi-domain, computer-generated reality.

[0262] Item 24): The method of item 23), wherein one or more time points associated with the object identify when a three-dimensional representation of the object is visible in the multi-domain, computer-generated reality.

[0263] Item 25): The method of item 23), wherein the one or more time points associated with the object identify a start time and an end time for viewing the three-dimensional representation of the object.

[0264] Item 26): The method according to Item 25), wherein the plurality of thresholds include a maximum time period between a start time and an end time.

[0265] Item 27): The method according to Item 26), wherein the plurality of thresholds include a minimum time period between a start time and an end time.

[0266] Item 28): The method of Item 25), wherein the plurality of thresholds comprises an earliest start time for viewing the three-dimensional representation of the object.

[0267] Item 29): The method according to item 25), wherein the plurality of thresholds comprises a latest time for viewing the three-dimensional representation of the object.

[0268] Item 30): The method according to Item 25), wherein the plurality of thresholds comprises a maximum distance for viewing the three-dimensional representation of the object, the maximum distance being calculated from a position of the object in a real-world scenario.

[0269] Item 31): A method according to item 23), wherein the three-dimensional representation of the object includes licensing information or information defining rights and permissions associated with the object when provided in a multi-domain, computer-generated reality.

Claims

1. A method for operating a virtual reality system, comprising: providing a first virtual or augmented reality scenario including a first scene; Gathering one or more participants within the first scene; presenting an image of the object in the first virtual or augmented reality context; and A virtualized version of the object is provided in the first virtual or augmented reality context or in a second virtual or augmented reality context.

2. The method of claim 1 , wherein providing the virtualized version of the object comprises: A three-dimensional representation of the object is provided within the first scene.

3. The method of claim 1 , wherein providing a virtualized version of the object comprises: providing a portal, the portal providing a passage between the first virtual or augmented reality situation and the second virtual or augmented reality situation; and A three-dimensional representation of the object is provided within a second scene provided in the second virtual or augmented reality context.

4. The method according to claim 1, further comprising: causing the one or more participants to gather near the virtualized version of the object; and Visual representations of a plurality of participants provided to a first participant within the virtual reality system are suppressed, thereby placing the plurality of participants into an invisible mode.

5. The method according to claim 4, further comprising: When the second participant is separated from the first participant by a minimum distance configured for the virtual reality system, the visual representation of the second participant is restored.

6. A method for creating a multi-domain, computer-generated reality, comprising: For each of multiple activities defined in one or more documents: generating a three-dimensional representation of an object associated with the first activity, wherein the three-dimensional representation of the object is assigned a unique identifier and configured for use in a first reality context; calibrating the position of the object in the real situation; defining one or more time points associated with the object, the one or more time points being expressed relative to a start time of the first activity; and defining a plurality of thresholds for the object, the plurality of thresholds comprising a time-based threshold and a location-based threshold; and When one or more participants in the three-dimensional representation of the object participate in the first activity, the three-dimensional representation of the object is provided in the multi-domain, computer-generated reality.

7. The method according to claim 6, wherein: The one or more time points associated with the object identify when a three-dimensional representation of the object was visible in the multi-domain, computer-generated reality. 8 . The method of claim 6 , wherein the one or more time points associated with the object identify a start time and an end time for viewing a three-dimensional representation of the object. 9 . The method of claim 8 , wherein the plurality of thresholds comprises a maximum time period between the start time and the end time for viewing the three-dimensional representation of the object. 10 . The method of claim 9 , wherein the plurality of thresholds comprises a minimum time period between the start time and the end time for viewing the three-dimensional representation of the object. The method of claim 8 , wherein the plurality of thresholds comprises an earliest start time for viewing the three-dimensional representation of the object. 12 . The method of claim 8 , wherein the plurality of thresholds comprises a latest time for viewing the three-dimensional representation of the object.

13. The method of claim 8, wherein the plurality of thresholds comprises a maximum distance for viewing the three-dimensional representation of the object, the maximum distance being calculated from a position of the object in the real-world situation.

14. The method of claim 6, wherein the three-dimensional representation of the object includes licensing information or information defining rights and permissions associated with the object when rendered in the multi-domain, computer-generated reality.

Images