Systems and methods for virtual reality environments

Abstract

To provide a system for a virtual reality environment.SOLUTION: A method includes a reception step, a search step, a display step, a detection step, an identification step, and an instantiation step. The reception step receives data. The search step searches a database for a resource address about each virtual object according to context associated with a virtual environment. The display step displays a virtual environment view. The detection step detects interaction with a first virtual object by a virtual environment user. The identification step identifies a first resource address. The instantiation step instantiates the rendering of the first resource data and displays the searched resource in the virtual environment.SELECTED DRAWING: Figure 3B

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims priority to and the benefit of U.S. patent application Ser. No. 17 / 139,771, entitled "SYSTEMS AND METHODS FOR VIRTUAL REALITY ENVIRONMENTS," filed December 31, 2020, the entire contents of which are incorporated herein by reference in their entirety for all purposes.

[0002] FIELD OF THE DISCLOSURE The present disclosure relates generally to systems and methods for virtual reality environments, and more particularly to systems and methods for providing dynamic virtual reality experiences that include virtual objects with externalized context-dependent links to data. [Background technology]

[0003] Virtual reality environments allow for the training and certification of users and operators in environments that would be dangerous in reality, such as nuclear power plants or chemical processing plants, with simulated emergency situations such as fires, gas leaks, or other such situations. Such virtual reality environments can provide excellent training opportunities because they can be highly immersive with detailed simulations and realistic graphics.

[0004] However, developing and programming such virtual reality environments and scenarios is complex and time consuming. Moreover, once programmed, the environment or scenario is usually fixed and difficult to change without reprogramming the whole thing again. Currently, a large amount of information already exists from past training systems, but that information has not yet been integrated into such virtual reality environments. Adding this data may require reprogramming the entire environment as well. Thus, the complexity, cost, and time constraints of transferring a scenario or training tool to a virtual reality environment have prevented few developers from taking advantage of the advanced capabilities and features of virtual reality. Summary of the Invention

[0005] Various objects, aspects, features, and advantages of the present disclosure will become more apparent and will be better understood by reference to the detailed description taken in conjunction with the accompanying drawings, in which like reference numbers generally indicate identical, functionally similar, and / or structurally similar elements. [Brief description of the drawings]

[0006] [Figure 1A] FIG. 1 is a diagram of a virtual reality environment for training and certification, according to some embodiments. [Figure 1B] FIG. 1 is a logical block diagram of a training system for virtual reality-based training and certification, according to some embodiments. [Figure 2A] 1 is a flowchart of a method for development and deployment of virtual reality based applications according to some embodiments. [Figure 2B] FIG. 1 is a tree diagram of a logical hierarchy for development and deployment of virtual reality based applications, according to some embodiments; [Figure 2C] 1 is a flowchart of a method for generating configuration elements for development and deployment of virtual reality based applications according to some embodiments. [Figure 2D] 1 is a flowchart of a method for modifying configuration elements for development and deployment of virtual reality based applications according to some embodiments. [Figure 3A] 1 is a flowchart of a method for providing an interactive virtual reality environment according to some embodiments. [Figure 3B] 1 is a flowchart of a method for providing an interactive virtual reality environment according to some embodiments. [Figure 4A] FIG. 1 is a block diagram of a system for providing an interactive virtual reality environment according to some embodiments. [Figure 4B] FIG. 1 is a diagram of a virtual reality environment for training and certification, according to some embodiments. [Figure 4C] 1 is an exemplary training and certification log for a virtual reality system for training and certification, according to some embodiments. [Figure 4D] 1 is a flowchart of a method for virtual reality based training and certification according to some embodiments. [Figure 5A] 1 is a flowchart of a method for activating a virtual reality environment according to some embodiments. [Figure 5B] 1 is a flowchart of a method for providing reliable application deployment according to some embodiments. [Figure 6A] FIG. 1 is a block diagram illustrating an embodiment of a computing device useful in connection with the methods and systems described herein. [Figure 6B] FIG. 1 is a block diagram illustrating an embodiment of a computing device useful in connection with the methods and systems described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] The details of various embodiments of the methods and systems are set forth in the accompanying drawings and the description below.

[0008] In reading the following description of the various embodiments, the following descriptions of sections of this specification and their contents may be of help: Section A describes embodiments of systems and methods for virtual reality environments. - Section B describes computing environments that may be useful in practicing the embodiments described herein.

[0009] A. Systems and Methods for Virtual Reality Environments Virtual reality environments allow for the training and certification of users and operators in environments that would be dangerous in reality, such as nuclear power plants or chemical processing plants, in simulated emergency situations such as fires, gas leaks, or other such situations. Such virtual reality environments can provide excellent training opportunities because they can be highly immersive with detailed simulations and realistic graphics.

[0010] For example, Figure 1A is a diagram of a virtual reality environment 10 for training and certification, according to some embodiments. The virtual reality environment 10 may have a three-dimensional environment and may be viewed from the perspective of a virtual camera, which may correspond to the viewpoint of a user or operator. In some embodiments, the virtual camera may be controlled via a joystick, keyboard, or other such interface, while in other embodiments, the virtual camera may be controlled via tracking of a head mounted display (e.g., virtual reality goggles or headset) or similar head tracking such that the user's view within the virtual environment corresponds to the user's physical movements and orientation.

[0011] The virtual reality environment may include one or more objects 20, which may include buttons, levers, wheels, panels, screens, gauges, pipes, ladders, signs, or any other type and form of object. The objects 20 may have three-dimensional boundaries in many embodiments and may include textures, shading, or coloring to enhance realism, including photorealistic textures or images in some embodiments. The objects 20 may be interactive, allowing the user to interact with the objects to control various aspects of the simulation. For example, in some embodiments, the user may select an object (e.g., physically by reaching for the object in embodiments where the user's movements are tracked; using a user interface device such as a joystick, mouse, tablet, pointer, or other device; verbally according to a voice-command interface; visually by pointing a virtual camera at the object and pressing a select button or waiting a predetermined period of time, or by any other such method) and various functions may be performed.

[0012] Developing and programming such virtual reality environments and scenarios can be complex and time-consuming. For example, in the example of FIG. 1A, a simple control panel may contain dozens of separate objects with corresponding functions. Once programmed, an environment or scenario is usually fixed and difficult to change without reprogramming everything again. Adjusting an environment or scenario (e.g., to reflect updated hardware in the real environment) may require reprogramming the environment or recompiling the entire virtual reality application. Furthermore, a large amount of information from past training systems already exists, but that information has not yet been integrated into such virtual reality environments. Adding this data may require reprogramming the entire environment as well. Thus, the complexity, cost, and time constraints of transferring scenarios and training tools to a virtual reality environment have prevented few developers from taking advantage of the advanced capabilities and features of virtual reality.

[0013] The systems and methods discussed herein provide dynamic and reconfigurable virtual reality environments with in-environment access to external data and resources. In embodiments of the systems and methods discussed herein, one of the most important aspects of training, the supplementary materials available to the students, is configurable by the end customer without the need for additional vendor involvement. These system embodiments also provide an external mechanism to modify other aspects of the virtual reality experience without the need to recode or compile the experience. This allows for changing the main flow of the experience, altering its behavior based on a particular user's access, and adding brand or customer specific aspects to the application. Not only can the same level or the same environment provide significantly different experiences for different users, from beginners to experts, but simple configuration also allows for options for random or sequenced events that can be controlled by an instructor or administrator.

[0014] In some embodiments, during creation and / or modification of a virtual reality environment or scenario, objects 20 may be tagged with metadata, such as a unique identifier for the object (a globally unique identifier or GUID, or in some embodiments a Uniform Resource Identifier (URI)). During compilation, a configuration is generated that provides key / value pairs for the endpoint URI and other metadata. During runtime, objects with associated metadata may be annotated with an icon 30, as shown in the example of FIG. 1A, or otherwise identified for interaction. General "help" files, as well as teaching aids, may be specified in additional URIs. Default values ​​for these parameters may be provided at the time of experience creation. This configuration structure, once packaged, may be available for editing by a user or administrator without the need to recompile the virtual reality application. Changes to values ​​in the configuration may be reflected in the experience the next time it is launched, and the configuration (and associated key / value pairs) are read and interpreted. In some embodiments, for each object encoded with these parameters, if a value exists (or if a particular type of value exists indicating a help file or other information), an icon 30 for that information may be displayed. This may be limited to a particular execution mode, such as a training mode or a guest mode. In response to a user selecting an icon, the system may instantiate an in-environment web browser or other interface 40 that may display a view of the corresponding content or resource rendered within the virtual environment, in a configurable presentation style, along with associated controls.

[0015] Thus, the systems and methods discussed herein provide for dynamic content delivery in virtual reality without the need to re-create existing content, while providing real-time information updates and access to historical data that remains readily available, such as documents, audio, video, and other file formats. The systems and methods allow for updating URI addresses or endpoint resources through reconfiguration of external configuration elements, without requiring programming knowledge or the need to recode or recompile executable applications.

[0016] 1B is a logical block diagram of a virtual reality-based training system for training and certification, according to some embodiments. A developer or designer 154 or other administrator (or a computing device under the control of or acting on behalf of an administrator) may utilize a development environment 150 to generate a virtual reality experience 152. The development environment 150 may include any suitable development application, library, or system for generating three-dimensional virtual reality content, including a mapping engine, a rendering engine, or other such application or system, such as the Unity Engine developed by Unity Technologies, Inc. of San Francisco or the Unreal Engine developed by Epic Games, Inc. of North Carolina, or any other such development kit or engine. The virtual reality experience 152 may broadly refer to a virtual environment (e.g., including objects, textures, images, etc.), a scenario for the virtual environment (e.g., including events that occur in response to triggers or time that cause changes to one or more objects in the virtual environment), or a compiled virtual reality application. Thus, virtual reality experience 152 may include an application or data executable by the application to provide an immersive three-dimensional virtual environment and may be developed in development environment 150.

[0017] The virtual reality experience 152 may be compiled to generate a virtual reality runtime package 156 and a configuration structure 158. The virtual reality runtime package 156 may have compiled instructions or executable code for providing a virtual reality environment. As mentioned above, a virtual reality environment, once compiled, is typically self-contained and fixed and must be recompiled for any changes to be made. However, by using linked configuration structures 158, embodiments of the systems and methods discussed herein may enable dynamic modification of a virtual reality environment or scenario.

[0018] Specifically, objects in the environment and specified in the runtime package 156 may be associated with a unique identifier, which may be referred to as a resource identifier or GUID. The configuration structure 158 may include an index, array, database, or other data structure that associates resource identifiers or GUIDs with addresses (URIs) of external resources. When a student 160 or other user (or a computing device operated by or on behalf of a student or other user) executes the compiled virtual reality experience, the user's computing device may identify metadata including resource identifiers or GUIDs of objects in the virtual environment, read the linked URI addresses from the configuration structure 158, and retrieve the associated resources for display within an embedded browser or renderer 154 in the virtual environment. To dynamically change a scenario or environment, the configuration structure 158 may be edited without modifying the compiled virtual reality runtime package 156 to allow for the selection and embedding of different resources, input of additional features, etc. In some implementations, external linked resources may be modified or replaced without modifying the configuration structure to embed different resources as well. In some embodiments, all objects in an environment or scenario may have a unique resource identifier or GUID, but may not necessarily have a linked resource URI in the configuration structure 158, and such linked resources may be added after compilation to add additional functionality or data to the virtual environment without requiring runtime packages or code modifications.

[0019] 2A is a flowchart of a method for development and deployment of virtual reality based applications, according to some embodiments. In step 202, a developer or designer's computing device or other operator may launch a development environment, such as a software development kit or other such engine, for generating and configuring a virtual reality environment or scenario. In step 204, the developer or designer may add virtual objects to the environment or scenario. In step 206, a unique identifier may be assigned to the virtual object (e.g., randomly, incrementally, as a hash of the object's data, etc.), and in step 208, other metadata and / or a name may be set for the object (e.g., including locking settings for whether the object can be manipulated or interacted with, animations, or other such data, etc.). In some embodiments, default metadata may be utilized and / or may be set in step 210 to be applied to any virtual objects for which no object-specific or custom metadata has been specified (step 210 may be performed as shown and / or after step 202, or before step 214, or at any other suitable time prior to runtime compilation). In some embodiments, the visibility of the object or linked resource may be set in the metadata in step 212. Steps 204-212 may be repeated iteratively for additional objects.

[0020] Once all objects have been added, the virtual reality runtime code may be compiled into an application or execution package in step 214. The package may contain the executable code as well as resources required for execution, such as textures, images, sounds, or other media content. In some embodiments, during compilation, one or more of the following may occur: The system identifies and collects all objects to be processed, ignoring those with certain metadata tag values. For each object that is processed, the system collects its metadata. If metadata has a preset value for an object, this value is used as the default for configuration elements of a particular information type, such as Info, Supplement, or Help. Objects may have associated types to define the values ​​that can be stored in associated properties and the basic operations that can be performed on the object. Metadata values ​​can be set as locked to prevent values ​​from being overwritten and / or may be designated as public or private to identify whether the metadata value should be looked up to lower levels in the hierarchy or only in the current context. Once processing is complete, the configuration structure is saved as a virtual reality package or application.

[0021] In step 216, a configuration structure may be generated having an association between each GUID and either a predefined resource (or other metadata, such as that specified in step 210) and an object-specific resource. Such an association may include a key-value pair of a GUID and an address (URI) or identifier of an external resource. For example, in some embodiments, the URI may include a private URL scheme (e.g., a custom scheme such as "tdxr: / / " rather than "http: / / " or other common schemes) to allow for interception of the request by an agent or URI handler or redirection of the request to an agent or URI handler. The URI may further include a host server identifier (e.g., a server in a domain such as tdxr.example.com, which may include a port number or other address information), a portal identifier, and / or a resource identifier. In some embodiments, the URI may include an action to use in a query or request for the associated resource, such as "download" or "run".

[0022] The host, portal, and resource identifiers may be used to organize objects within the system. For example, the host, portal, and resource identifiers (and in some embodiments other identifiers) may be part of a classification and categorization system that organizes all three-dimensional objects in a project into categories and types. A set of templates is created for objects to be represented in the virtual environment. These templates include a GUID or other identifier, the name of the object, and other metadata common to the particular object type. For example, with brief reference to FIG. 2B, a tree diagram of a logical hierarchy for development and deployment of virtual reality based applications is shown, according to some embodiments. As shown, objects may be associated with leaf nodes of a hierarchical tree, with higher tiers or levels including a group / user tier, a project tier, a portal tier, an organization tier, and a site tier. In some embodiments, these tiers are defined as follows: Site: This level may be associated with an application manager. Since the virtual reality application or environment may be provided as a Software-as-a-Service (SaaS) model, this tier may represent the top-level vendor that maintains the entire site and content provided by the entire organization. · Organization: This level may be associated with an organization or a customer and may be managed by a representative of the organization. Project: A project is a grouping of cohesive content. That content can be shared in many places known as portals, so at this level you set the metadata that should be there by default regardless of the context in which the information is presented. · Portal: This level can contain collections within a project and can be configured to override settings at higher levels, thus providing a collection-specific experience. User / Group: At this level, configuration elements can be added to provide tailored features for users or groups of users based on their login environment, thus providing URLs and metadata tailored to their individual experiences. There are also various categories of metadata that are tracked at different levels of the system, such as: Identities such as username or login name, display name, or other SAML / LDAP values The entire experience, including properties that describe the details of the experience, such as geographic location, VR capabilities, controller type, training methods, etc. Objects that contain metadata related to aspects of virtual objects in the environment This metadata guides the operation of the experience by managing launch parameters based on the loaded data, dictating which links are presented, the training mode in which the experience is launched, and other related actions. If metadata is not set as a required property, in some implementations a menu may be displayed to allow the user to select metadata options for the application instance.

[0023] Returning to FIG. 2A, a configuration structure may be added to each virtual object or element by determining whether a default property should be applied to the element (e.g., for elements that do not have a custom property set but in some implementations were set to visible in step 212), in which case adding a default value to the element's key in the configuration structure in step 218. The default value may specify a general help message or other interaction, or other such resource. This process may be repeated for each virtual object or element, and once completed, the configuration structure may be saved in association with the virtual reality runtime in step 220.

[0024] Thus, the compiled virtual reality experience includes the configuration structure as an external object that can be edited and put back into the package. An administrator or instructor can modify this file, changing the metadata and endpoint URI addresses by identifying the relevant objects, finding the specific metadata properties, and modifying the value parameters. Changing the values ​​in the configuration will be reflected in the experience the next time it is launched. For each object coded with these parameters, if a value is present, an icon will be displayed for information in modes where display is permitted. Selecting the icon provides a view of the content with associated controls in a configurable presentation manner. Other metadata can also be manipulated at various levels up to delivery to modify the virtual experience, allowing different users to run the same virtual reality application but interact with entirely different training events.

[0025] FIG. 2C is a flow chart of another method for generating a configuration structure for development and deployment of a virtual reality based application, according to some embodiments. The method of FIG. 2C is similar to and an extension of the process described above. In step 250, a computing device may receive a compiled virtual reality experience (e.g., an application or data file generated in step 214). In step 252, objects may be read from the virtual reality experience (e.g., extracted from object code, XML data, or other such data structure) and a configuration structure may be generated (e.g., as a data array or data structure with keys corresponding to the GUID of each object). For each object, custom metadata for the object (e.g., added by an administrator or trainer) may be read and added in association with the object's GUID in the data structure in step 254. If no custom metadata exists and a default metadata value exists (e.g., for a user group corresponding to the project, or for a portal corresponding to the project if no default metadata exists for the tier, or for a project corresponding to the project if no metadata exists for the tier, etc.), this default data may be added in association with the object's GUID in step 256. Similarly, if custom metadata exists but the value is locked at a default level (e.g., the state of the metadata object does not allow overriding the default value and the default value must be used), the default data may be added associated with the object's GUID in step 256. This may be repeated iteratively for each additional object. Finally, the metadata may be saved as an externalized (e.g., separate from the virtual reality runtime) configuration in step 258.

[0026] As described above, a compiled virtual reality experience includes a configuration configuration as an external object that can be edited and returned to the system representation of the VR experience (e.g., a container or other compiled structure). An administrator or developer can modify this file to change the metadata and endpoint URI by identifying the relevant object, locating specific metadata properties, and modifying value parameters. Once editing is complete, the configuration is packaged back to the system representation of the experience. FIG. 2D is a flowchart of a method for modifying configuration configurations for development and deployment of virtual reality-based applications, according to some embodiments. In step 280, an administrator can select a virtual reality experience (and associated configuration configuration) to modify, and in step 282, the configuration configuration can be retrieved. In step 284, an editor interface can be launched, which can include a dedicated application or a text editor (e.g., in embodiments where the configuration configuration is stored in a human-readable format such as XML data, comma-separated values, or a spreadsheet). In step 286, the administrator can modify the URI and other metadata of the configuration configuration, and in step 288, the modified configuration can be saved. Subsequently, when the virtual reality runtime executes, it may retrieve the modified configuration and may utilize the modified URI and other metadata to retrieve the content or resource.

[0027] The final step in the process is to execute the runtime application so that the student can run the virtual training session. When the virtual reality experience is run, the configuration is loaded, all metadata values ​​are seeded with data, and the correct endpoints are identified and displayed during runtime. All information, supplemental, and help icons are displayed on objects that have metadata corresponding to their keys.

[0028] 3A-3B are flowcharts of a method for providing an interactive virtual reality environment, according to some embodiments. In step 302, a virtual reality experience may be launched. The virtual reality experience may be launched as an executable file or as an address (URL or URI) of a data file or executable file for the virtual reality experience, which may be downloaded and retrieved. For example, in some embodiments, a browser application or other application on the client device may read a single argument as a private scheme URL or URI (e.g., a com.example.txdr: address) or file path to an executable file. If the argument is a private scheme address, a local agent or handler may receive the address by a redirect from the browser or operating system (e.g., by registering a private scheme) or by intercepting a request from the browser.

[0029] Specifically, to facilitate certain operations between portal pages or resources and native platform applications, in some embodiments a private URL scheme may be used that is com.example.tdxr or a similar scheme. The scheme name may conform to IETF RFC 7595 to avoid namespace collisions with registered URL schemes or other private schemes in many embodiments. The scheme URL may be used to identify extended reality resources regardless of a particular storage system or retrieval location, so that resources on a local file system or in a remote data server can be referenced by the same tdxr URL or URI.

[0030] In many implementations, URLs for private schemes follow the following template: com.example.tdxr: / / {xr-host} / {xr-portal} / {xr-id}(?action) Here, xr-host may specify the authority (eg, host name and port number) of the remote data server on which the portal can be found. xr-portal is a portal that has XR resources. The xr-id is a path component that uniquely identifies an XR resource within its portal. The xr-id may be any valid URL path component, but it is recommended that it be a human-readable identifier for the resource. It does not have to match the file name or directory, but may be derived from it.

[0031] In some embodiments, the optional action query parameter can have a value of either download or run. The default value may be run, in which case the parameter may be omitted. For example, the link can be interpreted by the appropriate protocol handler and the VR experience with id very-cool-vr can be downloaded from the xr.example.com / xr-demo portal.

[0032] In some embodiments, to avoid unnecessary downloads, the handler or local agent may scan a local library or data store for extended reality experiences with the given xr-id (e.g., those that incorporate external configuration elements and have links to external resources) from a specified portal (e.g., associated hierarchical layers other than the site, and in some embodiments, the organization layer, may be replicated on local storage, and associated resources corresponding to the user, group, portal, or project layers may be stored or cached locally). If the file exists and the URL action is not "download", the handler may launch the experience. If the file does not exist or the URL action is "download", the handler may make a download request for the experience to the portal given at xr-host / xr-portal. Once downloaded, the package may be unzipped and, in some embodiments, decrypted. If the action is "run", the downloaded experience is launched (conversely, if the action is "download", the user is notified that the download is complete and is prompted to manually launch the experience).

[0033] To launch an application, in some embodiments, the file may be validated (e.g., checking for corruption) and the system may verify that the file is licensed to run in the current context (e.g., user and / or platform). In some embodiments, if the experience includes a configuration file, in some embodiments, a form may be displayed to collect runtime context information specified in the configuration file. The VR experience may then be launched as a new process. In step 304, default metadata may also be loaded for use with objects that lack specific or custom metadata.

[0034] In step 306, the configuration configurations are loaded and read to seed the data of all metadata values ​​so that the correct endpoints can be identified and displayed during runtime. The separately stored configuration configurations can be downloaded separately from the VR runtime application or data. In some embodiments, during the process of downloading or running an experience, only the components of the experience that have changed (based on the comparison of data), such as new icons, revised configuration configurations, or the experience itself if modified, can be downloaded (e.g., in an incremental or differential update). During the execution of the experience, all information, supplemental, and help icons can be displayed on objects that have metadata corresponding to their keys. For example, for each object, the system can identify its GUID and determine whether a value exists in the configuration configuration for that GUID. If not, default metadata can be used, or the system can determine whether the object has a locked property. If so, the object can be unavailable for interaction. This may be due to, for example, the mode of operation (e.g., in test mode, additional help resources that are available in learn or guide modes may not be available) or due to the scenario (e.g., when a user is being trained to perform a sequence of operations, some objects or elements may be locked initially and unlocked later in the scenario). The mode (e.g., learn, guide, and test) may be specified in the metadata, and in some implementations, different modes may have different values ​​(e.g., different modes have different visibility) such that values ​​are displayed or ignored based on the mode supported for that metadata property. For example, in test mode, the help icon may be disabled; however, an information icon may still be enabled or visible to provide answers to questions or actions for which an incorrect answer was indicated.The visibility of an icon within an experience can be guided by many environmental and metadata factors to maintain a realistic setting: voice commands, controller movements, and key combinations are three examples, and / or in some implementations, metadata can specify how information about that object is displayed, such as always on, always off, on failure, distance, etc.

[0035] If the object is not locked, the object metadata may be read from the configuration structure and applied to the object in step 308. Once all objects have been processed, in step 310 the virtual reality experience may be loaded and executed.

[0036] Continuing with FIG. 3B, in step 320, objects in the virtual environment may be drawn (e.g., at coordinates or positions according to the configuration of the environment). For each object, the system may determine whether the simulation is in a mode in which an icon may be displayed (e.g., according to metadata for that object) and whether such metadata is present; if so, in step 322, an icon may be displayed in the virtual environment to indicate that information is available (e.g., as in FIG. 1A above). This may be repeated for each additional object, and may be repeated as the simulation or scenario progresses (e.g., to enable icons for subsequent steps or disable icons for steps that have already been performed to reduce visual clutter).

[0037] During execution, the system may detect user interaction with the object, step 322. This interaction may include pressing a button, pulling a lever, turning a knob or dial, etc., and the interaction may be made in any suitable manner (e.g., by tracking the user's hand position and determining the intersection between the corresponding hand position of the user's virtual avatar and the object, by tracking the position of a virtual "laser pointer" or other device, by selection via a mouse, keyboard, joystick, or other interface element, via verbal commands received by a voice-to-text or voice-command engine (e.g., "press the blue button" or "turn the dial to 20"), by selection via head tracking (e.g., looking at a particular button and holding the user's head position for a few seconds), or any other such manner or combination of manners). Upon detecting interaction with the object, in step 324, the local agent or handler may identify in the object's metadata the resource path or address and identifier of the resource to display. In some implementations, the metadata is loaded at runtime from the configuration configuration and values ​​are applied to the object's metadata such that addresses are read directly from the object's metadata. In another embodiment, the object's metadata may consist of a GUID, and the local agent or handler may read the corresponding metadata value from a configuration element in response to detecting an interaction.

[0038] In step 326, the handler or local agent may instantiate a browser within the virtual environment. For example, the handler or local agent may run a web browser, such as a Chromium-based web browser, in a viewport displayed within the virtual environment (e.g., as a floating window in proximity to a virtual object, in a virtual representation of a tablet computer or other computing device, in a model menu, or in any other representation within the virtual environment). The browser may first attempt to retrieve the resource at the specified path and address from local storage or cache. If the resource is not available locally, in step 328, the browser may retrieve the resource at the specified host, path, and address (and in some embodiments store the resource locally). The resource may then be displayed by the browser in step 330. The resource may be of any type and form, including video, images, text, instruction lists, checklists, guides, help information, or any other type and form of useful information that may be associated with an object in the virtual environment.

[0039] 4A is a block diagram of a system for providing an interactive virtual reality environment, according to some embodiments. The client device 400 may comprise a laptop computer, a desktop computer, a portable computer, a tablet computer, a smartphone, a video game console, an embedded computer, an appliance, or any other type and form of computing device for providing a virtual reality environment. In some embodiments, the client device 400 may comprise a wearable computer, such as a standalone or all-in-one virtual reality headset.

[0040] The client device 400 may have one or more processors 402, a network interface 404, and input / output devices 406. The processor 402 may have any type and form of processing device, including a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), a coprocessor, an ASIC-based processor, or any other such device for executing logical instructions. The network interface 404 may have any type and form of interface, including a cellular interface, a wireless interface including 802.11 (WiFi), a Bluetooth interface, an Ethernet interface, or any other type and form of network interface. The network interface 404 may be used to communicate with a resource server 430 over a network (not shown), such as the Internet, which may in turn be a computing device including one or more processors, network interfaces, and input / output devices (not shown). In some embodiments, the resource server 430 may be comprised of one or more physical computing devices, such as a server farm, or one or more virtual computing devices executed by one or more physical computing devices, such as a server cloud.

[0041] The client device 400 may have or communicate with one or more sensors 408 for tracking the user's movements, and one or more displays, including a virtual reality or augmented reality display 410. Although shown separately (e.g., for outside-in tracking, or tracking by measuring the displacement of an emitter or reflector on the headset and / or controller from another sensor), in some embodiments the sensors 408 and virtual reality / augmented reality display 410 may be integrated (e.g., for inside-out tracking, or tracking by measuring translation between successive images of the physical environment obtained from a sensor on the headset). A variety of tracking systems may be implemented, including inside-out, outside-in, stereo camera-based tracking, time-of-flight measurement and tracking, artificial intelligence-based tracking, etc. In some embodiments, the tracking system may track the user's head (e.g., position and orientation), the user's hands (e.g., via image recognition from a camera imaging the controller or hands), and / or any other limbs or appendages of the user.

[0042] The client device 400 may include a memory device 412 that stores data and applications for execution by the processor 402. For example, the memory 412 may have a browser application 414, which may include a web browser, remote access application, or other such application for selecting, downloading, and launching a virtual reality environment or application. For example, the browser application 414 may be used to log into a remote server and download virtual reality training scenarios for local storage and execution.

[0043] The VR application 416 may comprise an application, server, service, daemon, routine, or other executable logic for providing an interactive virtual reality environment and interacting with objects or events in the environment. The VR application 416 may comprise a plug-in executed by the browser application 414 or may be a separate application. The VR application 416 may execute data files prepared by a developer for an environment or scenario, or may be a standalone application compiled by a developer and containing configuration details within the application.

[0044] A browser, such as browser application 414, can deliver content to a user based on URLs, which consist mostly of static or dynamic HyperText Markup Language (HTML) pages interspersed with scripts, rich media, and other data. To access the data, a URL is used, which is a reference to a web resource that specifies a location on a computer network and a mechanism for retrieving it. The protocol for accessing the information can be common, such as Hypertext Transfer Protocol (HTTP), Hypertext Transfer Protocol Secure (HTTPS), File Transfer Protocol (FTP), or specialized protocols, such as Titinia Delivery Extended Reality Protocol (TDXR), which are handled by applications installed on the system and registered with the operating system of the client device 400. This allows for sophisticated delivery of virtual reality content, which is handled by applications 416 on the client system, based on specialized packaging and unpacking of the content.

[0045] As mentioned above, a VR application 416 or data for such an application may be associated with a configuration structure, which may be downloaded or bundled with the application or obtained separately from a resource server 430. The configuration structure may map GUIDs of objects in the virtual environment to paths and resource identifiers for retrieval at runtime.

[0046] In some embodiments, the memory 412 may include a local agent 418. The local agent 418 may comprise a browser application 414 or a VR application 416 application, service, server, daemon, routine, or plug-in, or other executable logic for retrieving configuration settings and resources and providing the resources for display or rendering by an embedded browser or renderer within the VR application 416. The local agent 418 may be variously referred to as an application handler, a scheme handler, a content delivery agent, a link data platform (LDP) agent, a thin client, a helper program, or similar terms.

[0047] Content may be conditionally delivered based on the state of each individual file, using a locally stored file if it is the same as specified by the server location, or downloading the file if it is not locally stored. The local agent 418 may be configured to launch, discover, and download the VR experience or application 416. The local agent 418 may work in conjunction with a web portal or server provided by an application server 432 (e.g., a web server, FTP server, data server, or similar service) of the resource server 430 computing device to provide a seamless user experience between the portal and the host operating system platform. The local agent 418 may include a "thin client" that provides minimal functionality that cannot be easily provided by the web browser application 414. The local agent may intercept or receive redirection requests using a private scheme (e.g., com.example.tdxr, as described above) and may communicate with the application server 432 for most of its data and business logic using a representational state transfer (RESTful) API defined for the portal or other communication functions. For example, application server 432 may comprise a Linked Data Platform (LDP) or an HTTP server conforming to the requirements of the LDP specification for providing RESTful interactions (GET, POST, PUT, DELETE, etc.) with Resource Description Framework (RDF) and non-RDF resources. RDF resources can support ontology, taxonomy, provenance, linking, metadata, and rule capabilities, providing the system with greater functionality and flexibility, including searching for specific elements (through an ontology or taxonomy-based search, a user may search for and retrieve virtual training scenarios that include a model number or type of equipment (such as personal protective equipment or respirators) specified in the object's metadata).

[0048] Local agent 418 may retrieve and store configuration settings via application server 432 in remote resource 434, or may store a copy of the settings locally in local resource 420. As described above, local agent 418 may receive requests, determine whether resources are available locally, and provide local resources or download remote resources accordingly.

[0049] 5A, a flow chart of a method for launching a virtual reality environment is illustrated, according to some embodiments. At step 502, a local agent may intercept or receive a request utilizing a private URL scheme via a redirect by the operating system. In some embodiments, if a local agent is not installed, the request may be received or intercepted by a virtual reality application and / or a browser application, and at step 504, the local agent may be downloaded and / or installed.

[0050] As described above, the request may identify the resource by a GUID or resource identifier, and in some embodiments, a portal identifier, project identifier, or other such identifier. The local agent may determine whether a local copy of the resource exists, and in some embodiments, may compare the local copy to a remote copy to determine whether an update or modification has occurred (e.g., sending a request to the server including an identification of a hash value, version number, or other identifier of the local copy of the resource, and receiving a notification that no update has occurred or receiving a new copy of the resource). If the resource does not exist locally, or if the local version is out of date, then in step 506 the resource may be retrieved from a remote server. The remote server may be identified via a host identifier in the request or a domain, as described above, as part of the path or address of the resource. In step 508, the resource may be executed, for example, by instantiating an embedded browser or renderer in the virtual environment and displaying or rendering the resource in a window of the embedded browser or renderer in the virtual environment.

[0051] In some embodiments, access to resources may be controlled and / or resources may be encrypted. Extended reality content may in some embodiments be subject to a number of checks and eligibility for distribution may be determined based on one or more factors, such as whether the client device has an active license for the content, whether the user has an active license, whether the distribution portal has an active license, whether the client device and / or user has authorization to access the distribution portal, whether a valid key pair exists for the user, client device, or organization, or whether the user or client device successfully authenticates with a remote server (e.g., via username, password, biometric identifier, two-factor authentication scheme, etc.). Encrypted content may be unencrypted and executed only in memory using a just-in-time (JIT) paradigm and not stored locally unencrypted, enforcing access control in some embodiments.

[0052] FIG. 5B is a flow chart of a method for providing a reliable application deployment, according to some embodiments. In step 520, the developer may prepare a virtual reality package (e.g., compiled application or data and / or configuration configuration) or may prepare modifications to the configuration configuration. If encryption is not selected or access control is disabled, the package or configuration may be uploaded in step 526 and the package or configuration may remain in step 530 (e.g., saved or archived for use across multiple sessions, especially for cloud-based deployments with virtualized application or resource servers). If encryption is selected, depending on whether the developer is uploading site-specific or customer-specific content, the system may check for a corresponding encryption key pair. If no key is available, an error may be returned to the client in step 522 so that an encryption key may be provided (e.g., sharing a public key or a symmetric encryption key). If a key is available, encryption may be enabled in step 524. The package or configuration may be uploaded in step 526 and encrypted using the key in step 528. The encrypted package or configuration may remain in step 530.

[0053] As mentioned above, in some embodiments, the virtual environment may be used for training and / or certification. For example, a trainer or an experienced user may record a series of interactions with objects in the environment. The interactions may be played back in the environment in a second user's or student's view, for example as a virtual or "ghost" avatar. This may allow the student user to simultaneously perform the same actions as the recorded "ghost" avatar and may allow them to copy the instructor and learn intuitively. The "ghost" avatar may be displayed in a semi-transparent form in some embodiments, allowing the user to see their own avatar overlapping with or within the "ghost" avatar.

[0054] For example, turning back to FIG. 4B, a virtual reality environment for training and certification is illustrated, according to some embodiments. As shown, the user's avatar may include hands 450, which may be displayed positioned to correspond to tracking of the user's actual hands or a controller held by the hands. Similarly, a "ghost" avatar 452 may be displayed in the environment, following previously recorded tracking information of an instructor's hands or controller, and may be able to perform the same movements as the instructor. During training, a user can easily and intuitively see when they are making a mistake by noting how their movements deviate from the recorded "ghost" avatar.

[0055] The same functionality may be used for authentication purposes by disabling the display of the instructor's "ghost" avatar 452. The training user's movements may still be tracked and compared to the instructor's recorded movements, and in some implementations a score may be generated to determine the amount of deviation of the training user from the instructor's movements.

[0056] For example, FIG. 4C is an exemplary training and certification log for a virtual reality system for training and certification, according to some embodiments. The illustrated embodiment shows events or interactions with objects in a virtual environment, such as buttons and dials, along with the time in the scenario at which the instructor performed the action (e.g., relative to the start of the scenario, rather than absolute time), and the time at which the user interacted with the object. The distance between the position of the training user and the instructor user when the event or interaction was recorded for each may be determined (e.g., the distance between the position or rotation of the user's hand when turning a dial and the position or rotation of the instructor's hand when turning the same dial), and a score may be generated. The score may be calculated inversely proportional to the distance in some embodiments (e.g., such that a smaller distance results in a higher score).

[0057] In some embodiments, the sequence of interactions may be sequenced, i.e., the instructor's interactions may be performed in a particular order, and each user interaction may be compared to a corresponding interaction. This may be useful in some embodiments where a task must be performed in a particular order, or where a particular object interaction may not be recorded. For example, in one such embodiment, if the instructor turns a dial (e.g., dial 2) as the fourth interaction, and the training user turns a different dial (e.g., dial 3) as the fourth interaction, the distance between the instructor's hand position and the training user's hand may be significant (e.g., on an entirely different dial). The distance measurement may automatically account for this. In other embodiments, the particular object or type of interaction interacted may be recorded (e.g., which button was pressed), and a score may be calculated based on the match with the appropriate object or type of interaction in addition to or instead of the distance. Additionally, in some embodiments, the score may be adjusted based on the difference between the recorded relative time of the instructor's interaction and the recorded relative time of the training user's interaction. These times may be calculated relative to the start of the scenario in some implementations (e.g., where a penalty for delay is applied continuously, encouraging the trainee user to speed up to return after a delay), or may be calculated relative to a previous interaction. For example, in the table shown in FIG. 4C, the instructor took 25 seconds to move from the second interaction to the third interaction, and 5 seconds to move from the third interaction to the fourth interaction. For the trainee user, the corresponding interactions took 41 seconds and 5 seconds, respectively. In such implementations using relative timing between subsequent interactions, the trainee user's score may be penalized based on the delay between the second and third interactions, but not based on the delay between the third and fourth interactions (even if the overall time was slower).This can avoid excessively penalizing users.

[0058] The scores may be summed, averaged, or otherwise aggregated and compared to a threshold for purposes of authentication. In some embodiments, if the user's aggregate score falls below a threshold, the virtual scenario may be automatically relaunched, possibly in a training or guided mode, to provide further instruction.

[0059] 4D is a flow chart of a method for virtual reality based training and certification according to some embodiments. When an interaction is detected, values ​​of the interaction may be recorded in step 480 (e.g., relative time, interacted object, object values ​​such as dial settings or switch positions in some embodiments, user position, position of one or both hands of the user, or any other such information to preserve the state of the simulation at the time of the interaction). If the simulation is in record mode (e.g., for an instructor), the values ​​may be saved in the configuration settings in step 482 or saved in the metadata of the object. If the simulation is not in record mode, differences between the previously recorded interaction and the new interaction may be compared in step 484 (e.g., including differences in positions, timing, settings, objects, or any other data). In step 486, a score may be generated based on the difference, such as inversely proportional to the difference such that the more accurate the score, the higher the score. During the simulation and / or once the simulation is completed, in some embodiments, the scores of the interactions may be tallied and compared to a threshold. If the score falls below a threshold, a notification is displayed and the simulation may be automatically restarted, either in the same mode or in the guided mode described above.

[0060] Thus, the systems and methods discussed herein provide dynamic, reconfigurable virtual reality environments with in-environment access to external data and resources. Implementations of these systems also provide external mechanisms to modify other aspects of the virtual reality experience without requiring recoding or compiling the experience. This can change the main flow of the experience, vary behavior based on the particular user accessing it, or add brand or customer specific aspects to the application. The same level or environment can provide vastly different experiences for different users, from novice to expert, and simple configuration also allows for options for random or sequenced events that can be controlled by an instructor or administrator.

[0061] In one aspect, the present disclosure relates to a method for providing a context-sensitive dynamic link in a virtual environment. The method includes a receiving step, in which data is received by a computing device, the data specifying a virtual environment associated with a context and including at least one virtual object, each at least one virtual object being associated with an identifier. The method also includes a searching step, in which the computing device searches from a database for an address of a resource for each at least one virtual object according to the context associated with the virtual environment, the address of the resource being associated with a corresponding identifier. The method also includes a displaying step, in which the computing device displays a view of the virtual environment. The method also includes a detecting step, in which the computing device detects an interaction with a first virtual object by a user of the virtual environment. The method also includes an identifying step, in which the computing device identifies an address of the first resource, the address of the first resource being associated with the identifier of the first virtual object. The method also includes an instantiation step in which an embedded browser or renderer of data for the first resource is instantiated by a computing device within the virtual environment, the embedded browser or renderer retrieving the first resource at the address and displaying the retrieved resource within the virtual environment.

[0062] In some embodiments, the context is associated with a node in the multi-layer hierarchy. In a further embodiment, the first virtual object is associated with a first identifier in a first layer in the multi-layer hierarchy and a second identifier in a second layer in the multi-layer hierarchy, and the search for the address of the first virtual object further includes searching for an address associated with the first identifier in response to the first layer being lower than the second layer. In another further embodiment, the context is associated with a node in each layer in the multi-layer hierarchy, and each node has a parent or child relationship to other nodes of the context in other layers in the multi-layer hierarchy.

[0063] In some embodiments, the address of the first resource comprises an address in a private Uniform Resource Identifier (URI) scheme, the private URI scheme comprising a host identifier, a portal identifier, and a resource identifier. In further embodiments, the instantiation of the embedded browser or renderer further comprises determining whether a copy of the first resource exists in a local storage library corresponding to the portal identifier. In yet another embodiment, the instantiation of the embedded browser or renderer further comprises retrieving a copy of the first resource from a remote storage library corresponding to the host identifier in response to determining that a copy of the first resource does not exist in the local storage library corresponding to the portal identifier.

[0064] In some embodiments, the address of the first resource includes an identifier of the execution action, and the embedded browser or renderer processes the first resource based on the identified execution action. In some embodiments, each virtual object is associated with a set of coordinates in the virtual environment, and the display of the view of the virtual environment further comprises displaying one or more virtual objects in the view at their respective associated coordinates. In further embodiments, each virtual object is associated with a display mode, and the display of the view of the virtual environment further comprises displaying a subset of the virtual objects, the associated display mode corresponding to a current display mode of the virtual environment.

[0065] In another aspect, the disclosure relates to a system for providing context-sensitive dynamic linking in a virtual environment, the system having a computing device with a processor, the processor receiving data, the data specifying a virtual environment associated with a context, including at least one virtual object, each at least one virtual object associated with an identifier, the processor retrieving from a database an address of a resource for each at least one virtual object according to a context associated with the virtual environment, the address of the resource being associated with a corresponding identifier, the processor displaying a view of the virtual environment, the processor detecting an interaction by a user of the virtual environment with a first virtual object, the processor identifying an address of the first resource, the address of the first resource being associated with the identifier of the first virtual object, the processor instantiating an embedded browser or renderer in the virtual environment, the embedded browser or renderer retrieving the first resource at the address and displaying the retrieved resource in the virtual environment.

[0066] In some embodiments, the context is associated with a node in the multi-layer hierarchy. In a further embodiment, the first virtual object is associated with a first identifier in a first layer in the multi-layer hierarchy and a second identifier in a second layer in the multi-layer hierarchy, and the processor further searches for an address associated with the first identifier in response to the first layer being lower than the second layer. In another further embodiment, the context is associated with a node in each layer in the multi-layer hierarchy, and each node has a parent or child relationship to other nodes of the context in other layers in the multi-layer hierarchy.

[0067] In some embodiments, the address of the first resource comprises an address in a private Uniform Resource Identifier (URI) scheme, the private URI scheme comprising a host identifier, a portal identifier, and a resource identifier. In further embodiments, the processor further determines whether a copy of the first resource exists in a local storage library corresponding to the portal identifier. In further embodiments, the processor further retrieves a copy of the first resource from a remote storage library corresponding to the host identifier in response to determining that a copy of the first resource does not exist in the local storage library corresponding to the portal identifier.

[0068] In some embodiments, the address of the first resource includes an identifier of the execution action, and the embedded browser or renderer processes the first resource based on the identified execution action. In some embodiments, each virtual object is associated with a set of coordinates in the virtual environment, and the processor further displays one or more virtual objects in the view at their associated coordinates. In further embodiments, each virtual object is associated with a display mode, and the processor further displays a subset of the virtual objects, where the associated display mode corresponds to a current display mode of the virtual environment.

[0069] In another aspect, the disclosure relates to a method for providing virtual environment based training and certification, the method including: (a) tracking, by a sensor of a computing device, a position of a user in a physical environment, (b) displaying, by the computing device, to the user via a virtual reality display, an avatar corresponding to the user's tracked position in the virtual environment, (c) detecting, by the computing device, an interaction of the avatar with a virtual object in the virtual environment, (d) measuring, by the computing device, a difference between the detected interaction and a predefined interaction associated with the virtual object, and (e) generating, by the computing device, a score inversely proportional to the measured difference.

[0070] In some embodiments, the method includes repeating steps (c)-(e) for the series of interactions and aggregating the generated scores. In further embodiments, the detected series of interactions corresponds to a series of predefined interactions having a predefined order. In further embodiments, the method includes adjusting the aggregated score in response to the detected series of interactions having a different order than the series of predefined interactions. In another further embodiment, the method includes comparing the aggregated score to a threshold and repeating steps (a)-(e) in response to the aggregated score being less than the threshold. In yet another further embodiment, the method includes comparing a time between an initially detected interaction and a subsequently detected interaction to a time between corresponding predefined interactions and adjusting the generated score for the subsequently detected interaction based on the comparison of the times.

[0071] In some embodiments, the method includes measuring a difference between the tracked position of the user and a recorded tracked position of a second user corresponding to the given interaction. In further embodiments, the method includes tracking a hand position of the user and measuring a difference between the tracked hand position of the user and a recorded tracked hand position of the second user.

[0072] In some embodiments, the method includes displaying, within the virtual environment, a predefined interaction associated with the virtual object as a second avatar, hi further embodiments, the method includes recording the predefined interaction while tracking, with a sensor of the computing device, a position of the second user within the physical environment.

[0073] In another aspect, the present disclosure relates to a system for providing virtual environment-based training and certification. The system includes a computing device including at least one sensor and a processor in communication with a virtual reality display. The processor (a) tracks a user's position in a physical environment via the sensor, (b) displays to the user via the virtual reality display an avatar corresponding to the user's tracked position in the virtual environment, (c) detects an interaction of the avatar with a virtual object in the virtual environment, (d) measures a difference between the detected interaction and a predefined interaction associated with the virtual object, and (e) generates a score inversely proportional to the measured difference.

[0074] In some embodiments, the processor further repeats steps (c)-(e) for the series of interactions and aggregates the generated scores. In further embodiments, the detected series of interactions corresponds to a series of predefined interactions having a predefined order. In further embodiments, the processor further adjusts the aggregated score in response to the detected series of interactions having a different order from the series of predefined interactions. In another further embodiment, the processor further compares the aggregated score to a threshold and repeats steps (a)-(e) in response to the aggregated score being less than the threshold. In yet another further embodiment, the processor further compares a time between the initially detected interaction and the subsequently detected interaction with a time between the corresponding predefined interactions and adjusts the generated score for the subsequently detected interaction based on the comparison of the times.

[0075] In some embodiments, the processor further measures a difference between the tracked position of the user and a recorded tracked position of a second user corresponding to the given interaction. In further embodiments, the processor further tracks a hand position of the user and measures a difference between the tracked hand position of the user and a recorded tracked hand position of the second user.

[0076] In some embodiments, the processor further displays the predetermined interaction associated with the virtual object as a second avatar in the virtual environment, hi further embodiments, the processor further records the predetermined interaction while tracking a position of the second user in the physical environment with a sensor of the computing device.

[0077] In another aspect, the disclosure relates to a method for reliably providing a dynamic virtual environment. The method includes displaying, by a web browser application of a computing device, a web page having a selectable link including a private Uniform Resource Identifier (URI) scheme, a host identifier, a portal identifier, and a resource identifier. The method also includes instantiating, by the computing device, a local agent in response to selection of the link. The method also includes determining, by the local agent, that a copy of a first resource corresponding to the resource identifier does not exist in a local storage library at an address corresponding to the portal identifier. The method also includes retrieving, by the local agent, a copy of the first resource from a remote storage library corresponding to the host identifier in response to determining that a copy of the first resource does not exist in the local storage library corresponding to the portal identifier. The method also includes extracting, by the local agent, configuration information for the virtual environment. The method also includes launching, by the local agent, the virtual environment according to the extracted configuration information.

[0078] In some embodiments, the method includes sending a query to a database server including an identification of metadata associated with the virtual object, and receiving a web page for display from the database server, the web page being generated by the database server in response to the identification of the metadata associated with the virtual object in the configuration information of the virtual environment corresponding to the portal identifier and the resource identifier.

[0079] In some embodiments, the method includes decrypting a copy of the retrieved first resource. In a further embodiment, the method includes providing authentication information associated with the computing device to a remote server. In another further embodiment, the copy of the retrieved first resource is decrypted in active memory and flushed from active memory after termination of the virtual environment. In yet another embodiment, the copy of the retrieved encrypted first resource is stored undecrypted in a local storage library after termination of the virtual environment.

[0080] In some embodiments, the configuration information includes metadata for a subset of the virtual objects in the virtual environment, and the method includes, for each virtual object, determining whether the configuration information includes metadata for the virtual object or does not include metadata for the virtual object, and respectively either adding metadata from the configuration information for the virtual object to the virtual environment and adding default metadata associated with one of a host identifier or a portal identifier for the virtual object to the virtual environment. In a further embodiment, the metadata for the virtual object includes a URI scheme, a host identifier, a portal identifier, and a second resource identifier corresponding to a second resource that includes information about the virtual object. In another further embodiment, the method includes displaying each virtual object within the virtual environment. In some embodiments, the method includes registering a private URI scheme with an operating system of the computing device as associated with a local agent, and instantiating the local agent in response to generation of a request by the web browser using the private URI scheme.

[0081] In another aspect, the present disclosure relates to a system for reliably providing a dynamic virtual environment. The system includes a computing device including a memory device storing a local storage library, a network interface, and a processor. The processor displays, via a web browser application, a web page having a selectable link including a private Uniform Resource Identifier (URI) scheme, a host identifier, a portal identifier, and a resource identifier, instantiates a local agent in response to selection of the link, determines, via the local agent, that a copy of a first resource corresponding to the resource identifier does not exist in the local storage library at an address corresponding to the portal identifier, retrieves, via the network interface, a copy of the first resource from a remote storage library corresponding to the host identifier in response to determining that a copy of the first resource does not exist in the local storage library corresponding to the portal identifier, extracts, via the local agent, configuration information for the virtual environment, and launches, by the local agent, the virtual environment according to the extracted configuration information.

[0082] In some embodiments, the processor further transmits a query including the identification of metadata associated with the virtual object to a database server via the network interface and receives a web page for display from the database server, the web page being generated by the database server in response to the identification of metadata associated with the virtual object in the configuration information of the virtual environment corresponding to the portal identifier and the resource identifier.

[0083] In some embodiments, the processor further decrypts the copy of the retrieved first resource. In a further embodiment, the processor further provides authentication information associated with the computing device to the remote server. In another further embodiment, the copy of the retrieved first resource is decrypted in the active memory and flushed from the active memory after termination of the virtual environment. In yet another embodiment, the copy of the retrieved encrypted first resource is stored undecrypted in a local storage library after termination of the virtual environment.

[0084] In some embodiments, the configuration information includes metadata for a subset of the virtual objects in the virtual environment, and the processor further determines, for each virtual object, whether the configuration information includes virtual object metadata or does not include virtual object metadata, and respectively either adds metadata from the virtual object's configuration information to the virtual environment and adds default metadata associated with one of the virtual object's host identifier or portal identifier to the virtual environment. In a further embodiment, the virtual object's metadata includes a URI scheme, a host identifier, a portal identifier, and a second resource identifier corresponding to a second resource that includes information about the virtual object. In another further embodiment, the processor further displays each virtual object within the virtual environment. In some embodiments, the processor further registers the private URI scheme with an operating system of the computing device as being associated with the local agent, and instantiates the local agent in response to generation of a request by the web browser using the private URI scheme.

[0085] (B. Computing Environment) Having described specific embodiments of the present solution, it may be beneficial to describe aspects of the operating environment, along with associated system components (e.g., hardware elements), in connection with the methods and systems described herein.

[0086] The systems discussed herein may be deployed as and / or executed on any type and form of computing device, such as a computer, network device, or appliance, capable of communicating over any type and form of network and performing the operations described herein. Figures 6A and 6B show block diagrams of a computing device 600 useful for implementing an embodiment of a wireless communication device 602 or an access point 606. As shown in Figures 6A and 6B, each computing device 600 is useful for implementing an embodiment of a wireless communication device 602 or an access point 606. As shown in Figures 6A and 6B, each computing device 600 includes a central processing unit 621 and a main memory unit 22. As shown in Figure 6A, the computing device 600 may include a storage device 628, an installation device 616, a network interface 618, an I / O controller 623, display devices 624a-624n, a keyboard 626, and a pointing device 627, such as a mouse. The storage device 628 may include, but is not limited to, an operating system and / or software. As shown in FIG. 6B, each computing device 600 may also include additional optional elements, such as a memory port 603, a bridge 670, one or more input / output devices 630a-630n (collectively referred to with reference numeral 630), and a cache memory 640 in communication with the central processing unit 621.

[0087] Central processing unit 621 is any logic circuitry that responds to and processes instructions fetched from main memory unit 622. In many embodiments, central processing unit 621 is provided by a microprocessor unit such as those manufactured by Intel Corporation of Mountain View, Calif., International Business Machines, Inc. of White Plains, New York, or Advanced Micro Devices, Inc. of Sunnyvale, Calif. Computing device 600 may be based on these processors or any other processor capable of operating as described herein.

[0088] The main memory unit 622 may be one or more memory chips that store data and whose memory locations can be directly accessed by the microprocessor 621, such as any type or variation of static random access memory (SRAM), dynamic random access memory (DRAM), ferroelectric RAM (FRAM®), NAND flash, NOR flash, and solid-state drive (SSD). The main memory 622 may be based on any of the memory chips mentioned above or other available memory chips operable as described herein. In the embodiment shown in FIG. 6A, the processor 621 communicates with the main memory 622 via a system bus 650 (described in more detail below). FIG. 6B illustrates an embodiment of a computing device 600 in which the processor communicates directly with the main memory 622 via a memory port 603. For example, in FIG. 6B, the main memory 622 may be a DRDRAM.

[0089] FIG. 6B illustrates an embodiment in which the main processor 621 communicates directly with the cache memory 640 via a secondary bus, sometimes referred to as a backside bus. In other embodiments, the main processor 621 communicates with the cache memory 640 using a system bus 650. The cache memory 640 typically has a faster response time than the main memory 622 and is provided by, for example, SRAM, BSRAM, or EDRAM. In the embodiment illustrated in FIG. 6B, the processor 621 communicates with various I / O devices 630 via a local system bus 650. Various buses may be used to connect the central processing unit 621 to any of the I / O devices 630, such as, for example, a VESA VL bus, an ISA bus, an EISA bus, a MicroChannel Architecture (MCA) bus, a PCI bus, a PCI-X bus, a PCI-Express bus, or a NuBus. For an embodiment in which the I / O device is a video display 624, the processor 621 may use an Advanced Graphics Port (AGP) to communicate with the display 624. Figure 6B illustrates an embodiment of a computer 600 in which a main processor 621 may communicate directly with an I / O device 630b, for example, via HYPERTRANSPORT, RAPIDIO, or INFINIBAND communications technologies. Figure 6B also illustrates a mixed local bus and direct communication embodiment in which processor 621 communicates with an I / O device 630a using a local interconnect bus while also communicating directly with an I / O device 630b.

[0090] A wide variety of I / O devices 630a-630n may be present in computing device 600. Input devices include keyboards, mice, trackpads, trackballs, microphones, dials, touchpads, touch screens, and drawing tablets. Output devices include video displays, speakers, inkjet printers, laser printers, projectors, and dye-sublimation printers. The I / O devices may be controlled by an I / O controller 623, as shown in FIG. 6A. The I / O controller may control one or more I / O devices, such as a keyboard 626 and a pointing device 627, such as a mouse or optical pen. Additionally, the I / O devices may provide storage and / or installation media 616 for computing device 600. In yet other embodiments, computing device 600 may provide a USB connection (not shown) for receiving a handheld USB storage device, such as the USB flash drive line of devices manufactured by Twintech Industry, Inc. of Los Alamitos, Calif.

[0091] Referring again to FIG. 6A, the computing device 600 may correspond to any suitable installation device 616, such as a disk drive, a CD-ROM drive, a CD-R / RW drive, a DVD-ROM drive, a flash memory drive, various types of tape drives, a USB device, a hard drive, a network interface, or any other device suitable for installing software and programs. The computing device 600 may further include a storage device, such as one or more hard disk drives or a redundant array of independent disks, for storing an operating system and other related software, and for storing application software programs, such as any program or software 620 for implementing (e.g., configuring and / or designing) the systems and methods described herein. Optionally, any of the installation devices 616 may also be used as a storage device. Additionally, the operating system and software may be executed from a bootable medium.

[0092] Additionally, computing device 600 may include a network interface 618 for interfacing to network 604 via a variety of connections, including, but not limited to, a standard telephone line, a LAN or WAN link (e.g., 802.11, T1, T3, 56kb, X.25, SNA, DECNET), a broadband connection (e.g., ISDN, Frame Relay, ATM, Gigabit Ethernet, Ethernet-over-SONET), a wireless connection, or a combination of any or all of the above. Connections can be established using a variety of communication protocols (e.g., TCP / IP, IPX, SPX, NetBIOS, Ethernet, ARCNET, SONET, SDH, Fiber Distributed Data Interface (FDDI), RS232, IEEE 802.11, IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n, IEEE 802.11ac, IEEE 802.11ad, CDMA, GSM, WiMax, and direct asynchronous connections). In one embodiment, computing device 600 communicates with other computing devices 600' via any type and / or form of gateway or tunneling protocol, such as Secure Sockets Layer (SSL) or Transport Layer Security (TLS). Network interface 618 may include a built-in network adapter, a network interface card, a PCMCIA network card, a card bus network adapter, a wireless network adapter, a USB network adapter, a modem, or any other device suitable for interfacing computing device 600 to any type of network capable of communicating and performing the operations described herein.

[0093] In some embodiments, the computing device 600 may include or be connected to one or more display devices 624a-624n. As such, any of the I / O devices 630a-630n and / or I / O controller 623 may include any type and / or form of suitable hardware, software, or combination of hardware and software to accommodate, enable, or provide for the connection and use of the display devices 624a-624n by the computing device 600. For example, the computing device 600 may include any type and / or form of video adapter, video card, drivers, and / or libraries to interface with, communicate with, connect to, or otherwise use the display devices 624a-624n. In one embodiment, the video adapter may include multiple connectors for interfacing to the display devices 624a-624n. In other embodiments, the computing device 600 may include multiple video adapters, each video adapter connected to a display device 624a-624n. In some embodiments, any portion of the operating system of the computing device 600 may be configured to use multiple display devices 624a-624n. Those skilled in the art will recognize and understand the various ways and embodiments in which the computing device 600 may be configured to have one or more display devices 624a-624n.

[0094] In further embodiments, I / O device 630 may be a bridge between system bus 650 and an external communications bus, such as a USB bus, an Apple Desktop bus, an RS-232 serial connection, a SCSI bus, a FireWire® bus, a FireWire® 800 bus, an Ethernet bus, an AppleTalk® bus, a Gigabit Ethernet bus, an Asynchronous Transfer Mode bus, a Fibre Channel bus, a serial connected Small Computer System Interface bus, a USB connection, or an HDMI® bus.

[0095] A computing device 600 of the type depicted in Figures 6A and 6B may operate under the control of an operating system that controls the scheduling of tasks and access to system resources. Computing device 600 may run any operating system, such as any version of the MICROSOFT WINDOWS operating system, different releases of the Unix and Linux operating systems, any version of MAC OS for Macintosh computers, any embedded operating system, any real-time operating system, any open source operating system, any proprietary operating system, any operating system for mobile computing devices, or any other operating system capable of running on a computing device and performing the operations described herein. Representative operating systems include, but are not limited to, Android (registered trademark) from Google, Inc., WINDOWS (registered trademark) 8 and 10 from Microsoft Corporation of Redmond, Washington, MAC OS (registered trademark) from Apple Computer, Inc. of Cupertino, California, WebOS from Research In Motion (RIM), OS / 2 from International Business Machines, Inc. of Armonk, New York, and Linux (registered trademark), a freely available operating system distributed by Caldera, Inc. of Salt Lake City, Utah, or any type and / or form of a Unix (registered trademark) operating system.

[0096] Computer system 600 may be a workstation, a telephone, a desktop computer, a laptop or notebook computer, a server, a handheld computer, a mobile phone or other portable telecommunications device, a media playback device, a gaming system, a mobile computing device, or any other type and / or form of computing, telecommunications, or media device capable of communicating. Computer system 600 has sufficient processor power and memory capacity to perform the operations described herein.

[0097] In some embodiments, computing device 600 may have different processors, operating systems, and input devices consistent with the device. For example, in one embodiment, computing device 600 is a smartphone, a mobile device, a tablet, or a personal digital assistant. In yet other embodiments, computing device 600 is a Blackberry or WebOS based handheld device or smartphone, such as an Android® based mobile device, an iPhone® smartphone manufactured by Apple Computer, Inc. of Cupertino, Calif., or a device manufactured by Research In Motion, Inc. Additionally, computing device 600 may be a workstation, desktop computer, laptop or notebook computer, server, handheld computer, mobile phone, other computer, or other form of computing or telecommunications device capable of communications and having sufficient processor power and memory capacity to perform the operations described herein.

[0098] This disclosure may refer to one or more "users," which may refer to user-associated devices or stations (STAs), for example, consistent with the terms "user" and "multi-user" commonly used in the context of a multi-user multiple-input and multiple-output (MU-MIMO) environment.

[0099] While the example communication systems described above may include devices and APs operating according to the 802.11 standard, it should be understood that embodiments of the systems and methods described above may operate according to other standards and use wireless communication devices other than devices configured as APs. For example, multi-unit communication interfaces associated with cellular networks, satellite communications, vehicular communications networks, and other non-802.11 wireless networks may utilize the systems and methods described herein to achieve improved overall capacity and / or link quality without departing from the scope of the systems and methods described herein.

[0100] It should be noted that certain portions of this disclosure may refer to terms such as "first" and "second" in connection with devices, modes of operation, transmit chains, antennas, etc., for the purpose of identifying or distinguishing one from another or from others. These terms are not intended to associate entities (e.g., a first device and a second device) merely in time or sequence, although in some cases such a relationship may be included among the entities. Additionally, these terms are not intended to limit the number of entities (e.g., devices) that may operate within a system or environment.

[0101] It should be understood that the above-described system may provide multiples of any or each of these components, and that these components may be provided on either a standalone machine or, in some embodiments, on multiple machines in a distributed system. Additionally, the above-described systems and methods may be provided as one or more computer readable programs or executable instructions embodied on or in one or more articles of manufacture. The articles of manufacture may be floppy disks, hard disks, CD-ROMs, flash memory cards, PROMs, RAMs, ROMs, or magnetic tapes. In general, computer readable programs may be implemented in any programming language, such as LISP, PERL, C, C++, C#, PROLOG, or any bytecode language, such as JAVA. The software programs or executable instructions may be stored in one or more articles of manufacture as object code.

[0102] While the foregoing written description of the methods and systems will enable one of ordinary skill in the art to make and use what is presently believed to be the best mode, one of ordinary skill in the art will understand and appreciate the existence of variations, combinations, and equivalents to the specific embodiments, methods, and examples herein. Thus, the methods and systems should not be limited by the embodiments, methods, and examples described above, but rather by all embodiments and methods within the scope and spirit of the present disclosure.

Claims

1. A method for reliably providing a dynamic virtual environment, comprising: displaying, by a web browser application on the local computing device, a web page having selectable links including a private Uniform Resource Identifier (URI) scheme, a host identifier, a portal identifier, and a resource identifier; the host identifier is associated with a remote storage library stored in a memory of a remote computing device; the portal identifier is associated with a local storage library stored in a memory of the local computing device; instantiating, by the local computing device, a local agent in response to selection of the link; determining, by the local agent, that a copy of a first resource corresponding to the resource identifier does not exist in the local storage library at an address corresponding to the portal identifier; retrieving, by the local agent, a copy of the first resource from the remote storage library associated with the host identifier in response to determining that a copy of the first resource does not exist within the local storage library corresponding to the portal identifier; extracting, by the local agent, configuration information for a virtual environment; and initiating, by the local agent, the virtual environment in accordance with the extracted configuration information. method.

2. In the method according to claim 1, The step of displaying the web page further comprises: sending a query to a database server, the query including an identification of metadata associated with the virtual object; receiving the web page for display from the database server, the web page being generated by the database server in response to identifying the metadata associated with the virtual object in configuration information for a virtual environment that corresponds to the portal identifier and the resource identifier. method.

3. In the method according to claim 1, and extracting the configuration information further comprises decrypting the retrieved copy of the first resource. method.

4. In the method according to claim 3, The method further comprises providing authentication information associated with the local computing device to the remote computing device. method.

5. In the method according to claim 3, the retrieved copy of the first resource is decrypted in active memory and flushed from the active memory after termination of the virtual environment. method.

6. In the method according to claim 5, the retrieved encrypted copy of the first resource is stored undecrypted in the local storage library after the virtual environment is terminated; method.

7. A method for reliably providing a dynamic virtual environment, comprising: displaying, by a web browser application on the computing device, a web page having selectable links including a private Uniform Resource Identifier (URI) scheme, a host identifier, a portal identifier, and a resource identifier; instantiating, by the computing device, a local agent in response to selection of the link; determining, by the local agent, that a copy of a first resource corresponding to the resource identifier does not exist in a local storage library at an address corresponding to the portal identifier; retrieving, by the local agent, a copy of the first resource from a remote storage library corresponding to the host identifier in response to determining that a copy of the first resource does not exist within the local storage library corresponding to the portal identifier; extracting, by the local agent, configuration information for a virtual environment, the configuration information including metadata for a subset of virtual objects in the virtual environment, the extracting of configuration information further including, for each virtual object: determining whether the configuration information includes metadata for the virtual object or does not include metadata for the virtual object; adding metadata from the configuration information of the virtual object to the virtual environment, and adding default metadata associated with one of the host identifier or the portal identifier of the virtual object to the virtual environment, respectively, the metadata including one or more key-value pairs for parameters of the virtual object; and initiating, by the local agent, the virtual environment in accordance with the extracted configuration information. method.

8. The method according to claim 7, the metadata for a virtual object includes the private URI scheme, the host identifier, the portal identifier, and a second resource identifier corresponding to a second resource that includes information about the virtual object; method.

9. The method of claim 7, further comprising: displaying each virtual object within the virtual environment; method.

10. The method according to claim 7, The step of instantiating the local agent further comprises: registering the private URI scheme associated with the local agent with an operating system of the computing device; and instantiating the local agent using the private URI scheme in response to a request being generated by the web browser application. method.

11. A system for reliably providing a dynamic virtual environment, the system comprising: a local computing device including a memory device storing a local storage library, a network interface, and a processor; The processor, displaying, via a web browser application, a web page having selectable links including a private Uniform Resource Identifier (URI) scheme, a host identifier, a portal identifier, and a resource identifier, wherein: the host identifier is associated with a remote storage library stored in a memory of a remote computing device; the portal identifier is associated with the local storage library stored in the memory device of the local computing device; instantiating a local agent in response to selection of the link; determining, via the local agent, that a copy of a first resource corresponding to the resource identifier does not exist within the local storage library at an address associated with the portal identifier; in response to determining that a copy of the first resource does not exist within the local storage library corresponding to the portal identifier, retrieving, via the network interface, a copy of the first resource from the remote storage library associated with the host identifier; Extracting configuration information for a virtual environment via the local agent; configured to launch the virtual environment by the local agent in accordance with the extracted configuration information; system.

12. In the system according to claim 11, The processor further comprises: Transmitting a query to a database server via the network interface, the query including an identification of metadata associated with a virtual object; and configured to receive from the database server the web page for display generated by the database server in response to identifying the metadata associated with the virtual object in configuration information of a virtual environment corresponding to the portal identifier and the resource identifier. system.

13. The system according to claim 11, The processor is further configured to decrypt the retrieved copy of the first resource. system.

14. In the system according to claim 13, The processor is further configured to provide authentication information associated with the local computing device to the remote computing device. system.

15. The system according to claim 13, the retrieved copy of the first resource is decrypted in active memory and flushed from the active memory after termination of the virtual environment. system.

16. The system according to claim 15, the retrieved encrypted copy of the first resource is stored undecrypted in the local storage library after the virtual environment is terminated; system.

17. The system according to claim 11, The configuration information includes metadata for a subset of virtual objects in the virtual environment, and for each virtual object, the processor further: determining whether the configuration information includes metadata for the virtual object or does not include metadata for the virtual object; and adding metadata from the configuration information of the virtual object to the virtual environment, and adding default metadata associated with one of the host identifier or the portal identifier of the virtual object to the virtual environment, respectively. system.

18. The system according to claim 17, the metadata for a virtual object includes the private URI scheme, the host identifier, the portal identifier, and a second resource identifier corresponding to a second resource that includes information about the virtual object; system.

19. The system according to claim 17, The processor is further configured to display each virtual object within the virtual environment. system.

20. The system according to claim 11, The processor is further configured to register the private URI scheme associated with the local agent with an operating system of the local computing device, and instantiate the local agent using the private URI scheme in response to generation of a request by the web browser application. system.

Images