Interactive data immersion and collaboration system and related methods

Abstract

An interactive data immersion and collaboration system comprising an augmented reality (AR) or virtual reality (VR) headwear (e.g., glasses, goggles, headset, etc.) and a computer software application configured to display to the user a dynamic, interactive presentation of data or data sets in a virtual environment that appears to surround the user and that enables a user to configure, arrange, and / or edit the data in a variety of ways.

Description

S1I-001-W01Interactive Data Immersion and Collaboration System and Related MethodsCROSS-REFERENCES TO RELATED APPLICATIONS

[0001] The present application claims the benefit of priority under Article 8 of the PCT to U.S. Provisional Application Serial No. 63 / 642,817, filed 5 May 2024 and entitled “Innovation Discovery System and Technology Platform Powered by Technology Repurposing, Combinatorial Innovation and a New Paradigm for Virtualized Command-and-Control Interfaces and Associated Decision Aids."FIELD

[0002] The present disclosure relates generally to spatial computing, and more particularly to a system and method for interactive data immersion to enable efficient collaboration, local and remote monitoring, and review.BACKGROUND

[0003] We live in a world awash with data. Ordinary citizens have access to a wealth of sensors, live data streams, and information at a global scale. We use that access almost daily - often unknowingly - to accomplish tasks that have come to feel routine like checking the weather, deciding on which route to take based on real-time traffic information, or tracking a package across the world.

[0004] Commercial and government organizations rely on high throughput streams of data for a wide array of purposes too. Some examples include tracking fleets of vehicles; monitoring parks or traffic; and monitoring power distribution and troubleshooting blackouts. Having grown used to this abundance of data, an increasing number of commercial and military organizations are reimagining their operations and deploying an increasingly agile, highly adaptable workforce, equipped with numerous instruments and monitoring platforms and assisted by unmanned, highly instrumented vehicles.

[0005] For example, the U.S. military has released several plans promoting a reimagining of the fleet as a highly adaptable force. This vision for rapid adaptation is often centered around the use of new rapid manufacturing technologies, unmanned vehicles, and new sensing capabilitiesS1I-001-W01 and it is inspired by the advent of small and agile technology platforms like drones, unmanned vehicles, etc. In this imagined adaptable fleet, commanders too will be expected to make decisions differently: evaluating multiple, changing streams of real-time data against other kinds of information - often before the problem is fully understood; all at a high tempo, collaboratively, and integrating tactical and strategic information wherever they are.

[0006] Many components are required to fulfdl those aspirations and assess, adapt, and develop action plans in response to a rapidly changing environment in a manner consistent with that vision. Those components include real-time field sensor data management; data fusion; systems to discover, test and deploy technologies before problems and requirements are fully understood; development of action plans live; remote collaboration; and even control of autonomous vehicles - or swarms of them - remotely.

[0007] The widespread aspiration for rapid adaptation and rapid deployment of innovation implies a need for a highly sophisticated and equally adaptable command-and-control interface that does not exist yet. The full set of expectations surrounding truly adaptive operations can only be met with a new paradigm for virtualized command-and-control interfaces and decisionaids.

[0008] The opportunity is to develop a new paradigm for a command-and-control interface that increases adaptability, sets the stage for information fusion, reduces response time, and enables strategic decision making to match the expectations of adaptability of modem operations. We envision a command-and-control interface that marries the strategic value of the war rooms that became popular in the 1980s with access to real-time data. Access to real time data and the capability to arrange information against a specific strategic or tactical objective turns the war room into a fully customizable real-time cockpit - to assess an evolving problem in real time, formulate strategy also in real time, monitor progress, develop action plans, and adapt. The holistic view of the problem afforded by this command-and-control interface will create new opportunities, such as the opportunity for discovery systems that accelerate transition of technology to operations by making technology discoverable by purpose.

[0009] The broader opportunity is to develop a means for people using this interface to engage with real-time data and collaborate with others in a manner that permits continualS1I-001-W01 iteration around information, problem monitoring, exploration of open-ended problems, development of action plans, and remote collaboration.

[0010] A set of new technologies - combinatorial innovation, 2ndgeneration augmented reality devices, technology repurposing, and machine learning - make it possible now to build an interface that achieves all this and upgrades the concept of war room often used in strategy.

[0011] This reimagining of the command-and-control interface is timely and has broad applicability. For instance, it is relevant to efforts like the US Navy Replicator initiative that require a step-up in information and data management interfaces to fulfdl the desire for rapid adaptation. Additionally, it can be immediately relevant in highly instrumented sensing environments like ranges, control rooms. Further, it may open up new applications for control rooms to application domains in which maintaining an old-fashioned, physical control room was prohibitively costly. Moreover, it hints at an opportunity to develop new, experiential STEM education programs like new engineering capstones centered on real-world problems. Furthermore, it may be extended as an aid in discovering technologies that may be repurposed for use in new environments - for example, uncovering new dual use opportunities inside or outside the government agencies for technologies researched with taxpayer dollars.

[0012] These opportunities are relevant across the public and private sectors. Additionally, the last two items suggest a new path to support STEM objectives, materialize the dual use ideal, and empower enterprising talent.

[0013] Viewing and editing multi-page documents, or multiple documents at the same time is a task that is necessary in many areas of business and / or academia. However, this can be very time consuming as traditional electronic document interfaces allow users to display information in one computer window at a time. These windows are independent from each other, and each displays a single document. For example, Fig. 1 illustrates a traditional configuration in which a user 2 has multiple windows 4 open on a single computer display screen 6. This setup is functional and can be portable (e.g., especially with a laptop or tablet computer) but the number and size of windows is limited, potentially making the work more complicated. Composing a multi-document or cross-document view to track a project and coordinate multiple documents may require special programming or configurations.S1I-001-W01

[0014] Physical world control rooms have multiple screens dedicated to specific pieces of information according to some pre-specified designs. For example, Fig. 2 illustrates a traditional configuration in which a user 2 has multiple windows 4 open on a multiple computer display screens 6. This can be useful in that this configuration allows multiple pages and / or documents and / or file types to be displayed on dedicated screens which are presumably large enough to be viewed comfortably. This makes collaboration easier since more people are able to view the larger display screens at the same time. However, the drawback to this setup is the amount of dedicated wall space needed to accommodate a large number of screens, and portability is not possible.

[0015] Furthermore, in each of these environments, the user is separated from the information being displayed. A need exists for a system that immerses users in relevant data, making the data easy to sort through, separate, and / or recombine into new combinations. The aspiration for rapid adaptation and rapid deployment of innovation implies a need for a highly sophisticated and equally adaptable command-and-control interface that does not exist yet.SUMMARY

[0016] The present disclosure addresses the shortcomings of the prior art by describing an interactive data immersion and collaboration system that systematizes technology repurposing, combinatorial innovation, and includes a general family of novel tools to move technologies to impact and solve problems with technology. The system and related methods disclosed herein is a new paradigm and innovation interface to capitalize on the opportunity outlined above.

[0017] Disclosed herein is a system and associated methods that leverage augmented reality to immerse users in data in a dynamic environment. A system that leverages highly interactive augmented reality paradigms (such as the spatial computing paradigm of Apple Vision Pro™ AR goggles or equivalent) to immerse users in data, enable informed decision-making, remote and in -person collaboration, and remote control of devices to name some. Depending on the configuration, the environment (or “room” or “virtual room” or “virtual project room” or “virtual command and control center”) can fully replace all kind of highly instrumented environments like control rooms and command and control interfaces, it can also be configured as aS1I-001-W01 collaboration environment, a design environment, a discovery system, a document viewing space, a presentation environment and a project planning and monitoring space.

[0018] In some embodiments of a virtualized command and control interface, the present disclosure describes an augmented or virtual reality environment that allows people to immerse themselves in real-time data from multiple sensor systems and control how data is displayed. This illustrates the opportunity for a virtualized, adaptable command-and-control interface - the “upgraded war room concept” outlined above. The augmented reality or virtual environment may be accessed through off-the-shelf interfaces and be populated with data displays that retain their configuration across sessions.

[0019] In some embodiments, the system disclosed herein virtualizes a command-control- interface and introduces new uses and possibilities that emerge from the virtualization of sensing environments. Broadly speaking, those elements fall in two broad categories: the interface layer and the application layer. By way of example, the interface layer includes elements like the development of an environment suitable for the integration of multiple sensor feeds, strategies and gestures to manipulate those feeds and documents effectively in augmented reality, access control, systems for remote collaboration, and alert-systems to name some. These elements are reasonably well understood in two dimensional environments but there’s no suitable standard for them in spatial computing. By way of example, the application layer includes elements like development of process technologies to help explore full problems iteratively, templates to rapidly deploy standard “rooms”, and the exploration of several machine learning application layers such as the aforementioned discovery system to make technologies findable by purpose.

[0020] The interactive data immersion and collaboration system disclosed herein is capable of displaying various kinds of static and live data around the user. The information may be organized and presented to the user as virtual data panels. The panels may be arranged in different geometries and rearranged to configure a highly customizable and personalized command and control interface, monitoring station, discovery system. The system is also capable of two-way communication with external devices and may be used in isolation or in collaboration with other users remotely or in the same physical space.S1I-001-W01

[0021] In some embodiments, the room may be equipped with mechanisms to track activity for analysis with machine learning algorithms.

[0022] In some embodiments, the interactive data immersion and collaboration system disclosed herein is a collaboration and discovery system that enables people from different domains of expertise to use the tools described herein for communication, collaboration, and joint decision-making.

[0023] In some embodiments, the interactive data immersion and collaboration system disclosed herein is a human-computer interface for data fusion and discovery in which users can develop action plans for anything from mission control to startup development.

[0024] In some embodiments, the interactive data immersion and collaboration system disclosed herein is a training system that helps users make the most of this capability and the associated tools.

[0025] In some embodiments, the interactive data immersion and collaboration system disclosed herein is a new paradigm for a command-and-control interface that increases adaptability, sets the stage for information fusion, reduces response time, and enables strategic decision making to match the expectations of business, government, and / or academia.

[0026] In some embodiments, the interactive data immersion and collaboration system disclosed herein empowers people using this interface to engage with real-time data and collaborate with others in a manner that permits continual iteration around information, problem monitoring, exploration of open-ended problems, development of action plans, and remote collaboration.

[0027] In some embodiments, the interactive data immersion and collaboration system disclosed herein includes a command-and-control interface that marries the strategic value of a war room “of old” with access to real-time data. Access to real time data and the capability to arrange information against a specific strategic or tactical objective turns the room into a fully customizable real-time cockpit — to assess an evolving problem in real time, formulate strategy also in real time, monitor progress, develop action plans, and adapt.S1I-001-W01

[0028] In some embodiments, the interactive data immersion and collaboration system disclosed herein presents a user with a virtual reality environment that allows people to immerse themselves in real-time data from multiple sensor systems and control how data is displayed. This illustrates the opportunity for a virtualized, adaptable command-and-control interface. The virtual environment may be accessed through off-the-shelf interfaces, such as the Apple Vision Pro™ AR goggles, and be populated with data displays that retain their configuration across sessions.

[0029] In some embodiments, the system virtualizes a command-control-interface and introduces new uses and possibilities that emerge from the virtualization of sensing environments. Broadly speaking, those elements fall in two broad categories: the interface layer and the application layer. By way of example, the interface layer includes elements like the development of an environment suitable for the integration of multiple sensor feeds, strategies and gestures to manipulate those feeds and documents effectively in augmented reality, access control, systems for remote collaboration, and alert-systems to name some. These elements are reasonably well understood in two dimensional environments but there’s no suitable standard for them in spatial computing. By way of example, the application layer includes elements like development of process technologies to help explore full problems iteratively, templates to rapidly deploy standard “rooms”, and the exploration of several machine learning application layers such as the aforementioned discovery system. This work brings together two bodies of expertise: our proprietary system for problem prototyping, combinatorial innovation, and technology repurposing; and a set of technologies that enable real-time integration of data across multiple modalities to help users create and explore options and develop action plans.

[0030] As additional disclosure to the embodiments described herein, the present disclosure describes the following embodiments.

[0031] Embodiment 1 is an interactive data immersion and collaboration system, comprising: a wearable device configured to render and display augmented reality within a user’s field of vision, the wearable device configured to be worn by a user, the wearable device comprising: a processor; a memory comprising a non-transitoiy computer readable medium; and a display unit; and a computer program product embodied in said non-transitory computer readable medium andS1I-001-W01 comprising a set of computer instructions that, when executed by the processor, cause the wearable device to: generate an augmented reality room comprising at least one virtual data panel oriented in physical space at least partially around the user; and display, on the display unit, an interactive presentation of data wherein the wearable device is further configured to: receive, based on the user’s interaction with said data presented on said at least one virtual data panel, user instructions to edit or rearrange said data; and display, on the display unit, a real-time rendering of the edited or rearranged data on said at least one virtual display panel in response to said user instructions to edit or rearrange said data.

[0032] Embodiment 2 is the interactive data immersion and collaboration system of embodiment 1, wherein said at least one virtual data panel comprises a plurality of virtual data panels positioned in a geometric arrangement.

[0033] Embodiment 3 is the interactive data immersion and collaboration system of embodiments 1 or 2, wherein the geometric arrangement is circular, spiral, continuous, open, or spherical.

[0034] Embodiment 4 is the interactive data immersion and collaboration system of any of embodiments 1 through 3, wherein the geometric arrangement includes at least one gap to simulate a door in the augmented reality room.

[0035] Embodiment 5 is the interactive data immersion and collaboration system of any of embodiments 1 through 4, wherein the augmented reality room is moveable in response to user hand or arm motion.

[0036] Embodiment 6 is the interactive data immersion and collaboration system of any of embodiments 1 through 5, wherein the at least one virtual data panel is independently movable in response to user hand or arm motion.

[0037] Embodiment 7 is the interactive data immersion and collaboration system of any of embodiments 1 through 6, wherein the data of said interactive presentation of data includes a single document, multiple documents, a video file, a real-time video feed, an image file, an external data source, a control actuator, a different data file, real-time sensor data, or access to a different augmented reality room.S1I-001-W01

[0038] Embodiment 8 is the interactive data immersion and collaboration system of any of embodiments 1 through 7, wherein the control actuator is configured to facilitate real-time control of an external controllable object.

[0039] Embodiment 9 is the interactive data immersion and collaboration system of any of embodiments 1 through 8, wherein the external controllable object is a drone, rover, or game piece.

[0040] Embodiment 10 is the interactive data immersion and collaboration system of any of embodiments 1 through 9, wherein the wearable device is further configured to register the augmented reality room to a fixed position and orientation within a real space.

[0041] Embodiment 11 is the interactive data immersion and collaboration system of any of embodiments 1 through 10, wherein the augmented reality room is configured for sharing with a second user present in the physical space and having a second wearable device configured to render and display augmented reality within the second user’s field of vision, the second wearable device comprising a second processor, a second memory stored on a second non- transitory computer readable medium, and a second display unit, the second wearable device configured to be worn by the second user.

[0042] Embodiment 12 is the interactive data immersion and collaboration system of any of embodiments 1 through 11, wherein the shared augmented reality room is registered in the same position and orientation for the user and the second user.

[0043] Embodiment 13 is the interactive data immersion and collaboration system of any of embodiments 1 through 12, wherein the wearable device is further configured to display, on the display unit, a real-time rendering of the second user’s interactions with the shared augmented reality room.

[0044] Embodiment 14 is the interactive data immersion and collaboration system of any of embodiments 1 through 13, wherein the second wearable device is further configured to display, on the second display unit, a real-time rendering of the user’s interactions with the shared augmented reality room.S1I-001-W01

[0045] Embodiment 15 is the interactive data immersion and collaboration system of any of embodiments 1 through 14, wherein the wearable device is further configured to register the augmented reality room to a position and orientation within a real space relative to the user.

[0046] Embodiment 16 is the interactive data immersion and collaboration system of any of embodiments 1 through 15, wherein the augmented reality room is configured for sharing with a second user present in a remote second physical space and having a second wearable device configured to render and display augmented reality within the second user’s field of vision, the second wearable device comprising a second processor, a second memory stored on a second non-transitory computer readable medium, and a second display unit, the second wearable device configured to be worn by the second user.

[0047] Embodiment 17 is the interactive data immersion and collaboration system of any of embodiments 1 through 16, wherein the shared augmented reality room is registered in the same position and orientation for the user and the second user.

[0048] Embodiment 18 is the interactive data immersion and collaboration system of any of embodiments 1 through 17, wherein the wearable device is further configured to display, on the display unit, a real-time rendering of the second user’s interactions with the shared augmented reality room.

[0049] Embodiment 19 is the interactive data immersion and collaboration system of any of embodiments 1 through 18, wherein the second wearable device is further configured to display, on the second display unit, a real-time rendering of the user’s interactions with the shared augmented reality room.

[0050] Embodiment 20 is the interactive data immersion and collaboration system of any of embodiments 1 through 19, further comprising an external computing device linked to the generated augmented reality room, the external computing device configured to edit or rearrange said data on said at least one virtual data panel upon receiving instructions from the user to edit or rearrange said data on said at least one virtual data panel.

[0051] Embodiment 2 lis the interactive data immersion and collaboration system of any of embodiments 1 through 20, wherein the external computing device is configured to edit orS1I-001-W01 rearrange the generated augmented reality room by editing a portable room file.

[0052] Embodiment 22 is the interactive data immersion and collaboration system of any of embodiments 1 through 21, wherein the external computing device is configured to edit or rearrange the generated augmented reality room by linking with the wearable device.

[0053] Embodiment 23 is the interactive data immersion and collaboration system of any of embodiments 1 through 22, further comprising an access token having an access feature required for the user to access the generated augmented reality room.

[0054] Embodiment 24 is the interactive data immersion and collaboration system of any of embodiments 1 through 23, wherein the access feature is a security feature configured to control user access to the generated augmented reality room.

[0055] Embodiment 25 is the interactive data immersion and collaboration system of any of embodiments 1 through 24, wherein the security feature comprises one or more of a printed image or logo, sticker, RFID tag, bar code, QR code, or a magnetic strip, digital code generator, or biologic identification factor.

[0056] Embodiment 26 is the interactive data immersion and collaboration system of any of embodiments 1 through 25, wherein the access feature is a portable room format file.

[0057] Embodiment 27 is the interactive data immersion and collaboration system of any of embodiments 1 through 11, wherein the portable room format file comprises a saved version of the augmented reality room configured such that, upon opening the portable room format file by the wearable device, the wearable device automatically populates the at least one virtual data panel of the augmented reality room with a specific set of data as determined by the user in an initial setup, or as the relevant information was saved during or after a previous session in the augmented reality room.

[0058] Embodiment 28 is the interactive data immersion and collaboration system of any of embodiments 1 through 27, wherein the generated augmented reality room comprises a portable command and control interface.

[0059] Embodiment 29 is a computer program product embodied in said non-transitoryS1I-001-W01 computer readable medium and comprising a set of computer instructions that, when executed by a computer processor within a computing device, cause the computing device to: record a save file in said non-transitory computer readable medium at a first point in time, said save file comprising a layout of data elements in a particular state in a first set of one or more virtual data panels of a first augmented reality room, and populate, at a second point in time later than the first point in time, said layout of data elements in said particular state on a second set of one or more virtual data panels of a second augmented reality room.

[0060] Embodiment 30 is the computer program product of embodiment 29, wherein said second set of virtual data panels is different from said first set of virtual data panels.

[0061] Embodiment 31 is the computer program product of embodiments 29 or 30, wherein said second augmented reality room is different than said first augmented reality room.

[0062] Embodiment 32 is the computer program product of any of embodiments 29 through31, wherein said save file comprises a portable room format file.

[0063] Embodiment 33 is the computer program product of any of embodiments 29 through32, wherein said save file is stored on an access token, hard drive, wearable device, portable memory stick, or cloud server.BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Many advantages of the present disclosure will be apparent to those skilled in the art with a reading of this specification in conjunction with the attached drawings, wherein like reference numerals are applied to like elements and wherein:

[0065] Fig. l is a depiction of an existing method of viewing multiple documents or multiple pages of a single document as well as multiple data feeds on a single display screen, according to some embodiments;

[0066] Fig. 2 is a depiction of an existing method of viewing information from multiple documents or multiple pages of a single document as well as multiple live data feeds on multiple display screens and standard control room equipment, according to some embodimentsS1I-001-W01

[0067] Fig. 3 is a block diagram depicting an example of an interactive data immersion and collaboration system of the present disclosure, according to some embodiments;

[0068] Fig. 4 is a perspective view of an example a virtual project room forming part of the interactive data immersion and collaboration system of Fig. 3, according to some embodiments;

[0069] Fig. 5 is a perspective view of the virtual project room of Fig. 4, illustrating in particular rotation of the virtual project room, according to some embodiments;

[0070] Fig. 6 is a perspective view of the virtual project room of Fig. 4, illustrating in particular swapping of certain virtual data panels within the virtual project room, according to some embodiments;

[0071] Fig. 7 is a perspective view of the virtual project room of Fig. 4, illustrating in particular physical movement of a user within the virtual project room, according to some embodiments;

[0072] Fig. 8 is a perspective view of the virtual project room of Fig. 4, illustrating in particular pages of a document being distributed on multiple virtual data panels, according to some embodiments;

[0073] Fig. 9 is a perspective view an example of a virtual project room forming part of the interactive data immersion and collaboration system of Fig. 3 illustrating in particular an open- ended panel configuration, according to some embodiments;

[0074] Fig. 10 is a perspective view an example of a virtual project room forming part of the interactive data immersion and collaboration system of Fig. 3 illustrating in particular continuous virtual data panel, according to some embodiments;

[0075] Fig. 11 is a perspective view an example of a virtual project room forming part of the interactive data immersion and collaboration system of Fig. 3 illustrating in particular an multilevel vertical spiral configuration, according to some embodiments;S1I-001-W01

[0076] Fig. 12 is a perspective view of the virtual project room of Fig. 4, illustrating in particular multiple users collaborating in the same physical space, according to some embodiments;

[0077] Figs. 13-15 are perspective views of the virtual project room of Fig. 4, illustrating in particular multiple users collaborating remotely while located in different physical spaces, according to some embodiments;

[0078] Fig. 16 is a perspective view of an example of a virtual data panel forming part of the virtual project room of Fig. 4, illustrating in particular a single panel with multiple different file types interactively displayed thereon, according to some embodiments;

[0079] Fig. 17 is a block diagram of the interactive data immersion and collaboration system of Fig. 3, illustrating in particular optional external devices, according to some embodiments;

[0080] Fig. 18 is a perspective view of the virtual project room of Fig. 4, illustrating in particular an example of a file abstraction configured to locate multiple different file types on different virtual data panels, according to some embodiments;

[0081] Fig. 19 is a perspective view of the virtual project room of Fig. 4, illustrating in particular use of a second computing device to edit data displayed on a virtual data panel within the virtual project room, according to some embodiments;

[0082] Figs. 20-21 are perspective views of the virtual project room of Fig. 4, illustrating in particular access discrimination based upon clearance levels of multiple users within the same virtual project room, according to some embodiments;

[0083] Fig. 22 is a block diagram illustrating an example of an access token forming part of the interactive data immersion and collaboration system of Fig. 17, according to some embodiments;

[0084] Fig. 23 is a block diagram depicting several examples of software module groups that a software app 14 forming part of the interactive data immersion and collaboration system of Fig.3 uses to achieve the effects disclosed herein, according to some embodiments;SlI-OOl-WOl

[0085] Fig. 24 is a block diagram depicting several examples of software modules within the “AUGMENTED REALITY RENDERING AND EXPERIENCE CREATION MODULES” group of Fig. 23, according to some embodiments;

[0086] Fig. 25 is a block diagram depicting several examples of software modules within the “ROOM SAVE AND RETRIEVAL MODULES” group of Fig. 23, according to some embodiments;

[0087] Fig. 26 is a block diagram depicting several examples of software modules within the “HUMAN-ROOM INTERACTION MODULES” group of Fig. 23, according to some embodiments;

[0088] Fig. 27 is a block diagram depicting several examples of software modules within the “ROOM COLLABORATION MODULES” group of Fig. 23, according to some embodiments;

[0089] Fig. 28 is a block diagram depicting several examples of software modules within the “SECURITY / PRIORITY MODULES” group of Fig. 23, according to some embodiments;

[0090] Fig. 29 is a block diagram depicting several examples of software modules within the “ROOM BROWSING AND HOSTING MODULES” group of Fig. 23, according to some embodiments;

[0091] Fig. 30 is a block diagram illustrating example computer system with which any of the devices or systems described herein may be implemented, according to some embodiments; and

[0092] Fig. 31 is a block diagram illustrating example mobile computer system with which any of the devices or systems described herein may be implemented, according to some embodiments.DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT

[0093] Illustrative embodiments of the disclosure are described below. In the interest of clarity, not all features of an actual implementation are described in this specification. It will ofS1I-001-W01 course be appreciated that in the development of any such actual embodiment, numerous implementation-specific decisions must be made to achieve the developers’ specific goals, such as compliance with system-related and business-related constraints, which will vary from one implementation to another. Moreover, it will be appreciated that such a development effort might be complex and time-consuming, but would nevertheless be a routine undertaking for those of ordinary skill in the art having the benefit of this disclosure. The interactive data immersion and collaboration system and related methods disclosed herein boasts a variety of inventive features and components that warrant patent protection, both individually and in combination.

[0094] Fig. 3 illustrates an example of an interactive data immersion and collaboration system 10 according to some embodiments of the disclosure. In some embodiments, the interactive data immersion and collaboration system 10 comprises an augmented reality (AR) or virtual reality (VR) headwear 12 (e.g., glasses, goggles, headset, etc., hereinafter referred to as “AR / VR headwear 12”) and a computer software application 14 (hereinafter “app 14”). By way of example, the AR / VR headwear 12 can be any AR or VR headwear currently available in the market or developed in the future. By way of example, the AR / VR headwear 12 is itself a mobile computing device, having at least a processor 252, memory 254, a display 256, and a communications interface 258 (see, e.g., Fig. 31 and accompanying disclosure describing a mobile computing device 250). By way of example, the app 14 comprises a computer program product embodied in a non-transitory computer-readable medium and comprising a set of computer instructions that, when executed by a processor, cause the AR / VR headwear 12 to display to the user a dynamic, interactive presentation of data or data sets in a virtual environment 16 that appears to surround the user and that enables a user to configure, arrange, and / or edit the data in a variety of ways.

[0095] In some embodiments, the interactive data immersion and collaboration system 10 may further include a first computing device 18 which is separate and distinct from the AR / VR headwear 12. By way of example, the first computing device 18 comprises at least a processor 202, memory 204, display 216, and a communications interface (see, e.g., Fig. 30 and accompanying disclosure describing computing device 200), and the software app 14. In some embodiments, the first computing device 18 may be used by a user to design the interface withS1I-001-W01 the data 20 in which the user is to be immersed (e.g., single document, multiple documents, images, video, external device control, etc.), and / or design the configuration and / or geometry of the virtual environment 16, including how the data is able to move within the virtual environment 16. In some embodiments, the first computing device 18 may be any type of computing device, including but not limited to (and by way of example only) a desktop computer, laptop computer, mainframe, table computer, smart phone, smart watch, and the like.

[0096] In some embodiments, the AR / VR headwear 12, the first computing device 18, and another other peripheral device configured to connect with the system 10 may be configured to send and / or receive data in a wired or wireless manner through a network connection, which may include a cloud server 22. For example, once the user has configured the virtual environment 16 to their liking, the user can upload the data 20 to the cloud server 22 or a physical data storage device. This data 20 may then be accessed by or transferred to the AR / VR headwear 12.

[0097] In some embodiments, by way of example, the system 10 presents the user wearing a connected AR / VR headwear 12 with a dynamic, interactive presentation of the data 20 or data sets in a virtual environment 16 (e.g., which may also be referred to as a “virtual project room” or “virtual room” or “room” or “virtual command-and-control center”) that appears to surround the user and that enables a user to configure, arrange, and / or edit the data in a variety of ways. In some embodiments, this interactive presentation may be in the form of augmented reality, in which the virtual data configuration is presented to the user against a background comprising the real world around them. In some embodiments, this interactive presentation may be in the form of virtual reality, in which the virtual data configuration is presented to the user within a virtual space.

[0098] By way of example, the system may present a virtual project room 16 comprising data 20 in the form of a document or multiple documents appearing to the user in an array of virtual data panels 24 configured surround the user, fully immersing the user in the data 20. In some embodiments, the user is able to fully rearrange the pages of a document and / or the order of documents and edit the documents to configure a customized virtual project room 16 to efficiently present and manipulate information. In some embodiments, the virtual project room 16 can be further adapted to incorporate live updates and expand control capabilities. MostS1I-001-W01 significantly, the system 10 enables saving of a virtual project room 16 so that a user may pick up where they left off upon returning to the virtual environment at a later time. In some embodiments, the virtual project room 16 may be used for creating, planning, collaborating, monitoring and presenting to others. Unlike standard display systems, the virtual project room 16 of the present disclosure makes full use of spatial computing to immerse the user in the task, project, data, and / or action, depending on the use given to the room.

[0099] Figs. 4-7 illustrate one example of a virtual project room 16 that may be presented to a user 2 by the system 10 by way of the AR / VR headwear 12, according to some embodiments. In some embodiments, the virtual project room 16 may be presented as one or more virtual data panels 24 surrounding the user in a circular arrangement. In some embodiments, the virtual project room 16 may be presented to the user as a plurality of virtual data panels 24. In some embodiments, each virtual data panel 24 in the plurality of virtual data panels 24 may be of equal size and shape as every other virtual data panel 24. In some embodiments, one or more virtual data panels 24 in the plurality of virtual data panels 24 may have a different size than other virtual data panels 24 in the plurality of virtual data panels 24. In some embodiments, the plurality of virtual data panels 24 may be rendered in any size, shape, and number as required by the user. In some embodiments, the virtual project room 16 is dynamic, in that the system 10 enables the user 2 to rearrange the data 20 in real time by manipulating the one or more virtual data panels 24 in space. For example, this may be done using a motion sensor control functionality of the AR / VR headwear 12, in which certain hand and / or finger gestures that the user performs in the space in front of the AR / VR headwear 12 are recognized by the AR / VR headwear 12 and interpreted to cause certain movement of the virtual data panels 24. In some embodiments, the system 10 may enable the user to rotate the entire virtual project room 16 about a central axis of the virtual room 16 so that a desired individual virtual data panel 24 of the plurality of virtual data panels 24 is positioned in immediately in front of the user in the virtual space, as depicted by way of example in Fig. 5. For example, this may be accomplished by the user making a sweeping hand gesture in the space in front of the AR / VR headwear 12. In some embodiments, the system 10 may enable the user to move individual virtual data panels 24 within the plurality of virtual data panels to rearrange the virtual data panels 24 to a customized configuration, as depicted by way of example in Fig. 6. In some embodiments, the system 10 may enable the user to move virtually within the virtual project room 16 so that physical stepsSlI-OOl-WOl taken by the user in the real world results in the user moving to a position in front of a different virtual data panel 24 within the virtual space of the virtual project room 16, as depicted by way of example in Fig. 7. In some embodiments, system 10 enables the user to manipulate the data in the virtual data panels 24 (or the virtual data panels 24 themselves or the entire virtual room 16 itself) from “inside” the virtual room 16 or from “outside” the virtual room 16. The user thus has three ways (at least) to navigate the virtual project room 16: walking, rearranging content, and rotating or shifting the virtual project room 16 itself.

[0100] For convenience, the virtual project room 16 is presented herein throughout as a family of flat virtual data panels 24 of equal size. However, the virtual data panels 24 may vary in size depending on content and users may wish to add a 3-dimensional (3D) object to a panel (as described below) or replace a panel with another 3D object.

[0101] In some embodiments, the virtual project room 16 may be configured such that each page of a single document 26 may be displayed on a different virtual data panel 24, as shown by way of example in Fig. 8. For example, a first page 26a of the document may be displayed on a first virtual data panel 24a, a second page 26b of the document may be displayed on a second virtual data panel 24b, a third page 26c of the document may be displayed on a third virtual data panel 24c, a fourth page 26d of the document may be displayed on a fourth virtual data panel 24d, a fifth page 26e of the document may be displayed on a fifth virtual data panel 24e, a sixth page 26f of the document may be displayed on a sixth virtual data panel 24f, a seventh page 26g of the document may be displayed on a seventh virtual data panel 24g, an eighth page 26h of the document may be displayed on an eighth virtual data panel 24h, a ninth page 26i of the document may be displayed on a ninth virtual data panel 24i, a tenth page 26j of the document may be displayed on a tenth virtual data panel 24j, and an eleventh page 26k of the document may be displayed on an eleventh virtual data panel 24k. Although shown and described herein as a one- page-per-panel display, the virtual project room 16 may be configured so that multiple pages of a single document may appear on a single virtual data panel 24. For example, the user may configure the system 10 such that two (2), four (4), six (6), eight (8) or more pages appear on each virtual data panel 24. In such a situation, the user may cause individual pages to move within each virtual data panel 24 and between different virtual data panels 24.S1I-001-W01

[0102] In some embodiments, as shown by way of example in Fig. 8, the virtual project room 16 may be configured such that a gap 28 exists between one or more virtual data panels 24. In some embodiments, this gap 28 is to simulate a doorway or exit from the virtual project room 16, even though it is possible to physically walk through the virtual data panels 24 without affecting the person or the virtual data panel 24. However, for some users, the presence of a gap 28 or simulated exit may make the virtual project room 16 seem less claustrophobic, thereby enhancing the experience of using the interactive data immersion and collaboration system 10 disclosed herein. By way of example, any configuration of virtual project room 16 shown or described herein, or otherwise imaginable by a user, may be configured to include a simulated exit 28.

[0103] By way of example, Figs. 9-11 illustrate several different examples of virtual project room 16 configurations that are possible with the interactive data immersion and collaboration system 10 disclosed herein. By way of example, the virtual project room 16 may have different geometric arrangements to meet different purposes and uses, for example including but not limited to circular with or without a door, spiral, continuous, open, and spherical. In some embodiments, each of these shapes comes with a distinctive panel motion and / or whole room movement. In some embodiments, the room may be configured to suit any 3-dimensional arrangement and motion pattern.

[0104] In some embodiments, the virtual project room 16 presented to the user by the system 10 may be configured in an open ended “horseshoe” shape, as shown by way of example in Fig. 9. For example, this configuration may be useful in a situation in which the user might not want to project an enclosed space.

[0105] In some embodiments, the virtual project room 16 may be presented to the user by the system 10 as a single virtual data panel 24 that extends around a space, as shown by way of example in Fig. 10. In some embodiments, the single virtual data panel 24 may be curved as shown, for example in a circle, oval, or ellipse. In some embodiments, the single virtual data panel 24 may have angled comers such that the virtual project room 16 comprised a polygonal geometric shape, including but not limited to square, rectangular, pentagonal, hexagonal, octagonal, and so forth.S1I-001-W01

[0106] In some embodiments, the virtual project room 16 may be presented to the user by the system 10 as a plurality of virtual data panels 24 arranged in an ascending coiled shape, as shown by way of example in Fig. 11. In such a configuration, the virtual project room 16 may be configured so that the user may move selected virtual data panels 24 vertically up and down, as well as around the coil, to maneuver the desired document pages (or other data files) in front of them. Additionally, the entire room 16 may be moved by hand gestures, for example a horizontal gesture may cause the room 16 to rotate around a central axis as if following the coil (so panels effectively move left or right and up), and a vertical gesture may cause the room 16 to shift up or down.

[0107] In some embodiments, the interactive data immersion and collaboration system 10 may be configured so that multiple users each having their own AR / VR headwear 12 can be in the same virtual project room 16. By way of example, the interactive data immersion and collaboration system 10 described herein empowers collaboration by multiple users within the same virtual project room 16 regardless of whether the users are in the same physical location, are engaged in remote collaboration (e.g., one or more users are not in same physical location), and / or have different access or clearance levels.

[0108] Referring to Fig. 12, in some embodiments, the interactive data immersion and collaboration system 10 may be configured such that multiple users 2a, 2b, 2c in the same physical location equipped with their own AR / VR headwear 12a, 12b, 12c will experience the same virtual project room 16 environment in the same location and orientation as everyone else in the virtual project room 16. Each user’s AR / VR headwear 12a, 12b, 12c renders the virtual project room 16 independently, but the system 10 recognizes that the users 2a, 2b, 2c are in the same virtual project room 16 and links (or coordinates) each user’s experience through an electronic handshake mechanism in which each user’s AR / VR headwear 12a, 12b, 12c registers the virtual project room 16 to the same point of reference and / or to each other. In some embodiments, within this shared experience, each user (e.g., user 2a) can see every other user’s (e.g., users 2b, 2c) data edits. In some embodiments, within this shared experience, each user can see every other user’s physical person within the virtual project room 16. In some embodiments, within this shared experience, each user can see every other user’s actions relative to the virtual project room 16. For example, if one user 2a points at a virtual data panel 24 in a particular spot,S1I-001-W01 every other user 2b, 2c is able to see that user 2a pointing to the virtual data panel 24 in the same spot (e.g., the same spot on the panel, the same spot in virtual space, and the same spot in physical space) as if the collaboration was occurring in a fully physical environment.

[0109] In some embodiments, there are several components that enable this functionality. First, the motion technology native to the AR / VR headwear 12 of each user is configured to recognize its wearer’s activity based on proximity, arm location, angle, gestures, etc., and so that the interactive data immersion and collaboration system 10 can determine which movements are attributable to which user and also which other users’ AR / VR headwear 12 displays need to show that movement to its user. Second, in some embodiments the virtual project room 16 may be hosted by one user and joined by guest users. In some embodiments, guest users that are physically present in same location as the host user may need to present an access key to be able to enter the shared virtual project room 16. For example, this access key may be a digital key stored in the guest user’s AR / VR headwear 12 that is recognized by the host users’ AR / VR headwear 12 to enable guest entry into the virtual project room 16. In some embodiments, the guest user may have a physical object that indicates or grants credentialed access to the hosted virtual project room 16. (See, e.g., discussion below).

[0110] Referring to Figs. 13-15, in some embodiments, the interactive data immersion and collaboration system 10 may be configured such that multiple users 2a, 2b, 2c in different physical locations equipped with their own AR / VR headwear 12a, 12b, 12c will experience the same virtual project room 16 environment in a different physical location but with the same orientation as everyone else in the virtual project room 16. Each user’s AR / VR headwear 12a, 12b, 12c renders the virtual project room 16 independently, but the system 10 recognizes that the users 2a, 2b, 2c are in the same virtual project room 16 and links (or coordinates) each user’s experience through a remote call mechanism to share information about the position relative to the virtual project room’s reference frame of each user in the call so each user’s AR / VR headwear 12a, 12b, 12c can render an avatar of the other remote users to display to the headwear’s user. In some embodiments, within this shared experience, each user (e.g., user 2a) can see every other user’s (e.g., users 2b, 2c) data edits. In some embodiments, within this shared experience, each user can see every other user’s person represented by an avatar within the virtual project room 16. In some embodiments, within this shared experience, each user can seeS1I-001-W01 every other user’s actions relative to the virtual project room 16, by virtue of the avatars moving in a similar manner as the user. In some embodiments, within this shared experience, the AR / VR headwear 12a, 12b, 12c may be equipped with a microphone and speakers to enable voice communication so that the users located remotely can talk to one another as if they are in the same physical location.

[0111] For example, Fig. 13 illustrates an example of a virtual project room 16 in which user 2a is physically present and the AR / VR headwear 12a renders avatars to represent to user 2a the locations within the virtual project room 16 of users 2b, 2c. If one of the remote users 2b points at a virtual data panel 24 in a particular spot, for example, the AR / VR headwear 12a will show user 2a the user 2b avatar pointing to that virtual data panel 24 in that same spot (e.g., the same spot on the panel and the same spot in virtual space) as if the collaboration was occurring in a fully physical environment. The same is true for user 2b and user 2c.

[0112] For example, Fig. 14 illustrates an example of a virtual project room 16 in which user 2b is physically present and the AR / VR headwear 12b renders avatars to represent to user 2b the locations within the virtual project room 16 of users 2a, 2c. If one of the remote users 2a points at a virtual data panel 24 in a particular spot, for example, the AR / VR headwear 12b will show user 2b the user 2a avatar pointing to that virtual data panel 24 in that same spot (e.g., the same spot on the panel and the same spot in virtual space) as if the collaboration was occurring in a fully physical environment.

[0113] Similarly, Fig. 15 illustrates an example of a virtual project room 16 in which user 2c is physically present and the AR / VR headwear 12c renders avatars to represent to user 2c the locations within the virtual project room 16 of users 2a, 2b. If one of the remote users 2a points at a virtual data panel 24 in a particular spot, for example, the AR / VR headwear 12c will show user 2c the user 2a avatar pointing to that virtual data panel 24 in that same spot (e.g., the same spot on the panel and the same spot in virtual space) as if the collaboration was occurring in a fully physical environment.

[0114] In some embodiments, the remote collaboration may occur without the rendering of avatars. In such a case the AR / VR headwear 12 will show the user only highlights of when each remote user interacts with the virtual project room, for instance when a user points to or touchesS1I-001-W01 one element in a virtual data panel, that element may become highlighted (e.g., brighter, color- coded, etc.) to indicate that the element is being manipulated by a remote user. In some embodiments, the system 10 may be configured such that the highlighting may be color-coded to indicate which particular user is manipulating the data element.

[0115] In some embodiments, the virtual data panels 24 presented to the user by the AR / VR headwear 12 as instructed by the app 14 are inherently two-way communication interfaces configured to display information to a user and capture instructions from the user manifested by the user’s interaction with the virtual data panels 24 to send the user’s instructions to a connected device. The information or data 20 presented in a virtual data panel 24 (and interactable with a user) may range from a single page of a document as shown for example in Fig. 8 to a combination of various kinds of information (e.g., static, dynamic, and real-time) and media. In some embodiments, the virtual data panel 24 may comprise elements to control external devices such as actuators, drones, swarms of drones, etc. By way of example only, Fig. 16 illustrates various kinds of information or data 20 that may populate one or more virtual data panels 24 within a virtual project room 16 and that a user may interact with, including but not limited to document(s) 24 (e.g., a single document parsed by pages as described above or multiple documents displayed on different virtual data panels), video files 30 in any format (e.g., MP3, MP4, MOV, AVI, WMV, AVCHD, WebM, FLV, etc.) and from any usable source (e.g., streaming, downloadable files, real-time camera feed, etc.), image files 32 in any format (e.g., JPG, TIFF, PNG, GIF, etc.), external data sources 34 (e.g., databases, Internet links, etc ), control actuators 36 (e.g., for controlling external controllable objects, such as drones, rovers, game pieces, etc.), other data files 38, real-time sensor data 40, and access to other virtual project rooms 42. In some embodiments, each virtual data panel 24 may be configured to display and facilitate user interaction with any type of data 20. In some embodiments, each virtual data panel 24 may be configured to display user interaction with multiple different types of data 20. In some embodiments the system 10 may be configured so that the AR interacts with physical objects within the space, for example, maps, game pieces, etc.

[0116] Fig. 17 illustrates the interactive data immersion and collaboration system 10 as shown in Fig. 3, with the addition of optional external devices or features that may be part of some embodiments of the interactive data immersion and collaboration system 10 but are not requiredS1I-001-W01 for the system to work properly. By way of example, these external devices include (but are not necessarily limited to) a file abstraction 44, a second computing device 46, and an access token 48.

[0117] Fig. 18 illustrates a file abstraction 44 that may be part of the interactive data immersion and collaboration system 10, according to some embodiments. By way of example, a file abstraction 44 (or portable room format (“PRF”) file 44) comprises a “map” or layout of all of the elements in the virtual project room 16, and may be used by the AR / VR headwear 12 to populate each of the virtual data panels 24 with the relevant data 20, for example as determined by the user in an initial setup, or as the relevant information was saved during or after a previous session in the virtual project room 16. By way of example, a PRF file 44 may in many ways be like a portable document format (PDF) file. Whereas a PDF file gives people an easy, reliable way to present and exchange documents - regardless of the software, hardware, or operating systems being used by anyone who views the document, a PRF file 44 gives credentialed users an easy, reliable way to load and enter virtual project rooms 16 - regardless of the hardware or operating systems being used by anyone with access who enters the virtual project room 16. By way of example, the PRF file 44, when accessed by a user’s AR / VR headwear 12. As illustrated in Fig. 18 and as mentioned previously, relevant data 20 may include (but is not limited to) single documents 26a, multiple documents 26b, streaming or downloaded video files 30a, realtime camera video feed 30b, image files 32, database files 34a, Internet links 34b, control actuators 36, other data 38, real-time sensor data 40, and / or access portals 42 to other virtual project rooms 16. When the user(s) is / are finished with a session, the data 20 including edits and / or configuration, may be saved within the PRF file 44 for use at a later time. This PRF file 44 in a memory in the user’s AR / VR headgear 12, on a separate memory device (e.g., thumb drive, external hard disk, etc.), on an external computing device, on a cloud server 22, or any other suitable storage medium. In some embodiments, the PRF file 44 may be opened and / or edited in an external computing device having the software app 14 installed thereon, and then later (or in real-time) accessed by the AR / VR headgear 12 to refresh the PRF file 44 that may be saved to the headgear. In some embodiments, the PRF file 44 may be edited on an external computing device located in the virtual project room 16.S1I-001-W01

[0118] In some embodiments, the interactive data immersion and collaboration system 10 disclosed herein may be configured to enable editing of data 20 through direct user interaction with the virtual data panels 24. For example, relocating and rearranging pages and / or documents by way of AR / VR headgear 12 recognition of user gestures has been discussed previously. However, the interactive data immersion and collaboration system 10 may be configured to enable further content editing by way of panel interaction, for example with a popup virtual keyboard. In some embodiments, the data 20 presented in the virtual project room 16 may be edited in real time by way of a connected (e.g., linked to the virtual project room 16) second computing device 46, as shown by way of example in Fig. 19. For example, in this embodiment, multiple users 2a, 2b are in the same virtual project room 16, each having their own AR / VR headgear 12a, 12b, respectively. User 2b is holding a connected secondary computing device 46 which may be any portable computing device with the software app 14 installed to enable real time updating of the data 20 presented to the users in the virtual project room 16, including but not limited to a tablet computer (shown by way of example only in Fig. 19), smart phone, smart watch, laptop computer, desktop computer, and the like. By way of example, as user 2b makes edits on the second computing device 46, the data presented in the virtual data panels 24 within the virtual project room 16 may be updated in real time, such that user 2a’ s AR / VR headgear 12 will present the data 20 reflecting user 2b ’s edits to user 2a as they happen. In some embodiments, use of a secondary computing device 46 to edit the presented data 20 provides a better or preferred user interface, leading to more efficient time management. In some embodiments, the user may edit the PRF 44 using the second computing device 46, as described above.

[0119] As disclosed herein, an import breakthrough feature of the interactive data immersion and collaboration system 10 is the system’s ability to have multiple AR / VR headgear 12 render and display to multiple users a virtual project room 16 that is in the same location and orientation in an augmented reality space. In some embodiments, the interactive data immersion and collaboration system 10 disclosed herein may be configured to discriminate between users such that certain data that is viewable by some users that have sufficient clearance or permissions to view such data may be redacted, obscured, or hidden from other users that are lacking sufficient clearance or permissions. For example, this situation may arise with certain govemment / military collaborations, business collaborations, legal collaborations, etc. in which certain data orS1I-001-W01 information that may be sensitive, classified, confidential, proprietary, and / or privileged may be displayed to a user or hidden from a user based on their permissions level. By way of example, Figs. 20-21 illustrate a virtual project room 16 that is presenting a data package having a combination of sensitive (e.g., secret or confidential) information 50 and non-sensitive information 52. In this example, user 2a has full access permission to view all data presented in the virtual project room 16, and user 2b has a lower-level or restricted access that permits user 2b to view non-sensitive information 52 but prohibits user 2b from viewing sensitive information. By way of example, Fig. 20 depicts the virtual project room 16 as displayed to user 2a (with unrestricted access) by AR / VR headgear 12a, in which both the sensitive information 50 and the non-sensitive information 52 are displayed on the virtual data panels 24. Fig. 21 depicts the virtual project room 16 as displayed to user 2b (with restricted access) by AR / VR headgear 12b, in which the non-sensitive information 52 are displayed on the virtual data panels 24 and the sensitive information 50 is blocked from view for user 2b. In some embodiments, an indication 54 of the presence of sensitive information may be displayed to user 2b. In some embodiments, the system 10 may not display the sensitive information to the user 2b without sufficient access privilege (e.g., not displaying anything on the virtual data panel 24 that displays the sensitive information 50 to the user 2a with access), or alternatively the system 10 may display something else on that panel so as to not alert the user 2b to the presence of the sensitive information 50 at all.

[0120] In some embodiments, the interactive data immersion and collaboration system 10 of the present disclosure may include an access token 48, which comprises a tangible object that a user may possess in the physical world that serves at least two purposes: first, as a security measure that may or may not include a multi-factor authentication feature; and second, as a convenient and / or secure way to access a hosted virtual project room 16 for example in a room hosting service location.

[0121] Regarding the first listed purpose, the access token 48 may include a security feature 56 that may be configured at various levels of security for the user to satisfy before being granted access to a virtual project room 16 (e.g., a virtual project room 16 hosted by another user or a virtual project room 16 that contains sensitive information requiring credentialed access. For example, at the lowest security level, mere possession of an access token 48 may grant the userS1I-001-W01 access to a virtual project room 16 (e.g., similar to a membership card). In some embodiments, the access token 48 may have an image or logo 58 printed on the access token 48 (or on a sticker affixed to the access token 48, for example) that notifies a gatekeeper that the user has the requisite permission to enter a particular virtual project room 16. In some embodiments, the access token 48 may include a machine-readable element that may convey the requisite permissions to a room gatekeeper, such as an RFID tag 60, bar code 62, QR code 64, or a magnetic strip (e.g., similar to a credit card). In some embodiments, the access token 48 may include a digital code generator 66 that produces a code that changes in a period of time (e.g., every minute) and that the user would have to enter into an authentication app (for example) to unlock access to the virtual project room 48. In some embodiments, the access token 48 may be configured to require a biologic identification factor that can be assessed with an external reader or by the access token 48 itself. For example, in some embodiments, the access token 48 may be configured to require and / or read a user’s fingerprint 68. In some embodiments, the access token 48 may be configured to require and / or perform a retinal scan or face ID 70 of the user. In some embodiments, the access token 48 may be configured to require and / or perform a DNA verification.

[0122] By way of example, in the embodiments with the readable permission including multifactor authentication and / or biologic authentication, a signal may be sent from an external reader or the access token 48 itself to the AR / VR headwear 12 instructing the AR / VR headwear 12 to grant access for the user to the secured virtual project room 16. In some embodiments, this access permission may also include a relevant permission level (e.g., restricted, unrestricted).

[0123] In some embodiments, the access permission may be geospecific in nature. For example, the access token 48 may be provided with a GPS locator (or equivalent), and the computer instructions included in the app 14 pertaining to a specific virtual project room 16 may include a requirement that the room may only be accessed if the user (e g., identified or recognized by possession of the access token 48) is in a specific physical location.

[0124] In some embodiments, the access token 48 may comprise a memory chip or feature 74 configured to store a saved file abstraction (e.g., PRF file) 44 that may be accessible by any AR / VR headwear 12 in which the associated user has permission. By way of example, thisSlI-OOl-WOl feature may enable access to a virtual project room 16 in a location in which there is limited or controlled network access, because the AR / VR headwear 12 may access the virtual project room 16 directly from the access token 48 without the need for a network. At the conclusion of the work session, the modified or edited virtual project room 16 may be saved to the memory feature 74 on the access token 48 and then (optionally) uploaded to a network server 22 when a connection becomes available.

[0125] In some embodiments, the software app 14 producing the Augmented Reality system comprises multiple modules that are used in producing the augmented reality experience. Together these modules translate various aspects of the idea into actions and objects that become physical in the augmented reality. Generally, the app 14 acts in two ways. First, the app 14 translates physical ideas into abstract ideas and then recreates the abstract ideas as objects that appear in a parallel (augmented) reality alongside or superimposed upon the reality perceived as real. In some embodiments, the augmented reality interacts with the actual physical environment in such a manner that to the human eye the AR is blended with the physical reality. Second, the app 14 translates abstract ideas into physical manifestations in the parallel (augmented) reality alongside the reality perceived as real. In some embodiments, this augmented reality interacts with the actual physical environment in such a manner that to the human eye the AR is blended with the physical reality.

[0126] By way of example, Fig. 23 depicts various module groups that the app 14 uses to achieve the effects of the interactive data immersion and collaboration system 10 disclosed herein. In some embodiments, the app 14 may comprise augmented physical reality rendering and experience creation modules 80, room save and retrieval modules 82, human-room interaction modules 84, room collaboration modules 86, security and / or privacy modules 88, and room browsing and hosting modules 90.

[0127] Referring to Fig. 24, and by way of example, the augmented physical reality rendering and experience creation modules 80 may include a panel creation and rendering module 100, data recreation module 102, document data and static recreation module 104, unchangeable data module 106, remote data connection module 108, room update and refresh module 110, and warnings communication module 112.S1I-001-W01

[0128] In some embodiments, the panel creation and rendering module 100 comprises a set of computer instructions that when executed by the processor cause the system 10 to create realistic virtual data panels in augmented reality. The word “panels” is used to describe the entity created and used in AR to display information for or create interaction with the user of the room. In some embodiments, the specific entity created may look like a panel (e.g., a parallelepiped). In some embodiments, the created entity may look like something other than a parallelepiped.

[0129] In some embodiments, the data recreation module 102 comprises a set of computer instructions that when executed by the processor cause the system 10 to display data on virtual data panels. This entails modifying the appearance of a virtual data panel (e.g., its texture) with some frequency to match the frequency at which the data is produced.

[0130] In some embodiments, the document data and static recreation module 104 comprises a set of computer instructions that when executed by the processor cause the system 10 to present static information, such as documents in the virtual data panels. Unlike data, documents do not necessarily have an associated update rate, rather, they may need updating based on events such as when a document is changed by a user on a secondary computing device 46 as described above.

[0131] In some embodiments, the unchangeable data module 106 comprises a set of computer instructions that when executed by the processor cause the system 10 to ensure data that is not meant to be changed is managed through a path that is distinct from the other kinds of data and documents described elsewhere in this disclosure.

[0132] In some embodiments, the remote data connection module 108 comprises a set of computer instructions that when executed by the processor cause the system 10 to manage communication at the data level with outside hosting services that may include (but is not limited to) real time data feeds, event signaling networks, the internet, data hosting servers, secure data hosting servers, etc.

[0133] In some embodiments, the room update and refresh module 110 comprises a set of computer instructions that when executed by the processor cause the system 10 to monitor the need to update the virtual project room and update as needed. By way of example, this module isS1I-001-W01 configured to detect signals that suggest for the need to update the aforementioned document and static data. It also monitors all other data feeds to make sure that real time feeds are up to date.

[0134] In some embodiments, the warnings communication module 112 comprises a set of computer instructions that when executed by the processor cause the system 10 to alert the user of changes in data (so the user knows about data updates that occur elsewhere in the room from when their attention was placed elsewhere). This module may also alert the user about problems with data feeds and other aspects of the room that may require a user’s attention.

[0135] Referring to Fig. 25, and by way of example, the room save and retrieval modules 82 may include a Portable Room Format (PRF) write (“save”) module 114, Portable Room Format (PRF) read (“load”) module 116, PRF history record module 118, PRF history browsing module 120, and PRF history retrieval module 122.

[0136] In some embodiments, the Portable Room Format (PRF) write (“save”) module 114 comprises a set of computer instructions that when executed by the processor cause the system 10 to record the state of a room (data overlay, position, geometry, history, etc.) in a PRF fde 44 for later retrieval.

[0137] In some embodiments, the Portable Room Format (PRF) read (“load”) module 116 comprises a set of computer instructions that when executed by the processor cause the system 10 to retrieve the state of a room (data overlay, position, geometry, room physics, history, etc) from a PRF fde 44 to populate a virtual project room 16.

[0138] In some embodiments, the PRF history record module 118 comprises a set of computer instructions that when executed by the processor cause the system 10 to record the history of a virtual project room 16 so a user can later retrieve the state at which the virtual project room 16 was at a given point in time or before certain changes were made to the virtual project room 16.

[0139] In some embodiments, the PRF history browsing module 120 comprises a set of computer instructions that when executed by the processor cause the system 10 to enable browsing the history of a virtual project room 16.SlI-OOl-WOl

[0140] In some embodiments, the PRF history retrieval module 122 comprises a set of computer instructions that when executed by the processor cause the system 10 to enable retrieval of the state of a virtual project room 16 the way it was at a given point in time or before certain changes were made to the virtual project room 16.

[0141] Referring to Fig. 26, and by way of example, the human / room interaction modules 84 may include a room activity monitoring module 124, room activity machine learning analysis module 126, room geometry module 128, room physics module 130, and human- AR interaction and dispatch module 132.

[0142] In some embodiments, the room activity monitoring module 124 comprises a set of computer instructions that when executed by the processor cause the system 10 to monitor the way a user interacts with the data in which they are immersed. In some embodiments, the outcomes of this activity monitoring may be used by the user to analyze information usage or by machine learning algorithms.

[0143] In some embodiments, the room activity machine learning analysis module 126 comprises a set of computer instructions that when executed by the processor cause the system 10 to use machine learning techniques to infer relevance and connectedness across different kinds of data from the way in which a user interacts with the data.

[0144] In some embodiments, the room geometry module 128 comprises a set of computer instructions that when executed by the processor cause the system 10 to arrange a plurality of virtual data panels 24 according to a specific geometry as instructed by the user and connects the virtual project room 16 with the required physics for the room.

[0145] In some embodiments, the room physics module 130 comprises a set of computer instructions that when executed by the processor cause the system 10 to create and implement the laws of physics of the virtual project room 16. In augmented reality, the virtual data panels 24 and their content are made to react to a user in the ways determined by this module. For instance, virtual data panels 24 may represent having friction, allowed and disallowed ranges of motion, or may be coupled with other virtual data panels 24 in pre-specified ways. In someSlI-OOl-WOl embodiments, each virtual project room 16 may be set with different physics parameters depending on the purpose of the virtual project room 16.

[0146] In some embodiments, the human- AR interaction and dispatch module 132 comprises a set of computer instructions that when executed by the processor cause the system 10 to govern the options available and the actions and responses following the interaction of the user with the content of the virtual project room 16.

[0147] Referring to Fig. 27, and by way of example, room collaboration modules 86 may include a two-way real time data and video communication module 134, synchronization module 136, remote communication module 138, and occlusion and ghosting module 140.

[0148] In some embodiments, the two-way real time data and video communication module 134 comprises a set of computer instructions that when executed by the processor cause the system 10 to manage all the communication needs associated with two-way communication between the virtual project room 16 and external connected devices.

[0149] In some embodiments, the synchronization module 136 is a module configured to synchronize AR across different AR devices and comprises a set of computer instructions that when executed by the processor cause the system 10 to manage the operations needed to keep to virtual project rooms synchronized. For instance, when two or more users share a physical and augmented reality room the two or more individually rendered AR rooms presented to the users need to appear in the exact same location and orientation, as described above. This feature is managed by the synchronization module 136.

[0150] In some embodiments, the remote communication module 138 is a module to create shared AR in remote communication experiences and comprises a set of computer instructions that when executed by the processor cause the system 10 to manage the operations needed to keep remote collaboration virtual project rooms synchronized. For example, similar to the situation where multiple users are in the same physical space, when two or more users are in different physical locations such that the collaboration is remote, system 10 may be configured to enable the AR / VR headgear 12 to exchange information about the behavior of each user in order to create a shared virtual project room experience within the individually rendered AR rooms inS1I-001-W01 different physical locations as described above. This feature is managed by the remote communication module 138.

[0151] In some embodiments, the occlusion and ghosting module 140 is a module to create realistic occlusion and ghosting of objects and people and comprises a set of computer instructions that when executed by the processor cause the system 10 to present objects and people to the user by way of the AR / VR headgear 12. For example, AR objects may be presented in the AR / VR headgear 12 in a customizable manner whereas real objects are presented where they are in reality. This creates an opportunity to choose different ways to present objects and people to the user of the AR / VR headgear 12. For example, physical walls may fully occlude a virtual data panel 24 or the user may choose to increase the transparency of the virtual data panel 24 when it has been rendered or pushed behind an object or a person. The occlusion and ghosting module 140 module manages the implementation of those choices.

[0152] Referring to Fig. 28, and by way of example, security / privacy modules 88 may include an access control module 142, security information rendering module 144, room geolocation module 146, access token recognition module 148, access token pairing module 150, access token information retrieval module 152, and multi-factor security module 154.

[0153] In some embodiments, the access control module 142 comprises a set of computer instructions that when executed by the processor cause the system 10 to implement different basic data access controls.

[0154] In some embodiments, the security information rendering module 144 comprises a set of computer instructions that when executed by the processor cause the system 10 to implement more advanced options for security and access control such as selective rendering of information.

[0155] In some embodiments, the room geolocation module 146 comprises a set of computer instructions that when executed by the processor cause the system 10 to verify when / if a virtual project room has geo-location restrictions imposed on what should be rendered within the virtual project room, if anything. For example, if the system 10 is unable to verify the geolocation of the user, or if the system 10 detects that the user is not within a permitted location, then theS1I-001-W01 system 10 will not render the virtual project room until it can verify that the user is in a permitted location.

[0156] In some embodiments, the access token recognition module 148 comprises a set of computer instructions that when executed by the processor cause the system 10 to manage the recognition of access tokens such as the access token 48 described above. By way of example access tokens may be used as a convenience to retrieve virtual project rooms 16 bypassing the usual file finder or explorer interfaces or as a multi-factor security measure. In some embodiments, the access token recognition module 148 facilitates a secure connection.

[0157] In some embodiments, the access token pairing module 150 comprises a set of computer instructions that when executed by the processor cause the system 10 to pair a virtual project room 16 to an access token 48 and enables the setting of associated optional security measures.

[0158] In some embodiments, the access token information retrieval module 152 comprises a set of computer instructions that when executed by the processor cause the system 10 to manage the reading of an access token 48 (e.g., the image or any other signal the access token 48 may emit or require as described above) and in some instances pass that information to other modules for interpretation.

[0159] In some embodiments, the multi -factor security module 154 comprises a set of computer instructions that when executed by the processor cause the system 10 to acquire and reconcile information from multi-factor authentication methods, such as those described above. In some embodiments, this feature may be hosted in the app 14. In some embodiments, this feature may be hosted on a separate secure server.

[0160] Referring to Fig. 29, and by way of example, the room browsing and hosting modules 90 may include a room hosting module 156, room browsing module 158, and room link module 160. In some embodiments, room hosting module 156 comprises a set of computer instructions that when executed by the processor cause the system 10 enable hosting, browsing, and / or retrieval of virtual project rooms. In some embodiments, room browsing module 158 comprises a set of computer instructions that when executed by the processor cause the system 10 toS1I-001-W01 implement advanced ways to quickly browse through and preview virtual project rooms. In some embodiments, room link module 160 comprises a set of computer instructions that when executed by the processor cause the system 10 to manage “teleports” (similar to links) between virtual project rooms.

[0161] By way of example, the interactive data immersion and collaboration system 10 of the present disclosure may be used in a variety of scenarios for a variety of users in the business, education, government, and military spaces. In some embodiments, the interactive data immersion and collaboration system 10 of the present disclosure may be used as discovery system and / or a project planning system. In some embodiments, the interactive data immersion and collaboration system 10 of the present disclosure may be used in the virtualization of highly instrumentalized environments, for example, as an AR environment that recreates the capabilities of a control room of a test range, like a laser test range, in AR. In some embodiments, the interactive data immersion and collaboration system 10 of the present disclosure may be used as a scenario editor for wargame play (e.g., at the Naval War College), for example, as an AR environment that recreates existing wargame scenarios including typical materials, game progression, and umpiring. In some embodiments, the interactive data immersion and collaboration system 10 of the present disclosure may be used for portable / mobile ranges, for example to copy instrument panels in existing static ranges so that laser tests can be conducted anywhere. In some embodiments, the interactive data immersion and collaboration system 10 of the present disclosure may be used for training on combinatorial innovation, for example as an educational course aid offered to participants to help them visualize solving problems they bring to the course. In some embodiments, the interactive data immersion and collaboration system 10 of the present disclosure may be used as virtual real estate hosting for virtual project rooms, for example as a database platform that stores and makes accessible virtual project rooms for customers. In some embodiments, the interactive data immersion and collaboration system 10 of the present disclosure may be used as a mobile cockpit for remote vehicle testing, for example as a virtual environment that allows pilots to control remote vehicles during field testing. In some embodiments, the interactive data immersion and collaboration system 10 of the present disclosure may be used as a digital library of technologies, for example a virtual environment containing browsable compendium of technologies that may be used to solve problems. In some embodiments, the interactive data immersion and collaboration system 10 of the presentS1I-001-W01 disclosure may be used in salvage operations, for example an immersive control room for salvage operations that puts the operator in a salvage site virtually. In some embodiments, the interactive data immersion and collaboration system 10 of the present disclosure may use machine learning to make virtual project rooms searchable, for example as part of a system that provides next gen features by accessing and populating information required in a project. In some embodiments, the interactive data immersion and collaboration system 10 of the present disclosure may be used in game scenarios, for example by rendering a copy of existing game cell layouts into a shared virtual project room that to allows teams of users to collaborate to solve a game problem. In some embodiments, the interactive data immersion and collaboration system 10 of the present disclosure may be used to digitize an existing war room / strategy room. In some embodiments, the interactive data immersion and collaboration system 10 of the present disclosure may be used to copy existing data interface tools offering different ways to communicate and collect data. In some embodiments, the interactive data immersion and collaboration system 10 of the present disclosure may be used in the virtualization of mobile testing rigs suitable for remote usage, for example as an environment for use in remote test sites that provides the same capabilities as a instrumentalized environment in AR. In some embodiments, the interactive data immersion and collaboration system 10 of the present disclosure may be used as an operational control room for example in rail management and State Departments of Transportation, e.g., an AR environment that recreates existing operational control rooms (e g., MBTA, VDOT, Railyards). In some embodiments, the interactive data immersion and collaboration system 10 of the present disclosure may be used as a collaborative meeting space for people who are in different locations, for example through the creation of a virtual project room where multiple people can see the same information and each other (e.g., as avatars) live in AR space. In some embodiments, the interactive data immersion and collaboration system 10 of the present disclosure may be used as a command center for cyber security defense testing, for example as an AR scenario oversight environment for conducting "attack" and "defense" cyber security testing. In some embodiments, the interactive data immersion and collaboration system 10 of the present disclosure may be used as a training aid for a course on solving problems with technology, allowing for collaboration with the instructor and other students in a shared virtual environment.S1I-001-W01

[0162] Figs 30-31 are example block diagrams of computer-implemented electronic devices 200, 250 that may be used to implement the systems and methods described in this document, as either a client or as a server or plurality of servers. Computing device 200 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Computing device 250 is intended to represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smart-phones, augmented reality headwear, virtual reality headwear, and other similar computing devices. In this example, computing device 250 may represent hand-held computing device 46 and / or AR / VR headwear 12, while computing device 200 may represent stationary computer 18 and / or computing systems that serve as the cloud 22 referenced in this disclosure. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations described and / or claimed in this document.

[0163] Referring to Fig. 30, computing device 200 includes a processor 202, memory 204, a storage device 206, a high-speed interface 208 connecting to memory 204 and high-speed expansion ports 210, and a low-speed interface 212 connecting to low speed bus 214 and storage device 206. Each of the components 202, 204, 206, 208, 210, and 212, are interconnected using various busses, and may be mounted on a common motherboard or in other manners as appropriate. The processor 202 can process instructions for execution within the computing device 200, including instructions stored in the memory 204 or on the storage device 206 to display graphical information for a graphic user interface (GUI) on an external input / output device, such as display 216 coupled to high-speed interface 208. In other implementations, multiple processors and / or multiple buses may be used, as appropriate, along with multiple memories and types of memory. Also, multiple computing devices 200 may be connected, with each device providing portions of the necessary operations (e.g., as a server bank, a group of blade servers, or a multi-processor system).

[0164] The memory 204 stores information within the computing device 200. By way of example only, the memory 204 may be a volatile memory unit, non-volatile memory unit, or another form of computer-readable medium, such as a magnetic or optical disk (for example).S1I-001-W01

[0165] The storage device 206 is capable of providing mass storage for the computing device 200. In one implementation, the storage device 206 may be or contain a non-transitory computer-readable medium (e.g., any and all computer-readable media except transitory, propagating signals), such as a floppy disk device, a hard disk device, an optical disk device, or a tape device, a flash memory or other similar solid state memory device, or an array of devices, including devices in a storage area network or other configurations. A computer program product can be tangibly embodied in an information carrier. The computer program product may also contain instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer- or machine-readable medium, such as the memory 204, the storage device 206, or memory on processor 202.

[0166] The high-speed interface 208 manages bandwidth-intensive operations for the computing device 200, while the low-speed interface 212 manages lower bandwidth-intensive operations. Such allocation of functions is by way of example only. In one implementation, the high-speed interface 208 is coupled to memory 204, display 216 (e.g., through a graphics processor or accelerator), and to high-speed expansion ports 210, which may accept various expansion cards (not shown). In the implementation, low-speed interface 212 is coupled to storage device 206 and low-speed expansion port 214. The low-speed expansion port may include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet) and may be coupled to one or more input / output devices, such as a keyboard 218, a printer 220, a scanner 222, or a networking device such as a switch or router 224, e.g., through a network adapter.

[0167] The computing device 200 may be implemented in a number of different forms. For example, it may be implemented as a standard server, or multiple times in a group of such servers. It may also be implemented as part of a rack server system. In addition, it may be implemented in a personal computer such as a laptop computer. Alternatively, components from computing device 200 may be combined with other components in a mobile device, such as device 250 (Fig. 31). Each of such devices may contain one or more of computing device 200, 250, and an entire system may be made up of multiple computing devices 200, 250 communicating with each other.S1I-001-W01

[0168] Referring to Fig. 31, computing device 250 includes a processor 252, memory 254, an input / output device such as a display 256, a communication interface 258, and a transceiver 260, among other components. The device 250 may also be provided with a storage device, such as a microdrive or other device, to provide additional storage. Each of the components 250, 252, 254, 256, 258, and 260, are interconnected using various buses, and several of the components may be mounted on a common motherboard or in other manners as appropriate.

[0169] The processor 252 can execute instructions within the computing device 250, including instructions stored in the memory 254. The processor may be implemented as a chipset of chips that include separate and multiple analog and digital processors. Additionally, the processor may be implemented using any of a number of architectures. For example, the processor 252 may be a CISC (Complex Instruction Set Computers) processor, a RISC (Reduced Instruction Set Computer) processor, or a MISC (Minimal Instruction Set Computer) processor. The processor may provide, for example, for coordination of the other components of the device 250, such as control of user interfaces, applications run by device 250, and wireless communication by device 250.

[0170] The processor 252 may communicate with a user through control interface 262 and display interface 264 coupled to a display 256. The display 256 may be, for example, a TFT (Thin-Film-Transistor Liquid Crystal Display) display or an OLED (Organic Light Emitting Diode) display, or other appropriate display technology. The display interface 264 may comprise appropriate circuitry for driving the display 256 to present graphical and other information to a user. The control interface 262 may receive commands from a user and convert them for submission to the processor 252. In addition, an external interface 266 may be provided in communication with processor 252, so as to enable near area communication of device 250 with other devices. External interface 266 may provide, for example, for wired communication in some implementations, or for wireless communication in other implementations, and multiple interfaces may also be used.

[0171] The memory 254 stores information within the computing device 250. The memory 254 can be implemented as one or more of a computer-readable medium or media, a volatile memory unit or units, or a non-volatile memory unit or units. Expansion memory 268 may alsoS1I-001-W01 be provided and connected to device 250 through expansion interface 270, which may include, for example, a SIMM (Single In Line Memory Module) card interface. Such expansion memory 268 may provide extra storage space for device 250 or may also store applications or other information for device 250. Specifically, expansion memory 268 may include instructions to carry out or supplement the processes described above, and may include secure information also. Thus, for example, expansion memory 268 may be provided as a security module for device 250, and may be programmed with instructions that permit secure use of device 250. In addition, secure applications may be provided via the SIMM cards, along with additional information, such as placing identifying information on the SIMM card in a non-hackable manner.

[0172] The memory may include, for example, flash memory and / or NVRAM memory, as discussed below. In one implementation, a computer program product is tangibly embodied in an information carrier. The computer program product contains instructions that, when executed, cause performance of one or more methods, such as those described above. The information carrier is a computer- or machine-readable medium, such as the memory 254, expansion memory 268, or memory on processor 252 that may be received, for example, over transceiver 260 or external interface 266.

[0173] Device 250 may communicate wirelessly through communication interface 258, which may include digital signal processing circuitry where necessary. Communication interface 258 may provide for communications under various modes or protocols, such as GSM voice calls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS, among others. Such communication may occur, for example, through radio-frequency transceiver 260. In addition, short-range communication may occur, such as using a Bluetooth, WiFi, or other such transceiver (not shown). In addition, GPS (Global Positioning System) receiver module 272 may provide additional navigation- and location-related wireless data to device 250, which may be used as appropriate by applications running on device 250.

[0174] Device 250 may also communicate audibly using audio codec 274, which may receive spoken information from a user and convert it to usable digital information. Audio codec 274 may likewise generate audible sound for a user, such as through a speaker, e.g., in a handset of device 250. Such sound may include sound from voice telephone calls, may include recordedS1I-001-W01 sound (e.g., voice messages, music files, etc.) and may also include sound generated by applications operating on device 250.

[0175] The computing device 250 may be implemented in a number of different forms, some of which are shown in the figure. For example, it may be implemented as a cellular telephone. It may also be implemented as part of a smart-phone, personal digital assistant, or other similar mobile device.

[0176] Additionally computing device 200 or 250 can include Universal Serial Bus (USB) flash drives. The USB flash drives may store operating systems and other applications. The USB flash drives can include input / output components, such as a wireless transmitter or USB connector that may be inserted into a USB port of another computing device.

[0177] Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0178] These computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor and can be implemented in a high-level procedural and / or object-oriented programming language, and / or in assembly / machine language. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, apparatus and / or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and / or data to a programmable processor.

[0179] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and aS1I-001-W01 pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0180] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (“LAN”), a wide area network (“WAN”), peer-to-peer networks (having ad-hoc or static members), grid computing infrastructures, and the Internet.

[0181] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0182] The term "coupled" is defined as connected, although not necessarily directly, and not necessarily mechanically. The use of the word "a" or "an" when used in conjunction with the term "comprising" in the claims and / or the specification may mean "one," but it is also consistent with the meaning of "one or more" or "at least one." The term "about" means, in general, the stated value plus or minus 5%. The use of the term "or" in the claims is used to mean "and / or" unless explicitly indicated to refer to alternatives only or the alternative are mutually exclusive, although the disclosure supports a definition that refers to only alternatives and "and / or."

[0183] The terms "comprise" (and any form of comprise, such as "comprises" and "comprising"), "have" (and any form of have, such as "has" and "having"), "include" (and any form of include, such as "includes" and "including") and "contain" (and any form of contain,S1I-001-W01 such as "contains" and "containing") are open-ended linking verbs. As a result, a method or device that "comprises," "has," "includes" or "contains" one or more steps or elements, possesses those one or more steps or elements, but is not limited to possessing only those one or more elements. Likewise, a step of a method or an element of a device that "comprises," "has," "includes" or "contains" one or more features, possesses those one or more features, but is not limited to possessing only those one or more features. Furthermore, a device or structure that is configured in a certain way is configured in at least that way but may also be configured in ways that are not listed.

[0184] All patents and publications mentioned in this specification are indicative of the levels of those skilled in the art to which the disclosure pertains. It is to be understood that while a certain form of the disclosure is illustrated, it is not to be limited to the specific form or arrangement herein described and shown. It will be apparent to those skilled in the art that various changes may be made without departing from the scope of the disclosure and the disclosure is not to be considered limited to what is shown and described in the specification and any drawings / figures included herein.

[0185] One skilled in the art will readily appreciate that the present disclosure is well adapted to carry out the objectives and obtain the ends and advantages mentioned, as well as those inherent therein. The embodiments, methods, procedures and techniques described herein are presently representative of the preferred embodiments, are intended to be exemplary and are not intended as limitations on the scope. Changes therein and other uses will occur to those skilled in the art which are encompassed within the spirit of the disclosure and are defined by the scope of the appended claims. Although the disclosure has been described in connection with specific preferred embodiments, it should be understood that the disclosure as claimed should not be unduly limited to such specific embodiments. Indeed, various modifications of the described modes for carrying out the disclosure which are obvious to those skilled in the art are intended to be within the scope of the following claims.

Claims

S1I-001-W01CLAIMSWhat is claimed is:

1. An interactive data immersion and collaboration system, comprising: a wearable device configured to render and display augmented reality within a user’s field of vision, the wearable device configured to be worn by a user, the wearable device comprising: a processor; a memory comprising a non-transitory computer readable medium; and a display unit; and a computer program product embodied in said non-transitory computer readable medium and comprising a set of computer instructions that, when executed by the processor, cause the wearable device to: generate an augmented reality room comprising at least one virtual data panel oriented in physical space at least partially around the user; and display, on the display unit, an interactive presentation of data wherein the wearable device is further configured to: receive, based on the user’s interaction with said data presented on said at least one virtual data panel, user instructions to edit or rearrange said data; and display, on the display unit, a real-time rendering of the edited or rearranged data on said at least one virtual display panel in response to said user instructions to edit or rearrange said data.

2. The interactive data immersion and collaboration system of claim 1, wherein said at least one virtual data panel comprises a plurality of virtual data panels positioned in a geometric arrangement.

3. The interactive data immersion and collaboration system of claim 2, wherein the geometric arrangement is circular, spiral, continuous, open, or spherical.

4. The interactive data immersion and collaboration system of claim 2, wherein the geometric arrangement includes at least one gap to simulate a door in the augmented reality room.S1I-001-W015. The interactive data immersion and collaboration system of claim 1 , wherein the augmented reality room is moveable in response to user hand or arm motion.

6. The interactive data immersion and collaboration system of claim 1, wherein the at least one virtual data panel is independently movable in response to user hand or arm motion.

7. The interactive data immersion and collaboration system of claim 1, wherein the data of said interactive presentation of data includes a single document, multiple documents, a video file, a real-time video feed, an image file, an external data source, a control actuator, a different data file, real-time sensor data, or access to a different augmented reality room.

8. The interactive data immersion and collaboration system of claim 7, wherein the control actuator is configured to facilitate real-time control of an external controllable object.

9. The interactive data immersion and collaboration system of claim 8, wherein the external controllable object is a drone, rover, or game piece.

10. The interactive data immersion and collaboration system of claim 1, wherein the wearable device is further configured to register the augmented reality room to a fixed position and orientation within a real space.

11. The interactive data immersion and collaboration system of claim 10, wherein the augmented reality room is configured for sharing with a second user present in the physical space and having a second wearable device configured to render and display augmented reality within the second user’s field of vision, the second wearable device comprising a second processor, a second memory stored on a second non -transitory computer readable medium, and a second display unit, the second wearable device configured to be worn by the second user.

12. The interactive data immersion and collaboration system of claim 11, wherein the shared augmented reality room is registered in the same position and orientation for the user and the second user.

13. The interactive data immersion and collaboration system of claim 11, wherein theS1I-001-W01 wearable device is further configured to display, on the display unit, a real-time rendering of the second user’s interactions with the shared augmented reality room.

14. The interactive data immersion and collaboration system of claim 11, wherein the second wearable device is further configured to display, on the second display unit, a real-time rendering of the user’s interactions with the shared augmented reality room.

15. The interactive data immersion and collaboration system of claim 1, wherein the wearable device is further configured to register the augmented reality room to a position and orientation within a real space relative to the user.

16. The interactive data immersion and collaboration system of claim 1, wherein the augmented reality room is configured for sharing with a second user present in a remote second physical space and having a second wearable device configured to render and display augmented reality within the second user’s field of vision, the second wearable device comprising a second processor, a second memory stored on a second non- transitory computer readable medium, and a second display unit, the second wearable device configured to be worn by the second user.

17. The interactive data immersion and collaboration system of claim 16, wherein the shared augmented reality room is registered in the same position and orientation for the user and the second user.

18. The interactive data immersion and collaboration system of claim 16, wherein the wearable device is further configured to display, on the display unit, a real-time rendering of the second user’s interactions with the shared augmented reality room.

19. The interactive data immersion and collaboration system of claim 11, wherein the second wearable device is further configured to display, on the second display unit, a real-time rendering of the user’s interactions with the shared augmented reality room.

20. The interactive data immersion and collaboration system of claim 1, further comprising an external computing device linked to the generated augmented reality room, the external computing device configured to edit or rearrange said data on said at least onevirtual data panel upon receiving instructions from the user to edit or rearrange said data on said at least one virtual data panel.

21. The interactive data immersion and collaboration system of claim 20, wherein the external computing device is configured to edit or rearrange the generated augmented reality room by editing a portable room file.

22. The interactive data immersion and collaboration system of claim 21, wherein the external computing device is configured to edit or rearrange the generated augmented reality room by linking with the wearable device.

23. The interactive data immersion and collaboration system of claim 1, further comprising an access token having an access feature required for the user to access the generated augmented reality room.

24. The interactive data immersion and collaboration system of claim 23, wherein the access feature is a security feature configured to control user access to the generated augmented reality room.

25. The interactive data immersion and collaboration system of claim 24, wherein the security feature comprises one or more of a printed image or logo, sticker, RFID tag, bar code, QR code, or a magnetic strip, digital code generator, or biologic identification factor.

26. The interactive data immersion and collaboration system of claim 23, wherein the access feature is a portable room format file.

27. The interactive data immersion and collaboration system of claim 26, wherein the portable room format file comprises a saved version of the augmented reality room configured such that, upon opening the portable room format file by the wearable device, the wearable device automatically populates the at least one virtual data panel of the augmented reality room with a specific set of data as as determined by the user in an initial setup, or as the relevant information was saved during or after a previous session in the augmented reality room.S1I-001-W0128. The interactive data immersion and collaboration system of claim 1 , wherein the generated augmented reality room comprises a portable command and control interface.

29. A computer program product embodied in said non-transitory computer readable medium and comprising a set of computer instructions that, when executed by a computer processor within a computing device, cause the computing device to: record a save file in said non-transitory computer readable medium at a first point in time, said save file comprising a layout of data elements in a particular state in a first set of one or more virtual data panels of a first augmented reality room, and populate, at a second point in time later than the first point in time, said layout of data elements in said particular state on a second set of one or more virtual data panels of a second augmented reality room.

30. The computer program product of claim 29, wherein said second set of virtual data panels is different from said first set of virtual data panels.

31. The computer program product of claim 29, wherein said second augmented reality room is different than said first augmented reality room.

32. The computer program product of claim 29, wherein said save file comprises a portable room format file.

33. The computer program product of claim 29, wherein said save file is stored on an access token, hard drive, wearable device, portable memory stick, or cloud server.

Images