Systems, methods, and media for displaying interactive augmented reality presentations

Abstract

The present invention provides systems, methods, and media for displaying interactive augmented reality presentations. In some embodiments, a system includes multiple head-mounted displays, a first head-mounted display including: a transparent display; and at least one processor, wherein the at least one processor is programmed to: determine that a first actual location among multiple actual locations in a physical environment of the head-mounted display is closest to the head-mounted display; receive first content including a first three-dimensional model; receive second content including a second three-dimensional model; display a first view of the first three-dimensional model at a first time using the transparent display; and display the first view of the second three-dimensional model at a second time after the first time using the transparent display based on one or more instructions received from a server.

Description

[0001] This application is a divisional application of the invention patent application with international application number PCT / US2017 / 064781, international application date December 5, 2017, national phase application number 201780085576.4, and invention name “System, method and medium for displaying interactive augmented reality display”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application is based upon and claims the benefit of and priority to U.S. Provisional Application No. 62 / 430,179, filed December 5, 2016, U.S. Provisional Application No. 62 / 479,214, filed March 30, 2017, U.S. Provisional Application No. 62 / 492,832, filed May 1, 2017, and U.S. Provisional Application No. 62 / 560,869, filed September 20, 2017. Each of the foregoing applications is hereby incorporated by reference in its entirety for all purposes. Background Art

[0004] Devices for displaying augmented reality and / or virtual reality content have recently become more prevalent. Displaying virtual reality content to a group of individuals who may or may not share a physical space is relatively easy because the experience is fully immersive and the content can be displayed within a common frame of reference that can be inserted into one or more users. For example, a virtual reality display can include a scene captured by one or more cameras (e.g., a natural scene, a sporting event, etc.), and multiple users accessing the content can be placed at the same location within the content, but presented to these users with different fields of view based on their selected orientations. As another example, a virtual reality display can include computer-generated content, and users can participate in an interactive experience where different users can be positioned at different locations within the computer-generated content and can interact with each other. In such examples, the content can have a common frame of reference, and the content displayed to the users can be based on the users' positions and orientations relative to the common frame of reference. While virtual reality content has the potential to allow interaction between users within the content scene, due to the fully immersive nature of virtual reality, users' ability to interact within their physical space is severely limited. In contrast, while devices displaying augmented reality content can allow users to interact with the physical environment and each other relatively easily, displaying the same content to multiple users is more difficult because different augmented reality devices used by users in the same room may not use the same coordinate system. Therefore, even if different users are viewing the same augmented reality content, the content may not be displayed corresponding to the same physical space and may have different orientations, etc. Furthermore, augmented reality devices are typically not configured to cooperate with the display of content according to instructions from the presenter.

[0005] Therefore, new systems, methods, and media for displaying interactive augmented reality presentations are needed. Summary of the Invention

[0006] According to some implementations of the disclosed subject matter, systems, methods, and media are provided for displaying an interactive augmented reality presentation.

[0007] According to some embodiments of the disclosed subject matter, a system for displaying an interactive augmented reality presentation is provided, the system comprising: a plurality of head-mounted displays, wherein a first head-mounted display of the plurality of head-mounted displays comprises: a transparent display; and at least one processor, wherein the at least one processor is programmed to: determine that a first actual location among a plurality of actual locations in a physical environment of the head-mounted display is closest to the head-mounted display; receive first content comprising a first three-dimensional model; receive second content comprising a second three-dimensional model; use the transparent display to present a first view of the first three-dimensional model at a first time; and use the transparent display to present a first view of the second three-dimensional model at a second time after the first time based on one or more instructions received from a server.

[0008] In some embodiments, the at least one processor is further programmed to receive the instructions from the server at a third time after the first time and before the second time, wherein the one or more instructions include at least one instruction to cause the first head-mounted display to stop displaying the first content and start displaying the second content in response to receiving the one or more instructions.

[0009] In some embodiments, the system further includes a server, wherein the server: receives an indication to present the second content; and in response to receiving the indication to present the second content, sends the one or more instructions to at least the first head-mounted display.

[0010] In some embodiments, the system further includes a second head-mounted display among the plurality of head-mounted displays, wherein the second head-mounted display includes at least one second processor, wherein the at least one second processor is programmed to cause the second head-mounted display to act as the server.

[0011] In some implementations, the first content and the second content are received as part of a content library prior to presenting the first content.

[0012] In some implementations, the first content is received before the first time, and the second content is received after the first time.

[0013] In some implementations, the at least one processor is further programmed to request the second content in response to receiving an indication to present the second content.

[0014] In some embodiments, the at least one processor is further programmed to receive the second content from the server as pushed content to be presented without sending a request for the second content.

[0015] In some embodiments, the system also includes a head tracking system, wherein the at least one hardware processor is further programmed to: receive an indication to display the first content at a first actual location among multiple actual locations in the physical environment of the first head-mounted display; receive an indication to display the second content at a second actual location among multiple actual locations in the physical environment of the first head-mounted display; display the first view of the first three-dimensional model at the first actual location using the transparent display based at least in part on a first head position relative to the first actual location determined based on information output by the head tracking system; determine a second head position relative to the first actual location based on information output by the head tracking system, the second head position being different from the first head position; in response to determining the second head position, display a second view of the first three-dimensional model based on the second head position; determine, from the multiple actual locations, that the second actual location among the multiple actual locations in the physical environment of the head-mounted display is closest to the head-mounted display; in response to determining that the second actual location is closest to the head-mounted display from the multiple actual locations, display the second three-dimensional model anchored at the second actual location.

[0016] In some embodiments, the at least one hardware processor is further programmed to, in response to determining that the second actual location is located closest to the head-mounted display from the plurality of actual locations, disable presentation of the first three-dimensional model at the first actual location.

[0017] In some embodiments, the head tracking system includes an inertial measurement unit.

[0018] In some implementations, the at least one processor is further programmed to request the second content from the server in response to determining that the head mounted display moves closer to the second actual location.

[0019] In some embodiments, the at least one processor is further programmed to receive the second content from the server as pushed content to be presented without sending a request for the second content.

[0020] In some embodiments, the system also includes: a computing device comprising: at least one input device; and at least one third processor programmed to: receive input from the input device indicating that the first content is to be associated with the first physical location and the second content is to be associated with the second physical location; and send information indicating that the first content is to be associated with the first physical location and the second content is to be associated with the second physical location to a server, wherein the information is formatted as an Extensible Markup Language (XML) document.

[0021] In some embodiments, the system further comprises: a user input device configured to communicate with the first head-mounted display, wherein the user input device comprises: a second inertial measurement unit; and at least one fourth processor programmed to: send information indicating physical movement of the user input device to the first head-mounted display; and wherein the at least one processor of the first head-mounted display is further programmed to: display a first user interface element in combination with the first three-dimensional model at a position based on the information indicating physical movement of the user input device; and send information indicating the position of the user interface element relative to the first three-dimensional model to a server; and a second head-mounted display among the multiple head-mounted displays comprises at least one fifth processor programmed to: receive information from the server One or more instructions , the instruction indicates the position of displaying the user interface element relative to the first three-dimensional model; based on the information related to the position of the user interface element relative to the first three-dimensional model received from the server, displaying the user interface element at the position relative to the first three-dimensional model.

[0022] In some implementations, the user interface element is displayed as a dot at the location.

[0023] In some embodiments, the system further includes at least one image capture device, wherein the at least one processor is further programmed to: capture an image of a code displayed by a mobile computing device; and send information encoded in the image to a server to pair the first head-mounted display with the mobile computing device.

[0024] In some implementations, the at least one processor is further programmed to: prompt a user to log into a service provided at least in part by the server using the mobile computing device; and prompt the user to capture an image of the code to log into the service in the first head-mounted display.

[0025] In some embodiments, the first actual location is associated with at least a first portion of the physical environment on a first side of a barrier in the physical environment and a second portion of the physical environment on the first side of the barrier, and the second actual location is associated with at least a third portion of the physical environment on the second side of the barrier and a fourth portion of the physical environment on the second side of the barrier, the second portion being disposed in the physical environment between the first portion and the third portion, and wherein the at least one hardware processor is configured to: determine that the first head-mounted display is disposed within the first portion; display the first content at the first actual location in response to determining that the first head-mounted display is disposed within the first portion; determine that the first head-mounted display has moved into the third portion; and display the second content at the second actual location in response to determining that the first head-mounted display has moved into the third portion.

[0026] In some embodiments, the at least one hardware processor is configured to: receive third content including a third three-dimensional model; determine that the first head-mounted display has moved to the fourth portion; load the third content into a working memory of the first head-mounted display in response to determining that the first head-mounted display has moved to the fourth portion; determine that the first head-mounted display has moved from the fourth portion to the first portion; and display the third content loaded in the working memory of the first head-mounted display in response to determining that the first head-mounted display has moved from the fourth portion to the first portion.

[0027] In some embodiments, the system further comprises a server, wherein the server is programmed to: receive information about the position of the first head-mounted display during a first time period while the first content is presented by the first head-mounted display; receive a video corresponding to the field of view of the wearer of the first head-mounted display captured during the first time period; record information about the position of the first head-mounted display; record video corresponding to the field of view of the first head-mounted display during the first time period; receive information about the position of the second head-mounted display during a first time period while the first content is presented by the second head-mounted display; receive a video corresponding to the field of view of the wearer of the second head-mounted display captured during the first time period; record information about the position of the second head-mounted display; the first computing device receiving a request from the first computing device to display the content recorded during the first time period; and causing the first computing device to simultaneously: display the first content; display a first avatar at a first position relative to the first content based on the recorded information about the position of the second head-mounted display; display a second avatar at a second position relative to the first content based on the recorded information about the position of the second head-mounted display; when the first head-mounted display is in the first position, a first window displays a portion of the video captured by the first head-mounted display; and when the second head-mounted display is in the second position, a second window displays a portion of the video captured by the second head-mounted display.

[0028] In some embodiments, the server is further programmed to: display a first user interface element in conjunction with the first avatar that indicates the line of sight of the wearer of the first head-mounted display; and display a second user interface element in conjunction with the second avatar that indicates the line of sight of the wearer of the second head-mounted display.

[0029] In some embodiments, a second head-mounted display among the multiple head-mounted displays includes: a virtual reality display; and at least one processor, wherein the at least one processor is programmed to: receive the first content; receive the second content; present the first view of the first three-dimensional model at the first time using the virtual reality display; and present the first view of the second three-dimensional model at the second time after the first time using the virtual reality display.

[0030] According to some embodiments of the disclosed subject matter, another system for displaying an interactive augmented reality presentation is provided, the system comprising: a plurality of head-mounted displays, wherein each of the plurality of head-mounted displays is configured to: receive content comprising first content and second content from a server; present the first content based at least in part on a proximity of the head-mounted display to a first actual location in a physical environment, wherein a first head-mounted display of the plurality of head-mounted displays and a second head-mounted display of the plurality of head-mounted displays that are located near the first actual location at a first time concurrently present the first content at the first actual location in a same orientation relative to the first actual location based on the proximity to the first actual location; and present the second content based at least in part on a proximity of the first head-mounted display to a second actual location in the physical environment, wherein a third head-mounted display of the plurality of head-mounted displays and a fourth head-mounted display of the plurality of head-mounted displays that are located near the second actual location at the first time concurrently present the second content at the second actual location in a same orientation relative to the second actual location based on the proximity to the second actual location.

[0031] According to some embodiments of the disclosed subject matter, a system for displaying interactive content is provided, the system comprising: a head-mounted display comprising a virtual reality display, wherein the head-mounted display is configured to: receive content comprising first content, second content, and third content; receive first presentation information and second presentation information; present the first content at a first location based on the first presentation information based at least in part on a proximity of a position of the head-mounted display to the first location in an environment; present the second content at a second location based on the first presentation information based at least in part on a proximity of the head-mounted display to the second location in the environment; and present the third content at the first location based on the second presentation information based at least in part on the proximity of the position of the head-mounted display to the first location in the environment.

[0032] According to some embodiments of the disclosed subject matter, a method for displaying an interactive augmented reality presentation is provided, the method comprising: determining that a first actual location among multiple actual locations in a physical environment of a head-mounted display is closest to the head-mounted display; receiving first content including a first three-dimensional model; receiving second content including a second three-dimensional model; using a transparent display to present a first view of the first three-dimensional model at a first time; and using the transparent display to present a first view of the second three-dimensional model at a second time after the first time based on one or more instructions received from a server.

[0033] According to some embodiments of the disclosed subject matter, another method for displaying an interactive augmented reality presentation is provided, the method comprising: receiving content including first content and second content from a server; presenting, by a head-mounted display, the first content based at least in part on a proximity of the head-mounted display to a first actual location in a physical environment, wherein a first head-mounted display of the plurality of head-mounted displays and a second head-mounted display of the plurality of head-mounted displays that are located near the first actual location at a first time concurrently present the first content at the first actual location in a same orientation relative to the first actual location based on the proximity to the first actual location; and presenting, by the head-mounted display, the second content based at least in part on a proximity of the first head-mounted display to a second actual location in the physical environment, wherein a third head-mounted display of the plurality of head-mounted displays and a fourth head-mounted display of the plurality of head-mounted displays that are located near the second actual location at the first time concurrently present the second content at the second actual location in a same orientation relative to the second actual location based on the proximity to the second actual location.

[0034] According to some embodiments of the disclosed subject matter, a method for displaying interactive content is provided, the method comprising: receiving content comprising first content, second content, and third content; receiving first presentation information and second presentation information; presenting the first content at a first location based on the first presentation information and using a virtual reality display of the head-mounted display based at least in part on a proximity of a position of the head-mounted display to the first location in an environment; presenting the second content at a second location based on the first presentation information and using the virtual reality display based at least in part on a proximity of the head-mounted display to the second location in the environment; and presenting the third content at the first location based on the second presentation information and using the virtual reality display based at least in part on the proximity of the position of the head-mounted display to the first location in the environment.

[0035] According to some embodiments of the disclosed subject matter, a non-transitory computer-readable medium containing computer-executable instructions is provided, which, when executed by a processor, causes the processor to perform a method for displaying an interactive augmented reality presentation, the method comprising: determining that a first actual location among multiple actual locations in a physical environment of a head-mounted display is closest to the head-mounted display; receiving first content including a first three-dimensional model; receiving second content including a second three-dimensional model; using a transparent display to present a first view of the first three-dimensional model at a first time; and based on one or more instructions received from a server, using the transparent display to present a first view of the second three-dimensional model at a second time after the first time.

[0036] According to some implementations of the disclosed subject matter, another non-transitory computer-readable medium containing computer-executable instructions is provided that, when executed by a processor, causes the processor to perform a method for displaying an interactive augmented reality presentation, the method comprising: receiving content including first content and second content from a server; presenting, by a head-mounted display, the first content based at least in part on a proximity of the head-mounted display to a first actual location in a physical environment, wherein a first head-mounted display of a plurality of head-mounted displays and a second head-mounted display of the plurality of head-mounted displays that are located near the first actual location at a first time concurrently present the first content at the first actual location in a same orientation relative to the first actual location based on the proximity to the first actual location; and presenting, by the head-mounted display, the second content based at least in part on a proximity of the first head-mounted display to a second actual location in the physical environment, wherein a third head-mounted display of the plurality of head-mounted displays and a fourth head-mounted display of the plurality of head-mounted displays that are located near the second actual location at the first time concurrently present the second content at the second actual location in a same orientation relative to the second actual location based on the proximity to the second actual location.

[0037] According to some embodiments of the disclosed subject matter, a non-transitory computer-readable medium containing computer-executable instructions is provided that, when executed by a processor, causes the processor to perform a method for displaying interactive content, the method comprising: receiving content comprising first content, second content, and third content; receiving first presentation information and second presentation information; presenting the first content at a first location based on the first presentation information and using a virtual reality display of the head-mounted display based at least in part on a proximity of a position of a head-mounted display to the first location in an environment; presenting the second content at a second location based on the first presentation information and using the virtual reality display based at least in part on a proximity of the head-mounted display to the second location in the environment; and presenting the third content at the first location based on the second presentation information and using the virtual reality display based at least in part on the proximity of the position of the head-mounted display to the first location in the environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Various objects, features, and advantages of the disclosed subject matter may be more fully understood by referring to the following detailed description of the disclosed subject matter when considered in conjunction with the following drawings, wherein like reference numerals represent like elements.

[0039] Figure 1An example of a head-mounted display that can be used in accordance with some implementations of the disclosed subject matter is shown.

[0040] Figure 2 An example of a system of networked head-mounted displays is shown in accordance with some implementations of the disclosed subject matter.

[0041] Figure 3A An example of a system for interacting with a presented image according to some implementations of the disclosed subject matter is shown.

[0042] Figure 3B An example of another system for interacting with presented images according to some implementations of the disclosed subject matter is shown.

[0043] Figure 4 An example of hardware that can be used to implement at least one head-mounted display, at least one server, and at least one user input device in accordance with some implementations of the disclosed subject matter is shown.

[0044] Figure 5 An example of information flow between a user computing device selecting and / or uploading content and / or presentation to a server and a head mounted display receiving the content and presenting the content according to the presentation is shown in accordance with some implementations of the disclosed subject matter.

[0045] Figure 6 An example of a process for distributing content to one or more head-mounted displays is shown in accordance with some implementations of the disclosed subject matter.

[0046] Figure 7 An example of a process for generating an augmented reality presentation using a head-mounted display is shown, in accordance with some implementations of the disclosed subject matter.

[0047] Figure 8 An example of a user device and a head mounted display performing a pairing operation with a server is shown in accordance with some implementations of the disclosed subject matter.

[0048] Figure 9 Shown in accordance with some embodiments of the disclosed subject matter in the login and / or above combined Figure 8 An example of information flow between a user device, a server, and a head-mounted display during a pairing operation is described.

[0049] Figure 10 An example of information flow between a first head-mounted display, a server, and a second head-mounted display for associating a 3D model with a user of the first head-mounted display (e.g., as an avatar) and presenting the 3D model through the second head-mounted display is shown in accordance with some implementations of the disclosed subject matter.

[0050] Figure 11 An example of a display space for sequentially presenting various models is shown, according to some implementations of the disclosed subject matter.

[0051] Figure 12 An example of a process for presenting a series of content in a limited physical space is shown, in accordance with some implementations of the disclosed subject matter.

[0052] Figure 13A An example of a first review mode according to some implementations of the disclosed subject matter is shown, in which a user of a first head-mounted display and a user of a second head-mounted display interact with a virtual accident victim.

[0053] Figure 13B Shown are some embodiments of the disclosed subject matter. Figure 13A An example of another view of the scene shown in at a different point in time.

[0054] Figure 13C Shown are some embodiments of the disclosed subject matter. Figure 13A and Figure 13B An example of a bird's-eye view of the scene shown at yet another point in time.

[0055] Figure 13D An example of a replay of an augmented reality session is shown, in accordance with some implementations of the disclosed subject matter.

[0056] Figure 14 An example of a process for reviewing media recorded by one or more head-mounted displays is shown in accordance with some implementations of the disclosed subject matter. DETAILED DESCRIPTION

[0057] Before describing in detail any embodiment of the disclosed subject matter, it should be understood that the disclosed subject matter is not limited in its application to the details of the construction and arrangement of the components set forth in the following description or shown in the following figures. The disclosed subject matter can have other embodiments and can be practiced or executed in various ways. In addition, it should be understood that the wording and terminology used herein are for descriptive purposes and should not be considered restrictive. The use of "including," "comprising," or "having" and their variations herein is intended to cover the items listed thereafter and their equivalents and additional items. Unless otherwise specified or limited, the terms "mount," "connect," "support," and "couple" and their variations are widely used and include direct and indirect mounting, connection, support, and coupling. In addition, "connect" and "couple" are not limited to physical and mechanical connections or couplings.

[0058] The following discussion is given to enable those skilled in the art to make and use embodiments of the disclosed subject matter. It will be apparent to those skilled in the art that various modifications to the illustrated embodiments will be apparent, and without departing from the embodiments of the disclosed subject matter, the general principles herein may be applied to other embodiments and applications. Therefore, the embodiments of the disclosed subject matter are not intended to be limited to the embodiments illustrated, but are consistent with the widest range of principles and features disclosed herein. The following detailed description will be read with reference to the accompanying drawings, in which the same elements in different drawings have the same reference numerals. The drawings, which are not necessarily drawn to scale, depict selected embodiments and are not intended to limit the scope of the embodiments of the disclosed subject matter. The skilled person will recognize that the examples provided herein have many useful alternatives and fall within the scope of the embodiments of the disclosed subject matter.

[0059] According to some embodiments of the disclosed subject matter, mechanisms (which may include systems, methods, and / or media) are provided for displaying interactive augmented reality presentations. In some embodiments, a user can generate an interactive presentation to be displayed using an augmented reality device. For example, a teacher can create and / or select content for displaying various holograms (e.g., 3D models, diagrams, 2D graphics, etc.) to aid in understanding the concepts the teacher is trying to convey to students. In a more specific example, a teacher can create a variety of different holograms showing different aspects of human anatomy.

[0060] In some embodiments, a user can select different parts of an interactive presentation to be presented at different real locations, different virtual locations, and / or at different times. For example, a teacher can connect different 3D models of human anatomy that illustrate different concepts (or illustrate the same concept in different ways) into an organized presentation, and / or can assign different 3D models of human anatomy to different real locations (e.g., as shown below in conjunction with Figure 2 described sites) and / or virtual locations (e.g., Figure 10 and Figure 11 In a more specific example, a teacher can discuss content displayed in a series of 3D models (e.g., in a manner similar to a slide show), while students have the opportunity to move freely around the models while listening to and / or interacting with the teacher. After the presentation is completed, different models can be displayed in different physical and / or virtual locations (e.g., based on instructions entered by the teacher when building the presentation), and students can move freely between the different models to further explore the concepts discussed in class.

[0061] In some embodiments, a user can use one or more user input devices to highlight one or more locations in the content displayed by the augmented reality device worn by the user. The same one or more locations can be highlighted by other augmented reality devices that display the same content to one or more other users. For example, a teacher can use an augmented reality device to view a 3D model and can use the augmented reality device and / or a separate device as a virtual "laser pointer" to highlight one or more features in the 3D model displayed by the augmented reality device worn by the teacher. In such an example, the corresponding pointer can be replicated at the same location on the same 3D model by other augmented reality devices worn by the teacher's students.

[0062] In some embodiments, multiple augmented reality devices can be networked with a common one or more servers to provide relevant content based on time and / or the location of the augmented reality devices. For example, as described above, a teacher can assign different 3D models to different physical spaces. In such an example, the server can only send content for the space closest to a particular student. This can allow teachers to make a large amount of content available without overwhelming the individual augmented reality devices. Additionally or alternatively, this can allow collaboration between different students (or other users) by making the input provided by a first user (e.g., through a "laser pointer" used by the teacher) visible on a hologram projected by a second user's device (e.g., a student's augmented reality device). In addition, providing content and / or instructions from a server to augmented reality devices in similar locations can promote collaboration among wearers of different augmented reality devices by ensuring that the various augmented reality devices all display the same content.

[0063] Although the mechanisms described herein are generally described with reference to their use in a classroom environment, these mechanisms can be configured for a variety of applications. For example, the mechanisms described herein can be configured to provide interactive augmented reality museum displays, provide multi-player augmented reality gaming experiences, provide a collaborative workspace for creating highly detailed 3D designs, and the like.

[0064] Figure 1 An example of a head mounted display (HMD) 100 is shown that can be used in accordance with some implementations of the disclosed subject matter. Figure 1As shown, the head-mounted display 100 may include a display processor 104 and a transparent display 102 that can be used to present images, such as holographic objects, to the eyes of a wearer of the HMD 100. In some embodiments, the transparent display 102 can be configured to visually enhance the appearance of a physical environment to a wearer viewing the physical environment through the transparent display 102. For example, in some embodiments, the appearance of the physical environment can be enhanced by graphical content (e.g., one or more pixels each having a corresponding color and brightness) displayed via the transparent display 102 to create an augmented reality environment. Additionally or alternatively, in some embodiments, the transparent display 102 can be configured to render a completely opaque virtual environment (e.g., by using one or more techniques to block the physical environment from being visible through the HMD 100). In some such embodiments, the HMD 100 can be used to present a virtual reality environment.

[0065] like Figure 1 As shown, in some embodiments, the transparent display 102 can include one or more image-generating elements (e.g., display pixels) located within a lens 106 (e.g., pixels of a see-through organic light-emitting diode (OLED) display). Additionally or alternatively, in some embodiments, the transparent display 102 can include a light modulator located at an edge of the lens 106.

[0066] In some embodiments, HMD 100 may include various sensors and / or other related systems. For example, HMD 100 may include gaze tracking system 108, which may include one or more image sensors that may generate gaze tracking data representing the gaze direction of the wearer's eyes. In some embodiments, gaze tracking system 108 may include any suitable number and any suitable arrangement of light sources and / or image sensors. For example, Figure 1 As shown, the gaze tracking system 108 of the HMD 100 can utilize at least one inward-facing sensor 109. In some implementations, the user can be prompted to allow gaze information to be acquired and used to track the position and / or movement of the user's eyes.

[0067] In some embodiments, the HMD 100 may include a head tracking system 110 that may utilize one or more motion sensors (e.g., Figure 1 ) to capture head pose data that can be used to track the position of the wearer's head, for example, by determining the direction and / or orientation of the wearer's head. In some embodiments, the head tracking system 110 may include an inertial measurement unit configured as a three-axis or three-degree-of-freedom position sensor system.

[0068] In some embodiments, the head tracking system 110 may also support other suitable positioning technologies, such as a global positioning system (GPS) or other global navigation system. In addition, although a specific example of a position sensor system has been described, it should be understood that any other suitable position sensor system may be used. For example, head posture and / or motion data may be determined based on sensor information from any suitable combination of sensors mounted on the wearer and / or external to the wearer, including but not limited to any number of gyroscopes, accelerometers, inertial measurement units (IMUs), GPS devices, barometers, magnetometers, cameras (e.g., visible light cameras, infrared cameras, time-of-flight depth cameras, structured light depth cameras, etc.), communication devices (e.g., Wi-Fi antennas / interfaces, Bluetooth, etc.), etc.

[0069] In some embodiments, HMD 100 can include an optical sensor system that can utilize one or more outward-facing sensors (e.g., optical sensor 114) to capture image data of the environment. In some embodiments, the captured image data can be used to detect motion captured in the image data, such as gesture-based input and / or any other suitable motion by a user wearing HMD 100, another person in the field of view of optical sensor 114, or a physical object within the field of view of optical sensor 114. Additionally, in some embodiments, one or more outward-facing sensors can capture 2D image information and / or depth information from the physical environment and / or physical objects within the environment. For example, the outward-facing sensors can include depth cameras, visible light cameras, infrared cameras, positional tracking cameras, and / or any other suitable image sensors or combinations of image sensors.

[0070] In some embodiments, a structured light depth camera can be configured to project structured infrared illumination and generate image data of the illumination reflected from the scene onto which the illumination is projected. In such embodiments, a depth map of the scene can be constructed based on the spacing between features in various areas of the imaged scene. Additionally or alternatively, in some embodiments, a time-of-flight depth camera can be incorporated into HMD 100 that is configured to project pulsed infrared illumination onto the scene and detect the illumination reflected from the scene. In some embodiments, illumination can be provided by infrared light source 116.

[0071] In some implementations, HMD 100 may include a microphone system that may include one or more microphones that can capture audio data, such as microphone 118. In other examples, audio can be presented to the wearer through one or more speakers (e.g., speaker 120).

[0072] In some embodiments, the HMD 100 may include a controller, such as the controller 122, which may include, for example, a processor and memory that communicate with various sensors and systems of the HMD 100 (as described below in conjunction with Figure 4 In some embodiments, the controller may store instructions in a memory that are executable by the processor to receive signal inputs from the sensor, determine the posture of the HMD 100, and adjust display properties of content displayed using the transparent display 102.

[0073] In some embodiments, the HMD 100 can have any other suitable features or combinations of features, such as those described in U.S. Patent No. 9,495,801 to Microsoft Technology Licensing, LLC, which is hereby incorporated by reference in its entirety. The description of the HMD 100 herein is merely illustrative of hardware that can be used in conjunction with the disclosed subject matter. However, the disclosed subject matter can be used with any suitable augmented reality device, such as a Manufactured and / or the devices described in U.S. Patent No. 8,847,988, U.S. Patent No. 8,941,559, U.S. Patent Application Publication No. 2014 / 0160001, each of which is hereby incorporated by reference in its entirety.

[0074] Figure 2 An example 200 of a system of networked HMDs 100 is shown according to some implementations of the disclosed subject matter. Figure 2As shown, system 200 may include various HMDs 100-1 through 100-5 located in the same physical space (e.g., in the same room). System 200 may include various stations 202-1 through 202-3, which may correspond to points in the physical space at which one or more images are to be displayed by HMD 100. For example, each station 202 may be used by HMD 100 as a predetermined spatial anchor for one or more images. In a more specific example, when an HMD (e.g., HMD 100) is instructed to display an image corresponding to station 202, as long as the station is in the field of view, even if the HMD is moved relatively far away from the station (e.g., more than 5 meters from the station), the station 202 may define a spatial anchor for the image to be displayed by the HMD. As another example, each station 202 may be used to define a coordinate system in which HMD 100 may place one or more images. In a more specific example, site 202 can define the origin of a global coordinate system, wherein an HMD (e.g., HMD 100) is instructed to display an image only when the user is relatively close to a particular site and / or when the HMD is instructed to display only one or more images corresponding to the nearest site in the field of view (e.g., as described below in conjunction with system 200 and in conjunction with Figure 6 The image corresponding to the site is displayed on the HMD 100 (as described above). In some embodiments, each site 202 can be passive and / or active. For example, one or more sites 202 can be a piece of paper with specific symbols (e.g., letters, numbers, icons, QR codes, etc.) that can be recognized by the HMD 100 (e.g., from an image captured by the optical sensor 114). As another example, one or more sites 202 can be specific objects or parts of objects (e.g., furniture, action figures, toys, etc.) that can be recognized by the HMD 100 (e.g., from an image captured by the optical sensor 114). As yet another example, one or more sites 202 can be active devices that can communicate with the HMD 100, such as Bluetooth devices (e.g., Bluetooth low energy beacons). As yet another example, a site 202 can be an active or passive RFID device that can communicate with the HMD 100. In some embodiments, the location of the sites 202 can be visually highlighted by the HMDs 100-1 to 100-5 to help the user move to each site. For example, one or more sites 202 in the user's field of view may be represented by an image of a ball, text, a summary of the content presented in connection with the site, and / or any other suitable visual aid.

[0075] In some embodiments, the system 200 may include a server 204 that can control the content displayed in connection with each site. In some embodiments, the server 204 can be implemented using any suitable computing device, such as a server computer, an HMD, a tablet computer, a smartphone, a personal computer, a laptop computer, etc. In some embodiments, each HMD 100 can be connected to a communication network 206 via a communication link 208, and the server 204 can be connected to the communication network 206 via a communication link 212. In some such embodiments (e.g., embodiments in which the site 202 is an active device), the site 202 can be connected to the communication network 206 via a communication link 210. In some embodiments, the user computing device 220 can be connected to the communication network 206 via a communication link 222. The communication network 206 can be any suitable communication network or combination of communication networks. For example, the communication network 206 may be a Wi-Fi network (which may include one or more wireless routers, one or more switches, etc.), a peer-to-peer network (e.g., a Bluetooth network, a Zigbee mesh network, etc.), a cellular network (e.g., a 3G network, a 4G network, etc., conforming to any suitable standard such as CDMA, GSM, LTE, LTE Advanced, WiMAX, etc.), a wired network, etc. The communication links 208, 210, and 212 may each be any suitable communication link or combination of communication links, such as a Wi-Fi link, a Bluetooth link, a cellular link, etc.

[0076] In some embodiments, a user can interact with server 204 via user computing device 220 to select content to be displayed in conjunction with each site 202. For example, a user can instruct server 204 to cause the HMD near site 202-2 to display an image representing an interactive 3D model of the human vascular system without certain other anatomical features (e.g., without muscles, without a skeletal system, etc.), while the user can instruct server 204 to cause the HMD near site 202-3 to display an image representing an interactive 3D model illustrating how the vascular system is integrated into certain major muscle groups. Note that these are provided as examples only, and the user can select any suitable content to be displayed in conjunction with each site. In some embodiments, a user can instruct server 204 to display different content for each site, causing server 204 to cause HMD 100 to display different content for each site 202. In some such embodiments, a wearer of HMD 100 can move between sites to view different content at each site, which the user may have selected to represent different concepts. Additionally or alternatively, in some embodiments, a user can instruct server 204 to display the same content at each site. For example, in such an embodiment, students in a class can follow along as the teacher discusses content displayed by all HMDs 100 in the system 200, regardless of which station each HMD 100 is located near. In some embodiments, a user can specify the location of one or more stations 202 relative to one or more reference points (e.g., a location in a room, a reference point relative to a particular device, etc.).

[0077] In some embodiments, a user can generate a presentation to be displayed in connection with one or more sites 202 via the user computing device 220 and / or the server 204. For example, such a presentation can include the content to be displayed at each site, the time at which the content will be displayed at the site (e.g., duration, time period during the presentation, etc.), the order in which the content will be displayed at each site (and / or across all sites), which content will be displayed to which HMD 100 and / or which user is associated with the HMD 100, and / or any other suitable information. In some embodiments, such a presentation can be transmitted by the server 204 to each HMD 100 at the time of presentation as a document that references the content to be displayed, the order in which the content will be displayed, the time at which the content will be displayed, which content will be displayed in connection with each site, etc. For example, the server 204 can send an XML file that the HMD 100 can use to generate a presentation based on the content already loaded on the HMD 100.

[0078] In some embodiments, a user can upload content and / or identification information of the content to be displayed via the HMD 100 from the user computing device 220 to the server 204. For example, a user can upload an anatomical model from the user computing device 220 to the server 204, which can be used to generate a 3D representation of one or more anatomical features (e.g., via the HMD 100). As another example, the user can provide location information (e.g., a URL) where the one or more anatomical models can be accessed. In some embodiments, the HMD 100 can download and / or save content at any suitable time. For example, an administrator can download, sideload, and / or otherwise transfer content to each HMD 100 for use during the presentation of augmented reality content and / or other presentations of such content. In a more specific example, if the HMD 100 is owned by an organization, users associated with the organization can preload content on each HMD 100. As another example, in some embodiments, users of the HMD 100 can download applications that can be used to display content. In such an example, a user who uploaded content can associate the content with an application, and the content can be downloaded along with the application and / or downloaded through the application after the application is installed on the HMD. As another example, in some embodiments, the user of the HMD 100 can select the content to download and / or the content can be automatically downloaded based on known information about the user. In a more specific example, the user can be allowed to download content based on the user's registration to participate in an event associated with the content (e.g., a class). In another more specific example, the content can be automatically downloaded based on the user's association with the event during which the content is presented (e.g., a class). As another example, when the HMD 100 needs the content, the content can be streamed and / or downloaded on demand. In some embodiments, the user can be prompted to pay for the content and / or sign up for a subscription to the content.

[0079] In some embodiments, the user computing device 220 can be any suitable computing device or combination of devices, such as a personal computer, a laptop computer, a tablet computer, a smartphone, a wearable computer, a head-mounted display (e.g., HMD 100), etc. In some embodiments, a user can use the user computing device 220 and / or the server 204 to select content, generate presentations, upload content, etc. using any suitable technology or combination of technologies. For example, the user computing device 220 can execute a presentation application from memory that is configured to generate and / or edit presentations of content for any suitable number of sites and / or any suitable number of HMDs. As another example, the user computing device 220 can interact with a presentation application executed by another computing device (e.g., server 204, a cloud server, etc.) over the network 206 via a web browser executed by the computing device 220 or other application that facilitates interaction with the remotely executed presentation application.

[0080] In some embodiments, each HMD 100 can execute an application that can interact with the server 204 (e.g., via the communication network 206) to present content associated with each site. When a particular HMD 100 comes within a predetermined distance of a site (e.g., when HMD 100-2 approaches site 202-2), the server 204 can provide content associated with site 202-2. Networking the HMD 100 with the server 204 can facilitate the HMD 100 to present more content than the HMD 100 can present from memory. Furthermore, networking the HMD 100 with the server 204 can facilitate a presenter (e.g., a user of HMD 100-1) to control content presented by various other HMDs 100 during an interactive presentation.

[0081] In some embodiments, the system 200 can use any suitable technique or combination of techniques to determine which content a particular HMD 100 will display. For example, the HMD 100 can determine which station (or stations) it is closest to and can request content associated with the station from the server 204 and / or can display content associated with the station from a memory. In such an example, the HMD 100 can use any suitable technique to determine which station (or stations) is closest, such as by analyzing image data captured by an outward-facing camera (e.g., optical sensor 114), analyzing the strength of various signals (e.g., Bluetooth signals) received from various stations 202, analyzing the GPS coordinates of the HMD 100 determined using a GPS receiver, etc. As another example, the HMD 100 can provide information (e.g., one or more images, signal strengths of various signals, GPS coordinates, etc.) to the server 204 that can determine which station (or stations) is closest to the HMD 100. As another example, the server 204 may receive information from the station 202 (e.g., as a list of HMDs that are closer to the station than other nearby stations, as distances from the station to various HMDs, etc.) that indicates which HMDs are closest to the station based on any suitable data (e.g., signal strengths of Bluetooth signals received by the station from the various HMDs). In such an example, each station may coordinate with other nearby stations so that each HMD is listed only by a single station. Which HMDs can be determined.

[0082] In some embodiments, HMD 100 may determine to present different content at any suitable time. For example, HMD 100-2 may determine that HMD 100-2 has left the vicinity of site 202-2 and is approaching site 202-3, and based on this determination, may present and / or request content to site 202-3 in response to determining that HMD 100-2 is approaching site 202-3. Additionally or alternatively, in some embodiments, server 204 may push instructions and / or content to HMD 100-2 at any suitable time. For example, server 204 may determine that HMD 100-2 has left the vicinity of site 202-2 and is approaching site 202-3, and in response to determining that HMD 100-2 is approaching site 202-3, may push instructions to present content associated with site 202-3 (and / or may push the content itself) to the HMD. As another example, server 204 may receive instructions to change the content currently being presented by any one or more suitable sites 202 (e.g., from a presenter). In response to such instructions, the server 204 can push instructions to display new content (and / or the new content itself) to the appropriate HMD 100. As another example, the server 204 can receive instructions that specify a specific HMD 100 to receive specific content at a specific time. In such an example, after receiving an instruction from the server 204 to associate content with a site (and / or after receiving the content itself), the wearer of the specific HMD 100 can be prompted to move to the specified site where the HMD 100 is displaying content.

[0083] In some embodiments, a presenter can use one or more sites 202 to control content being presented by HMDs in the vicinity of other sites. For example, in some embodiments, a wearer of HMD 100-1 can use any suitable user interface device to interact with content being presented at site 202-1 to control content being presented at one or more of sites 202-2 and 202-3 (and / or any other suitable sites). In such an example, the wearer of HMD 100-1 can use any suitable input device or combination of devices, such as a voice input to a voice recognition system, gestures as input to a gesture recognition system (e.g., integrated into HMD 100-1), an input device of another computing device (e.g., a touch screen of a computing device such as a smartphone, tablet, laptop; a mouse and / or keyboard of a computing device such as a laptop, personal computer), a dedicated input device (e.g., in combination with the following), or a combination of input devices. Figure 3AIn some embodiments, the server 204 may transmit additional information to the HMD 100 during content presentation, such as instructions for one or more HMDs 100 on how to present the content to be presented and / or additional content. Figure 3A and Figure 3B As shown, a user of a first HMD 100-1 can use an input device to point to (e.g., via a line in space, a point on the content, the user's hand, etc.) a specific portion of content being displayed by the HMD 100-1, and the server 204 can send instructions to one or more other HMDs 100 displaying the same content, causing each of those HMDs to display supplemental content that indicates that the user of the HMD 100-1 is pointing to the specific portion of the content. In some embodiments, such additional information can be used to control the display of content by the HMDs 100. For example, the user of the HMD 100-1 can control the display by inputting to the HMD 100-1 (and / or any other suitable device), and one or more other HMDs 100 can receive instructions and / or content from the server 204 that cause the one or more other HMDs 100 to change the content being displayed based on the input from the user of the HMD 100-1. In some embodiments, the HMD 100 can access a recording of the presentation (e.g., including changes to the presentation content), and instructions sent by the server 204 during the presentation can be included with the recording (which can include, for example, a document indicating which content to present) (e.g., as a file) and / or the instructions can be streamed while the HMD 100 presents the recording. For example, the HMD 100 can present a particular 3D model based on the first portion of the presentation, and during the presentation, an indicator can be presented in association with the 3D model based on instructions sent by the server 204 during the presentation of the 3D model. In such an example, audio recorded during the presentation can also be presented to the user (e.g., via the speaker 120). This can facilitate the user experiencing the presentation to experience the presentation as if they were present at the presentation.

[0084] In some embodiments, audio information can also be associated with each station, which can be presented in conjunction with the visual content by the HMD 100. Additionally or alternatively, in some embodiments, audio can be recorded at each station (e.g., by hardware that is part of the station 202 and / or by the microphone 118 of one or more HMDs 100). In some embodiments, audio can be recorded for later access (e.g., as a learning aid) at the request of the wearer of a particular HMD 100.

[0085] Although the HMDs 100-1 through 100-5 are described above as being in close proximity to one another (e.g., in the same room), the HMDs in the system 200 can be positioned in close proximity to one another and / or remotely located. For example, the system 200 can be used to collaborate and / or interact with one or more wearers of the HMDs 100 that are located at one or more remote locations. In some embodiments, two HMDs 100 can be remotely located if there is no line of sight between them. For example, two HMDs 100 can be considered remotely located if they are located in different rooms, regardless of whether they are connected to the same local area network (LAN) or to different networks. As another example, two HMDs 100 that are connected to different LANs can be considered remotely located from one another. As yet another example, two HMDs 100 that are connected to different subnets can be considered remotely located from one another. In some embodiments, for example, as described below in conjunction with Figure 3B As described above, two HMDs 100 that are at a distance from each other may be used to collaborate by representing remote users with avatars in conjunction with a hologram displayed by at least one of the two HMDs 100 .

[0086] In some embodiments, the server 204 can be located in close proximity or remotely from the HMD 100. Additionally, in some embodiments, multiple servers 204 (which can be located in different physical locations) can be used to provide different content, provide redundant functionality, and the like. In some embodiments, one HMD in the system 200 can perform one or more operations of the server 204 described herein, such as instructing other HMDs when to distribute update information by displaying movement, and the like. For example, the local HMDs 100 in the system 200 can be interconnected to form a mesh network, and the HMD acting as the server 204 (e.g., HMD 100-1) can control the operation of the other HMDs by providing update information. Additionally, in some embodiments, the HMD acting as the server 204 can be a node in the mesh network and can communicate over another network (e.g., LAN, cellular, etc.) to receive other information, such as information related to a remote user (e.g., described below in conjunction with Figure 3B In some such implementations, the HMD acting as server 204 may determine to which HMD(s) to distribute information indicating that the remote user's avatar is to be presented in conjunction with the hologram, placement information for the avatar, and the like.

[0087] In some embodiments, a user of any suitable HMD (e.g., HMD 100-1) can control the presentation of content (e.g., in a manner similar to a slideshow) by providing input to the HMD. For example, to move forward through an augmented reality presentation, the user can tap a user input (e.g., a button, a touch sensor, etc.) and / or perform a gesture that can be captured by an image sensor (e.g., optical sensor 114). As another example, to move backward through an augmented reality presentation, the user can tap and hold a user input (e.g., hold a button, press and hold on a touch sensor, etc.) and / or perform a different gesture. In some embodiments, HMD 100-1 can generate an instruction to move forward (or backward) through the augmented reality presentation and send the instruction to server 204 (which can be another HMD), which can cause the other HMD to present new content based on the user input received at HMD 100-1. Additionally or alternatively, in some embodiments, if HMD 100-1 acts as server 204, HMD 100-1 can, in response to receiving input, cause other HMDs to display new content based on the user input received at HMD 100-1. Although system 200 is generally described in conjunction with displaying an augmented reality display in a physical environment on an augmented reality device, the system can be configured to display any type of mixed reality (e.g., an augmented virtual reality display) or a full virtual reality display. For example, rather than sites 202 corresponding to locations in a physical environment, sites can correspond to points in a virtual environment. Additionally or alternatively, in some embodiments, one or more HMDs in system 200 can be augmented reality devices, while other HMDs are virtual reality devices. For example, a remotely located user can use a virtual reality device to join a group at a particular site. In such an example, the virtual reality device can display a 3D model, avatars corresponding to other users (e.g., users of the augmented reality device physically proximate to the site, and / or other users of the virtual reality device) that are positionally relative to the 3D model representing the user's current view of the 3D model, and the like. In some embodiments, a user of a virtual reality device can use any technique to cause the view of a displayed 3D model to change, such as using input received from a user input device (e.g., a game controller, a touchpad, etc.), output indicative of a user's body movement (e.g., rotation, translation, etc.), or any other suitable information. Additionally or alternatively, in some embodiments, a user of a virtual reality device can adopt the viewpoint of an augmented reality device that is viewing the same 3D model.In some embodiments, a virtual reality device used to join a group around a particular site can display one or more portions of video captured at and / or near the site (e.g., by a camera capturing 360-degree video of the site's surroundings) to generate a mixed reality display, can display the 3D model in a virtual environment that may or may not be similar to the site's physical environment (e.g., a virtual room that may include other sites), or can display only the 3D model and information about other users (e.g., displaying the 3D model in a blank environment (e.g., using a single background color)).

[0088] Figure 3A An example 300 of a system for interacting with a displayed image is shown according to some implementations of the disclosed subject matter. Figure 3A As shown, the user input device 302 can communicate with the HMD 100-1 via a communication link 304. In some embodiments, the communication link 304 can be any suitable communication link that can facilitate communication between the user input device 302 and the HMD 100-1. For example, the communication link 304 can be a wired link (e.g., a USB link, an Ethernet link, a dedicated wired communication link, etc.) and / or a wireless link (e.g., a Bluetooth link, a Wi-Fi link, etc.). In some embodiments, the user input device 302 can include any suitable sensor for determining the position of the user input device 302 relative to one or more other devices and / or objects (e.g., the HMD 100-1, the station 202, a particular body part of the wearer of the HMD 100-1, etc.), and / or a relative change in position (e.g., based on sensor output indicating that the user input device 302 has accelerated in a particular direction, that the user input device 302 has rotated in a certain direction, etc.). For example, in some implementations, user input device 302 may include one or more accelerometers, one or more gyroscopes, one or more electronic compasses, one or more image sensors, an inertial measurement unit, or the like.

[0089] In some implementations, the user input device 302 can be used as a pointing device by the wearer of the HMD 100-1 to highlight a particular portion of content being presented by the HMD 100 (e.g., a portion of the hologram 306-1), to select a particular portion of the hologram 306-1, to cause a particular portion of the hologram 306-1 to move in a particular manner (e.g., in a “click and drag” type action), etc. For example, a user interface element 308 that highlights a particular area of ​​the hologram 306-1 can be presented in conjunction with the hologram 306-1 at a position based on the direction in which the user input device 302 is pointed relative to the hologram 306-1. Figure 3AAs shown, the second HMD 100-2 presenting a second hologram 306-2 including the same content being presented in the hologram 306-1 may also present the user interface element 308 (e.g., based on instructions received from the server 204, which may be implemented by an HMD such as HMD 100-1, HMD 100-2, and / or another HMD).

[0090] In some implementations, HMD 100-1 and / or server 204 can receive data from user input device 302 indicating motion and / or position data of user input device 302. Based on the data from user input device 302, HMD 100-1 and / or server 204 can determine a position and / or orientation of user interface element 308 to be presented as part of hologram 306-1 (and as part of any other hologram presented by another HMD 100 presenting the same content as hologram 306-2).

[0091] In some embodiments, the user input device 302 can be an integral part of the HMD 100-1, which can determine the HMD 100-1 relative to the hologram 306-1 (as described above in conjunction with Figure 2 2-1 or any other suitable location). Information about which direction the HMD 100-1 is pointing can be used to infer the direction the wearer's eyes are looking (e.g., which can be augmented based on gaze information in some cases). In some embodiments, the inferred location where the wearer of the HMD 100 is looking can be used as input for positioning the user interface element 308 relative to the hologram 306-1 (e.g., as a line, point, multiple points, etc. that indicate where the wearer of the HMD 100-1 is looking).

[0092] In some embodiments, the user input device 302 can be a separate device that can transmit position information to the HMD 100-1 and / or the server 204. The position information can then be used to generate the user interface element 308 to indicate the location at which the wearer of the HMD 100-1 is pointing. Any suitable technique or combination of techniques can be used to generate the user interface element based on the position information of the user input device 302. For example, in some embodiments, the HMD 100-1 and / or the server 204 can determine the position of the user input device 302 relative to a portion of the wearer's body (e.g., the center of the user's body, which can be inferred based on the position of the HMD 100-1) and can draw a line that intersects the portion of the wearer's body and the user input device 302. As another example, in some embodiments, the HMD 100-1 and / or the server 204 can determine the position and / or orientation of the user input device 302 relative to the hologram 306-1 and can draw a line from the user input device 302 based on the position and orientation. In such an example, the position of user input device 302 may need to be calibrated more accurately than in the previous example and may be calculated in the local coordinates of the hologram. Thus, in such an example, the accuracy of the position of displayed user interface element 308 may vary based on the accuracy of the calibration, the distance from the anchor point (or other position reference) of hologram 306-1 (because the distance farther from the anchor point may be less accurately determined), etc.

[0093] As another example, in some embodiments, HMD 100-1 and / or server 204 can receive a first input from user input device 302 that can initiate generation of user interface element 308 on hologram 306 by signaling a wearer of HMD 100-1. Upon receiving the first user input (e.g., a first button press), HMD 100-1 and / or server 204 can generate user interface element 308 at a default location based on the wearer's current position relative to hologram 306-1 (e.g., a straight line from the user's body toward the center of hologram 306-1). After the first button press, HMD 100 and / or server 204 can change the orientation of the user interface element (e.g., the point at which the line passes through hologram 306-1 and / or the direction in which the line intersects hologram 306-1) based on the output received from user input device 302. In such an example, after the first button press, the wearer can translate the user interface device 302 to raise / lower and / or move the point where the line intersects the hologram 306-1 left / right, and can rotate the user input device 302 to change the orientation at which the line intersects the hologram 306-1. Upon receiving the second user input, the HMD 100-1 and / or the server 204 can freeze the position and / or orientation of the user interface element 308 relative to the hologram 306-1. This control scheme, in which the line is displayed in the hologram 306-1, can have some similarities to the operation of a mouse used to control a pointer (e.g., a cursor) in a 2D graphical user interface.

[0094] In some embodiments, the HMD 100 can be used to administer an exam for a user of the HMD 100. For example, a user of the server 204 can create a presentation that displays content to the HMD 100 such that no two HMDs display the same content in the same location and orientation. That is, in such an example, rather than promoting collaboration, content can be distributed so that each user is presented with personalized content. In some embodiments, the administering user (e.g., the instructor or proctor administering the exam) can be allowed to display the same content as any HMD 100 displaying the exam content. In some embodiments, a user (e.g., a student) can use a pointing device, such as the user input device 302, to point to a specific location within a hologram to provide an answer to an exam question. Additionally, in some embodiments, the HMD 100 and / or the server 204 can determine the location within the hologram that the user highlighted using the pointing device to automatically determine whether the user correctly answered the exam question. For example, a user can be prompted to locate a specific muscle on an anatomical hologram, and the server 204 can determine whether the location pointed to by the user using the user input device 302 corresponds to that muscle.

[0095] Figure 3BAn example 310 of another system for interacting with a displayed image is shown according to some implementations of the disclosed subject matter. Figure 3B As shown, a first HMD 100-1 worn by a first user 312 at a first location can display a hologram 306-1. In some embodiments, the HMD 100-1 can track the position of the user's 312 hand 314 relative to the hologram 306-1. In some embodiments, the HMD 100-1 can use any suitable technology or combination of technologies to track the position and / or orientation of the user's hand. For example, the HMD 100-1 can use images generated by one or more image sensors (e.g., optical sensor 114) and / or any other suitable data (e.g., depth information in the scene) to visually track the position of the user's hand. As another example, the HMD 100-1 can use one or more sensors to track the position of the user's hand to sense the position of a device held by the user's hand (or otherwise attached to the user's hand).

[0096] In some implementations, HMD 100-1 may send information to server 204 indicating the position of HMD 100-1 and the user's hand relative to hologram 306-1. Figure 3B As shown, the server 204 can send information to the second HMD 100-2 that displays a hologram 306-2, which includes the same content as the hologram 306-1, where the information can indicate the location of the avatar 316 representing the user 312 of the HMD 100-1 to be displayed relative to the hologram 306-2. The HMD 100-2 can use such information to display the avatar 316 and the handheld element 318 to the second user 320 using the hologram 306-2. In some embodiments, the HMD 100-1 can be caused to display the avatar of the user 320 in conjunction with the hologram 306-1 (not shown).

[0097] Figure 4 4 shows an example of hardware that may be used to implement at least one of the HMD 100, the server 204, and the user input device 302 according to some implementations of the disclosed subject matter. Figure 4 As shown, in some embodiments, the HMD 100 may include a processor 402, a display 404, one or more input devices 406, one or more communication systems 408, and / or a memory 410. In some embodiments, the processor 402 may be any suitable hardware processor or combination of processors, such as a central processing unit (CPU), a graphics processing unit (GPU), etc. In some embodiments, the display 404 may include any suitable display device, such as the one described above in combination with a processor. Figure 1In some embodiments, the input device 406 may include any suitable input device and / or sensor for receiving user input, such as the gaze tracking system 108, the head tracking system 110, the motion sensor 112, the optical sensor 114, the microphone 118, etc.

[0098] In some embodiments, the communication system 408 may include any suitable hardware, firmware, and / or software for transmitting information via the communication network 206 and / or any other suitable communication network. For example, the communication system 408 may include one or more transceivers, one or more communication chips and / or chipsets, etc. In more specific examples, the communication system 408 may include hardware, firmware, and / or software for establishing a Wi-Fi connection, a Bluetooth connection, a cellular connection, etc.

[0099] In some embodiments, the memory 410 may include any one or more suitable storage devices for storing instructions, values, etc., which may be used, for example, by the processor 402 to display content using the display 404, communicate with the server 204 via one or more communication systems 408, and the like. The memory 410 may include any suitable volatile memory, non-volatile memory, storage, any other suitable type of storage medium, or any suitable combination thereof. For example, the memory 410 may include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid-state drives, one or more optical drives, and the like. In some embodiments, the memory 410 may have encoded thereon a computer program for controlling the operation of the HMD 100. In some such embodiments, the processor 402 may execute at least a portion of the computer program to display content (e.g., one or more holograms), receive content from the server 204, send information to the server 204, and the like. In some embodiments, the HMD 100 may render the content received from the server 204 using any suitable hardware and / or software, such as Unity 3D, available from Unity Technologies. Additionally, in some embodiments, any suitable communication protocol may be used to transmit control data, image data, audio, etc. between the HMD 100 and the server 204, such networking software being available from Unity Technologies.

[0100] In some embodiments, the server 204 may include a processor 412, a display 414, one or more input devices 416, one or more communication systems 418, and / or memory 420. In some embodiments, the processor 412 may be any suitable hardware processor or combination of processors, such as a central processing unit, a graphics processing unit, etc. In some embodiments, the display 414 may include any suitable display device, such as a computer monitor, a touch screen, a television, etc. In some embodiments, the input device 416 may include any suitable input device and / or sensor that can be used to receive user input, such as a keyboard, a mouse, a touch screen, a microphone, etc.

[0101] In some embodiments, the communication system 418 may include any suitable hardware, firmware, and / or software for transmitting information via the communication network 206 and / or any other suitable communication network. For example, the communication system 418 may include one or more transceivers, one or more communication chips and / or chipsets, etc. In more specific examples, the communication system 418 may include hardware, firmware, and / or software for establishing a Wi-Fi connection, a Bluetooth connection, a cellular connection, etc.

[0102] In some embodiments, the memory 420 may include any one or more suitable storage devices that can be used to store instructions, values, etc., which can be used, for example, by the processor 412 to display content using the display 414, communicate with one or more HMDs 100, etc. The memory 420 may include any suitable volatile memory, non-volatile memory, memory, any other suitable type of storage medium, or any suitable combination thereof. For example, the memory 420 may include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid-state drives, one or more optical drives, etc. In some embodiments, the memory 420 may have encoded thereon a server program for controlling the operation of the server 204. In such embodiments, the processor 412 may execute at least a portion of the computer program to send content (e.g., one or more holograms) to one or more HMDs 100, receive content from one or more HMDs 100, and receive instructions from one or more devices (e.g., HMD 100-1, user input device 302, another server, a personal computer, a laptop, a tablet computer, a smartphone, etc.).

[0103] In some embodiments, the user input device 302 may include a processor 422, one or more input devices 224, one or more communication systems 426, and / or a memory 428. In some embodiments, the processor 422 may be any suitable hardware processor or combination of processors, such as a central processing unit, a graphics processing unit, etc. In some embodiments, the input device 424 may include any suitable input device and / or sensor that can be used to receive user input, such as one or more physical or software buttons, one or more motion sensors, a microphone, a touchpad, etc.

[0104] In some embodiments, the communication system 426 may include any suitable hardware, firmware, and / or software for transmitting information via the communication link 304 and / or any other suitable communication link. For example, the communication system 426 may include one or more transceivers, one or more communication chips and / or chipsets, etc. In a more specific example, the communication system 426 may include hardware, firmware, and / or software for establishing a Wi-Fi connection, a Bluetooth connection, a cellular connection, etc.

[0105] In some embodiments, the memory 428 may include any one or more suitable storage devices that can be used to store instructions, values, etc., which can be used, for example, by the processor 422 to determine when to receive user input, record sensor data, communicate sensor data with one or more HMDs 100, etc. The memory 428 may include any suitable volatile memory, non-volatile memory, memory, any other suitable type of storage medium, or any suitable combination thereof. For example, the memory 428 may include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid-state drives, one or more optical drives, etc. In some embodiments, the memory 428 may have encoded thereon a computer program for controlling the operation of the user input device 302. In such embodiments, the processor 422 may execute at least a portion of the computer program to send data (e.g., representing sensor output) to one or more HMDs 100, send data (e.g., representing sensor output) to one or more servers 204, etc.

[0106] Figure 5 An example 500 of information flow between a user computing device 220 selecting and / or uploading content and / or presentation to a server 204 and an HMD 100 receiving the content and presenting it according to the presentation is shown in accordance with some implementations of the disclosed subject matter.

[0107] like Figure 5As shown, at 502, in some embodiments, a user computing device 220 may select a content repository from which to create one or more presentations. In some embodiments, a user of the user computing device 220 may select a content repository using any suitable technique or combination of techniques. For example, in some embodiments, a user may create content using any suitable application or combination of applications executed at least in part by the user computing device 220. As another example, a user of the user computing device 220 may select content from a webpage associated with a creator and / or publisher of the content and may download the content to the user computing device 220 and / or the server 204. As yet another example, a user of the user computing device 220 may select content using an application executed at least in part by the user computing device 220 that provides access to such content. As yet another example, a user of the user computing device 220 may select content from a storage medium coupled to the user computing device 220 (e.g., a USB drive, an external hard drive, an optical storage medium, etc.).

[0108] In some embodiments, the content library may include a collection of pre-selected content generated by a content publisher, such as a collection of 3D anatomical models of human anatomy, from which a user may select a particular combination of features to be displayed. Additionally or alternatively, in some embodiments, the content library may include 3D models collected by a user of the user computing device 220 from various sources.

[0109] At 504, in some embodiments, the user computing device 220 may upload the content repository and / or information indicating where the content can be accessed to the server 204. In some embodiments, the user computing device 220 may upload the content and / or location information to the server 204 using any suitable technology or combination of technologies. For example, the user computing device 220 may upload the content and / or location information via an application program interface (API) associated with the server 204. As another example, the user computing device 220 may upload the content and / or location information via a web page and / or other application associated with the server 204. As yet another example, the user computing device 220 may upload the content and / or location information using any other suitable technology or combination of technologies (e.g., file transfer protocol (FTP)). Additionally or alternatively, in some embodiments, the user computing device 220 may store the content repository locally (e.g., using a hard drive coupled to the user computing device 220).

[0110] At 506, in some implementations, the server 204 may receive content uploaded by the user computing device 220 and / or may download content from a location identified by the user computing device 220, and may store the content in memory associated with the server 204 so that the user of the user computing device 220 can access it (e.g., via a web page, an API, etc.).

[0111] At 508, in some embodiments, server 204 may provide the library of selected content to one or more HMDs, which will display at least a portion of the content during a presentation. In some embodiments, server 204 may load a portion of the content library or the entire library onto the HMD using any suitable technique or combination of techniques. For example, server 204 may provide at least a portion of the library as part of an application downloaded to the HMD. As another example, server 204 may provide at least a portion of the library in response to a request from the HMD. As yet another example, server 204 may provide content on demand and / or may stream content to the HMD when needed. In a more specific example, server 204 may provide content used in the presentation to the HMD before the presentation begins.

[0112] At 510, in some embodiments, a user of the user computing device 220 can generate a presentation from the selected content using a locally executed presentation application that accesses the content repository locally and / or remotely (e.g., from the server 204). In some embodiments, the presentation application can facilitate the creation of a scene that includes a subset of content from the content repository. For example, the presentation application can allow the user to select one or more specific body parts and layers (e.g., skin, muscle, bone, nerve, etc.) to be presented in a 3D model of the human anatomy, as well as the order in which the various parts and / or layers are presented. Additionally, in such an example, the presentation application can allow the user to highlight a specific portion of the 3D model, such as a specific muscle or nerve, within a portion of the presentation.

[0113] In some implementations, at 510, the user of the user computing device 220 can select specific content for each of the various sites (e.g., site 202), each of the various users or user groups (e.g., student users of the HMD 100), each of the various presentation spaces, and the like.

[0114] At 512, in some embodiments, the user computing device 220 may upload presentation information to the server 204 indicating which content to use in the presentation, the order in which the content is to be presented, supplemental content to be presented (e.g., text, audio, etc.), etc. In some embodiments, the presentation information may be formatted in any suitable format. For example, the presentation information may be uploaded from the user computing device 220 as an Extensible Markup Language (XML) document. As another example, the presentation may be uploaded as a media file including the content itself (in which case, the content repository may not need to be uploaded to the server 204). Note that in some embodiments, the server 204 may be omitted and / or may perform a more complex operation than the server 204. Figure 5 For example, in some embodiments, content can be loaded from the user computing device 220 (or any other source) to the HMD 100 without first uploading the content to the server 204. As another example, in some embodiments, presentation information can be sent from the user computing device 220 to the HMD 100 without going through the server 204 (although one or more other servers, such as nodes on the Internet, one or more email servers, one or more storage servers, etc., may be involved in the transmission of the presentation information).

[0115] Additionally or alternatively, to generate a presentation using an application executed by user computing device 220, user computing device 220 may interact with server 204 to generate the presentation using the application executed at least in part by server 204. For example, user computing device 220 may be used to provide a graphical user interface for the application executed by server 204, and input received by user computing device 220 may be used to control the operation of the application. In some implementations, if a presentation is generated using an application executed at least in part by server 204, server 204 may generate presentation information (e.g., as an XML document) without receiving such information from user computing device 220.

[0116] At 516, in some embodiments, server 204 may provide presentation information for at least a particular presentation to HMD 100, the presentation information to be used to experience the presentation. In some embodiments, server 204 may provide the presentation information at any suitable time. For example, server 204 may provide access to the presentation information after creating the presentation information for any HMD associated with a user included in a particular class or group to be presented (e.g., identified by the user of user computing device 220). As another example, server 204 may provide access to the presentation information at the start of a presentation (e.g., as indicated by the user of user computing device 220, HMD 100-1, etc.) to an HMD at a particular location, an HMD associated with a user included in a particular class or group to be presented (e.g., identified by the user of user computing device 220), and the like. In some embodiments, the presentation information may be received in conjunction with content. For example, the presentation information may be a file that is transmitted to HMD 100 as part of the content.

[0117] At 518, in some embodiments, the HMD that has received the presentation information may present the content (e.g., the content loaded onto the HMD at 508) according to the presentation information. Figure 2 As described above, the HMD (eg, HMD 100 - 2 ) may display a specific 3D model in conjunction with the third site based on an instruction received in the display information, where the instruction instructs the HMD to display the specific 3D model in conjunction with the third site.

[0118] Additionally or alternatively, in some embodiments, at 520, the server 204 can stream content to one or more HMDs based on the presentation information and based on the information received from the HMDs. Figure 2 As described above, the HMD may provide information about which site is closest to the HMD and / or within the field of view of the wearer of the HMD, and the server 204 may stream appropriate content to the HMD. As another example, the HMD may provide information that the server 204 may use to determine which site(s) is closest to the HMD and / or which site(s) is within the field of view of the wearer of the HMD, and the server 204 may stream appropriate content to the HMD. As yet another example, the server 204 may receive information from a first HMD (e.g., HMD 100-2) and / or a user input device (e.g., user input device 302) indicating that the wearer of HMD 100-2 is pointing to a specific portion of a 3D model, and the server 204 may stream this information to another HMD (e.g., HMD 100-1) viewing the same content as supplemental content to be integrated into the content by the second HMD.

[0119] At 522, in some implementations, the HMD 100 can present the streamed content received from the server 204. In some implementations, the streamed content presented at 522 can be any suitable streamed content, such as an entire 3D model, changes to a 3D model, supplemental content, and the like.

[0120] Figure 6 An example 600 of a process for distributing content to one or more head-mounted displays is shown in accordance with some implementations of the disclosed subject matter. Figure 6 As shown, at 602, process 600 can receive content to be used by the HMD to create one or more augmented reality displays. In some embodiments, the content can be received in any suitable format and / or from any suitable source. For example, a user of a user computing device (e.g., user computing device 220) can upload content (e.g., to server 204). As another example, the user can submit identification information of an address and / or credentials that can be used to retrieve the content. In some embodiments, the content can include any suitable visual information that can be used to create one or more 3D models (e.g., as holograms) that can be displayed by the HMD. For example, in some embodiments, the content can include various 3D models of human anatomy, which can be used by the HMD to generate holograms displaying various systems from the human anatomy.

[0121] At 604, process 600 may receive one or more instructions that may be used to present at least a portion of the content as an augmented reality presentation. For example, the instructions may identify different portions of the content to be presented, the order in which the different portions of the content are to be presented, the location at which particular content is to be presented (e.g., in a room, in conjunction with a site, etc.), etc. In some embodiments, the instructions may be received in any suitable format and / or using any suitable technology or combination of technologies. For example, in some embodiments, the instructions may be received as an XML file generated on a user device that receives user input representing the instructions. As another example, the instructions may be received from the user device via a web page or other suitable user interface that can be presented by the user device and that can receive user input for representing an augmented reality presentation. In such an example, the user input may be transmitted to a server associated with the web page, which may generate the instructions (e.g., as an XML document).

[0122] At 606, process 600 can receive a request for content from a head-mounted display. For example, as described above, in response to input from a user, an HMD associated with a particular user (e.g., such an association can be created by the user logging into an application executed by the HMD) can request the content based on information associated with the user (e.g., information indicating that the user is associated with a class for which the content is specified) and / or using any other suitable technique or combination of techniques. In some embodiments, the request can be received using any suitable technique or combination of techniques. For example, the HMD can navigate to a specific address associated with the content (e.g., an IP address, a URL, etc.). As another example, the HMD can access a repository of applications and submit a request to download (and / or download) the application that includes the content.

[0123] At 608, the process 600 may determine whether the HMD is authorized to receive the content. For example, the process 600 may determine whether the HMD is logged into a user account that is authorized to receive the content. As another example, the process 600 may determine whether an application executed by the HMD is logged into a user account that is authorized to receive the content. If the process 600 determines at 608 that the HMD is not authorized to receive the content ("No" at 608), the process 600 may move to 620 and prompt the user to log in to access the content (e.g., by displaying a login screen, by incorporating the following Figure 8 The method may include prompting the user to capture an image of the appropriate visual code, etc.), and may return to 608 when the user successfully logs the HMD into the appropriate account.

[0124] Otherwise, if process 600 determines at 608 that the HMD is authorized to receive the content ("YES" at 608), process 600 can move to 610. At 610, process 600 can send the content to the HMD that requested the content. In some implementations, the content can be sent using any suitable protocol, in any suitable format, and / or with any suitable compression applied.

[0125] At 612, process 600 can send a token to the authorized HMD, which can be used to access instructions for generating an augmented reality presentation from the content (and / or otherwise access the content and / or services associated with the content). In some embodiments, the token can take any suitable form. Additionally, in some embodiments, the token can be configured to expire after a predetermined period of time.

[0126] At 614, process 600 may receive a request for an augmented reality presentation based on content previously obtained by the head-mounted display. In some embodiments, the HMD may send the request in response to any suitable input. For example, a user may provide user input indicating that the HMD is to request an augmented reality presentation. As another example, the HMD may be configured to automatically request an augmented reality presentation when used in a specific location and / or at a specific time (e.g., without user intervention, requiring only user input to authorize the request, etc.). As yet another example, the HMD may be configured to automatically request an augmented reality presentation in response to receiving a specific signal (e.g., from a Bluetooth beacon) and / or in response to capturing one or more images of a specific visual cue (e.g., a QR code) (e.g., without user intervention, requiring only user input to authorize the request, etc.). In some embodiments, the request may be sent using a token sent to the HMD at 612. In some embodiments, the request received at 614 may be a request for instructions for a specific augmented reality presentation. Alternatively, in some implementations, the request received at 614 may be a general request for instructions for the augmented reality presentation, and the returned instructions may be instructions selected by the administrator.

[0127] At 616, process 600 can determine whether the HMD requesting instructions for the augmented reality presentation is authorized to receive the augmented reality presentation. For example, process 600 can determine whether the HMD submitted a valid token and / or whether the HMD is logged into an authorized user account. If process 600 determines that the HMD is authorized to receive the augmented reality presentation ("yes" at 616), process 600 can move to 618 and provide instructions to the HMD for presenting the augmented reality presentation using at least a portion of the content. Otherwise, if process 600 determines that the HMD is not authorized to receive the augmented reality presentation ("no" at 616), process 600 can move to 620 and prompt the user of the HMD to log in, indicate that the user is not authorized to receive the presentation, display an error message, etc. In some implementations, one or more portions of process 600 can be performed by server 204 (or multiple servers 204).

[0128] Figure 7 An example 700 of a process for generating an augmented reality presentation using a head mounted display is shown in accordance with some implementations of the disclosed subject matter. Figure 7As shown, at 702, process 700 can receive input representing user identification information. For example, the user can use a virtual keyboard displayed by HMD 100 to type a username and / or password. As another example, the user can speak information that can be used to identify the user, and process 700 can use any suitable voice recognition technology or combination of technologies to verify the identity of the user. As yet another example, in some embodiments, the user can pair another device (e.g., a smart phone, a tablet, a keyboard) with the HMD (e.g., using Bluetooth) and use the input of the paired device to enter the user identification information. As yet another example, as shown below in conjunction with Figure 8 As described, the HMD may capture images that may be used to identify the user.

[0129] At 704, process 700 may request content for generating an augmented reality presentation. In some embodiments, the request may be generated using any suitable technology or combination of technologies. For example, the HMD executing process 700 may navigate to a specific address (e.g., an IP address, a URL, etc.) based on user input and may request content at that address (e.g., by selecting a link on a web page, automatically responding to a request for address information, etc.). As another example, the HMD may request content using an application executed by the HMD, which may automatically download the content based on other information (e.g., information indicating that the user specified at 702 is enrolled in a course that uses the content).

[0130] At 706 , process 700 can receive and store content to be used in generating an augmented reality presentation. In some implementations, the content can be stored in any suitable location, such as memory 410 .

[0131] At 708, process 700 can receive a token that will be used to authenticate the HMD performing process 700 when requesting further information (e.g., instructions for generating a particular augmented reality presentation based on the content). In some implementations, the token can be received in any suitable format and can include any suitable information that can be used by, for example, server 204 to determine that the HMD is authorized to access the augmented reality presentation.

[0132] At 710, process 700 can send a request for information (and / or a token) that can be used to generate an augmented reality presentation based on the content. Figure 6As described in 614, process 700 can send a request for an augmented reality presentation in response to any suitable input. For example, the HMD can be navigated to a specific address (e.g., a URL), which can be used as a request for an augmented reality presentation. As another example, the HMD can automatically (e.g., without user intervention, requiring only user input to authorize the request, etc.) request an augmented reality presentation when certain conditions are met (e.g., the HMD is within a specific geofence at a specific time, the HMD is receiving a specific signal from another device, etc.).

[0133] At 712, process 700 can receive information (e.g., instructions) that can be used to generate an augmented reality presentation. In some implementations, the information can be received in any suitable format. For example, the information can be an XML document that indicates which content to present, in what order to present the content, at which location or locations to present the content, and so on.

[0134] At 714, process 700 can generate an augmented reality presentation based on the content based on the received instructions. For example, process 700 can determine which site is closest to the HMD performing process 700, and can display a hologram corresponding to the site based on the information received at 712. In some embodiments, process 700 can cause the HMD to send information indicating which portion of the augmented reality presentation is currently being displayed to the user to a server (e.g., server 204). In some embodiments, the server can use such information to transmit updated information about the specific content being displayed. Alternatively, the server can send the update information with identification information about which portion of the augmented reality presentation the update information corresponds to, and the HMD can interpret the identification information to determine whether the update information is relevant to the currently displayed content.

[0135] At 716, process 700 can determine whether any update information has been received. In some implementations, the update information can include any suitable information from a server (e.g., server 204) with instructions to change the presentation of content currently being presented as part of the augmented reality presentation. If process 700 determines that update information has not been received ("No" at 716), process 700 can return to 716 (and / or 714) to continue determining whether update information has been received (and / or to determine whether another portion of the augmented reality presentation is to be generated and / or presented). Otherwise, if process 700 determines that update information has been received ("Yes" at 716), process 700 can move to 718.

[0136] At 718, process 700 can generate an updated augmented reality presentation based on the update information. In some implementations, the update information can include any suitable information that can affect the augmented reality presentation. For example, the update information can be an indication of the position of a user interface element corresponding to the orientation of the pointing device (e.g., as described above in conjunction with Figure 3B As described above). In such an example, process 700 can insert a user interface element into a hologram currently being displayed (when the user interface element is visible from the perspective of the hologram being viewed). As another example, the update information can be an instruction to change a portion of the augmented reality display generated by the HMD, such as an instruction to move forward through the display or move backward through the display. As yet another example, the update information can be a 3D model (e.g., an avatar) of a user of another HMD (e.g., remotely located from the HMD performing process 700) and position information indicating where the avatar will be displayed relative to the hologram being displayed.

[0137] Figure 8 8 shows an example of a user device 220 and an HMD 100-1 performing a pairing operation with a server 204 according to some implementations of the disclosed subject matter. Figure 8As shown, a user device 802 (e.g., a smartphone, tablet, personal computer, laptop, etc. that may or may not be used as the user computing device 220) displays a visual code (more specifically, a QR code). In some embodiments, the user device 802 can be used to log in to an application or service associated with the content and / or augmented reality presentation described herein. In some such embodiments, in response to selecting a link (or any other suitable user action), the user device 802 can be used to retrieve the QR code (e.g., from the server 204) via a communication link 804 (e.g., which may include a LAN connection, a cellular connection, etc.). In some embodiments, the user device 802 can be used to retrieve an email with a link to the QR code, can receive a push notification with a link to the QR code (and / or the QR code itself), can receive a text message with a link to the QR code (and / or the QR code itself), can access an application through which the QR code can be retrieved from the server 204, etc. HMD 100-1 can capture an image of the QR code (e.g., using optical sensor 114), which can cause HMD 100-1 to submit a request (e.g., to server 204) via communication link 806, which can indicate that HMD 100-1 will be paired with user 802. In some embodiments, server 204 can associate user device 802 with HMD 100-1 for any suitable duration (e.g., the duration of a course, a day, a week, a month, a semester, an indefinite period of time, etc.) and can provide supplemental content via user device 802. For example, server 204 can push (or otherwise provide) content to user device 802 that supplements a portion of an augmented reality presentation currently being presented by HMD 100-1. In a more specific example, user device 802 can present supplemental text related to the augmented reality presentation. In another more specific example, user device 802 can present exam questions related to the augmented reality presentation. As another example, a user can use user device 802 to take notes (e.g., using a virtual keyboard, voice recognition, etc.), which can be automatically associated with the portion of the currently displayed augmented reality presentation. As yet another example, user device 802 can receive information from HMD 100-1 (e.g., images, text, video, audio, etc. captured during the presentation), which can be used by the user to supplement notes, as a memory aid, etc.

[0138] Figure 9 The above combination of some embodiments according to the disclosed subject matter is shown. Figure 9 An example 900 of information flow between the user device 802, the server 204, and the HMD 100 during the login and / or pairing operation is shown. Figure 9As shown, the user device 802 can receive user input for logging into the HMD 100. For example, at 902, the user device 802 can navigate to a webpage and / or execute an application associated with using the HMD 100-1 to generate an augmented reality presentation. In such an example, the user device 802 can be used to provide login credentials (e.g., a user name, password, biometric information, etc.).

[0139] At 904, the user device 802 may request a visual code scanned by the HMD 100-1 from the server 204. For example, as described above in conjunction with Figure 8 As described, the user device 802 may request a QR code at the server 204 that the HMD 100 - 1 may scan to pair the HMD 100 - 1 with the user device 802 and / or authenticate the HMD 100 - 1 with the server 204 .

[0140] At 906, if it is determined that the user device 802 is authorized, the server 204 may provide the requested visual code. In some embodiments, the server 204 may provide the visual code in any suitable format and / or any suitable type of data. For example, although the above description may be combined with Figure 8 A QR code is described, but this is merely an example, and any suitable machine-readable visual code may be used, such as a barcode, an image with stegenographically encoded information, etc. Additionally, in some embodiments, in addition to or in lieu of a visual code, the server 204 may cause the user device 802 to emit a coded sound that may be used by the HMD 100-1 to pair (and / or log in) with the user device 802.

[0141] At 908, the user device 802 may display the visual code and may prompt the user to aim the image sensor (eg, optical sensor 114) of the HMD 100-1 at the visual code (eg, Figure 8 ). At 910, HMD 100-1 may capture an image of a visual code presented by user device 802, and may decode information included in the visual code at 912.

[0142] At 914, HMD 100-1 may send information based on the information decoded from the image of the visual code to server 204. For example, HMD 100-1 may navigate to a particular web page and / or send a request to a particular address in response to capturing the image of the visual code.

[0143] At 916, server 204 may pair user device 802 with HMD 100-1 and / or may log in HMD 100-1 in response to receiving the information sent by HMD 100-1 at 914. In some embodiments, the indication that the two devices are paired may be indicated using any suitable technique or combination of techniques, such as through an entry in a database of devices maintained in database 204.

[0144] At 918, HMD 100-1 may display content from the augmented reality display, for example, as the wearer of HMD 100-1 moves to a different location. At 920, HMD 100-1 may send information indicating which content from the augmented reality display HMD 100-1 is currently displaying. Additionally or alternatively, server 204 may track which content is being displayed (e.g., based on update instructions for the display, based on the location of HMD 100-1, etc.).

[0145] At 922, the server 204 can push supplemental content and / or instructions to the user device 802 based on the content currently being displayed by the HMD 100-1. Additionally or alternatively, the user device 802 can periodically request updated information from the server 204 (if available). At 924, the user device 802 can display the supplemental content sent by the server 204 at 922.

[0146] Figure 10 An example 1000 of information flow between a first HMD 100-1, a server 204, and a second HMD 100-2 for associating a 3D model with a user of the HMD 100-1 (e.g., as an avatar) and presenting the 3D model through the HMD 100-2 is shown in accordance with some implementations of the disclosed subject matter. Figure 10 As shown, HMD 100-1 can receive input indicating that a particular 3D model is to be associated with the user of 100-1. In some embodiments, HMD 100-1 can display any number of 3D models from which the user can select a particular 3D model to associate with the user. For example, the 3D model can be a mask displayed on the face and / or head of the user by another HMD. As another example, the 3D model can be a 3D avatar that can be used as a representative of the user by another HMD. In some embodiments, the user can have a user interface to customize and / or design the 3D model. Additionally or alternatively, in some embodiments, a paired user device (e.g., as described above in conjunction with Figure 8 and Figure 9 The user device 802) can be used to select, customize, design a 3D model, etc., and the 3D model can be displayed by the HMD 100-1 for the user to view.

[0147] At 1004, HMD 100-1 (and / or a paired user device) may submit a request to the server 204 to associate the selected 3D model with the user. In some implementations, the request may include the 3D model, identification information of the 3D model, an address of the 3D model, etc. At 1006, the server 204 may associate the user of HMD 100-1 with the selected 3D model so that the 3D model is used in conjunction with the user.

[0148] At 1008, HMD 100-2 may request the 3D model associated with HMD 100-1. Alternatively, server 204 may push the 3D model to HMD 100-2. In some embodiments, the 3D model may be requested by scanning a visual code provided by the user of HMD 100-1 to the user of HMD 100-2. For example, the user of HMD 100-1 may present the visual code for scanning by HMD 100-2 (e.g., using a paired user device). As another example, the user of HMD 100-1 may share the visual code with the user of HMD 100-2 (e.g., via a message, such as a text message, sent between paired user devices associated with each HMD). At 1010, server 204 may send the 3D model information of the user of HMD 100-1 to HMD 100-2.

[0149] At 1012, HMD 100-2 may render a 3D model associated with the user of HMD 100-1. In some implementations, HMD 100-2 may render the 3D model in conjunction with the user's actual presence. For example, the 3D model may be a visor, helmet, or other equipment that HMD 100-2 may overlay on a portion of user 100-1 to provide an augmented reality effect. Additionally or alternatively, HMD 100-2 may render the 3D model as an avatar of the user of HMD 100-1.

[0150] although Figure 10 , the user of HMD 100 - 1 is shown in FIG, and when selecting a 3D model, the user of HMD 100 - 2 can select a 3D model to be used in conjunction with the user of HMD 100 - 1.

[0151] Figure 11 An example 1100 of a display space for sequentially presenting various models is shown in accordance with some implementations of the disclosed subject matter. Figure 11As shown, display space 1100 may include a physical area 1102, which may or may not be coextensive with a structure (e.g., a room) of a building. In some embodiments, a barrier 1104 may divide physical area 1102 into two or more areas. Although barrier 1104 is shown as a straight barrier that roughly divides physical area 1102 in half, this is merely an example, and barrier 1104 may have any suitable shape that can subdivide physical area 1102 into any suitable number of areas. For example, barrier 1104 may form a more complex shape, such as a stepped pattern, a "U" shape, etc. In some embodiments, barrier 1104 may be a physical barrier formed by a permanent or temporary structure erected in physical space 1102, such as a wall, a curtain, etc. Alternatively, in some embodiments, barrier 1104 may be a completely virtual barrier displayed by an HMD (e.g., HME 100-1), but which does not correspond to a physical barrier.

[0152] In some implementations, barrier 1104 can create multiple display spaces 1106 and 1108, which can, for example, serve as focal points for a virtual room created by barrier 1104. Note that while two display spaces are shown, this is merely an example, and any suitable number of display spaces can be associated with a barrier, perhaps with multiple display spaces on the same side of the barrier and another set of display spaces on the other side of the barrier.

[0153] In some embodiments, physical space 1102 may be subdivided into various sections 1110-1116, each of which may correspond to a virtual sensor. For example, one or more physical points within physical space 1102 may be designated as reference points for determining the section in which HMD 100 is located. In such an example, HMD 100 may determine the distance to one or more points to determine the current section of HMD 100. In such an example, the virtual sensor may be implemented by HMD 100. As another example, signals from one or more active devices, such as Bluetooth beacons, may be used to determine the current section of HMD 100. In some embodiments, when HMD 100-1 is within section 1110, it may load content for display space 1106 and display that content when display space 1106 is within the field of view of user 312. When user 312 moves from section 1110 to section 1112, HMD 100-1 may load content for display space 1108 in preparation for the user's movement around barrier 1104. In some embodiments, the order in which content is presented in the display space can be based on any suitable information, such as presentation information generated by a user of the user computing device 220, as described above in conjunction with Figure 2 and Figure 5In some implementations, when user 312 wearing HMD 100 - 1 moves into portion 1114 so that display space 1108 is within the field of view of user 312 , HMD 100 - 1 can begin showing content loaded in conjunction with display space 1108 when the user enters portion 1112 .

[0154] In some embodiments, when user 312 moves from section 1114 to section 1116, HMD 100-1 can load the next portion of content for space 1106, replacing the previously loaded content associated with that space, so that as user 312 moves around barrier 1104, HMD 100-1 displays the new content in display space 1106. However, if user 312 moves back to section 1114 again, HMD 100-1 can load the previous content for display space 1106, allowing the user to move backward through the display. In some embodiments, the series of content in the display can move forward as the user moves clockwise around barrier 1104, and can move backward when the user begins to move counterclockwise around the barrier. However, this is merely an example.

[0155] In some embodiments, the presentation can be specified to begin when the HMD first enters section 1110 (or any other designated section), so that the same content is presented in the same display space regardless of where the user enters physical space 1102. Alternatively, in some embodiments, the presentation can be made in the display space closest to the user when the user first enters physical space 1102.

[0156] In some embodiments, when the users have moved around barrier 1104 the same number of times, the HMDs associated with the users moving around barrier 1104 can together display the same content in display spaces 1106 and 1108. This can facilitate a shared experience between different users. In some such embodiments, two different HMDs that have not moved around barrier 1104 the same number of times can display different content in display spaces 1106 and 1108. Additionally or alternatively, in some embodiments, two or more different HMDs can be linked together so that they display the same content in display spaces 1106 and / or 1108. For example, a particular HMD can be designated as a master HMD, and HMDs paired with the master HMD can display the same content in display spaces 1106 and / or 1108 even if they have not moved around barrier 1104 together with the master HMD. As another example, the linked HMDs can collectively determine which content to display (e.g., based on where the majority of HMDs are located, based on the HMD that has moved around barrier 1104 the most times, based on the HMD that has moved around barrier 1104 the least times, etc.). In some embodiments, two or more different HMDs can be linked together using any suitable technique or combination of techniques. For example, the HMDs to be linked can be based on the existence of a configuration file that specifies linking the HMD with one or more other HMDs. As another example, upon initially entering physical space 1102, upon launching an application associated with physical space 1102, etc., a user can be prompted to select a group as part of a setup routine. In such an example, users wishing to have a shared experience can indicate that they are part of the same group. As yet another example, each HMD to be linked together can capture an image of a specific visual code (e.g., a QR code) that causes the HMDs to be linked. As yet another example, a device paired with the HMD (e.g., a smartphone, tablet, etc.) can be used to select a group to join and / or to select one or more other HMDs to link.

[0157] In some embodiments, when the wearer moves around the barrier to navigate to specific content, the HMD 100-1 can skip content that would otherwise be displayed. For example, the user can cause a menu to be displayed by the HMD 100-1 on the HMD 100-1 (e.g., by pressing a hardware button), and from the menu, the user can select a command to display specific content, advance through the content, etc. As another example, the user can access an application on a paired device (e.g., a paired smartphone) to select content to be displayed, advance through the content, etc. As yet another example, the HMD 100-1 can receive instructions and / or content from a server (e.g., server 204) indicating which content to display in the display spaces 1106 and / or 1108, which can cause the HMD 100-1 to display specific content.

[0158] In some embodiments, any suitable technique or combination of techniques may be used to generate the order in which content is presented in the display space as the user moves around the barrier. Figure 2 As described on the website, an administrative user can use an application to assemble a display that includes an ordered list of content to be displayed in the display space. As another example, the order of the content to be displayed can be randomized. As yet another example, the order in which the content is displayed can be determined based on the order in which it is stored in the content library. In some embodiments, the content to be displayed in the display space 1106 and / or 1108 can be preloaded on the HMD and / or can be received as the wearer moves through the physical space 1102. For example, all of the content to be displayed can be downloaded, uploaded, sideloaded to the HMD, etc. before the HMD begins displaying the content. As another example, a portion of the content can be preloaded on the HMD, and while the preloaded content is being displayed, the HMD can obtain additional content (e.g., by downloading the content from the server 204, by streaming the content from the server 204, etc.). As yet another example, the HMD can stream the content in real time as needed.

[0159] Figure 12 An example 1200 of a process for presenting a series of content in a limited physical space is shown in accordance with some implementations of the disclosed subject matter. Figure 12As shown, at 1202, process 1200 can determine an initial portion of a barrier (e.g., barrier 1104) from various portions surrounding the barrier. In some embodiments, process 1200 can determine the initial portion of the HMD using any suitable technique or combination of techniques, such as based on signals from one or more beacons (e.g., Bluetooth beacons), images captured by an image sensor of the HMD, a GPS location of the HMD, etc. In some embodiments, the determination at 1202 can be made by the HMD and / or by another computing device such as a server (e.g., server 204).

[0160] At 1204, process 200 can access content for a first display space corresponding to at least the initial portion and another portion. Figure 11 The two parts can be associated with one display space. In some embodiments, the content to be displayed in the first display space can be, for example, the first part of a series of content. In a more specific example, the content to be displayed in the first display space can be the first presentation in a series of presentations.

[0161] At 1206, process 1200 can present content for the first display space. Figure 11 As described above, process 1200 may cause content for the first display space to be displayed when the first display space is within the user's field of view.

[0162] At 1208, process 1200 can determine whether the HMD has moved to the last portion corresponding to the first display space. In some embodiments, each display space can be associated with one or more portions from which the display space can be seen. Process 1200 can use any suitable technique or combination of techniques (e.g., the above in combination with Figure 11 and 1202) to determine when the HMD has moved to the last portion corresponding to the first display space. In some embodiments, the portions can be defined in a particular order such that when the HMD moves to the portion corresponding to the next display space that is not visible from the last portion corresponding to the first display space, moving through the portions in the first direction causes the next portion of content from the presentation to be presented. For example, the last portion can be determined to be the last portion of the first display space that is visible clockwise from the first portion. In such an example, as the user moves clockwise around the barrier, the presentation can advance through the sequence, such as the sequence of the 3D scenes.

[0163] At 1210, if process 1200 determines that the HMD has moved to the last area corresponding to the first display space ("yes" at 1210), process 1200 can move to 1212. At 1212, process 1200 can access content for the next display space. In some embodiments, process 1200 can access content from internal memory and / or from remote memory (e.g., by downloading content from a server such as server 204). In some embodiments, process 1200 can preload content (so that the next display space is ready to be presented when it enters the field of view of the wearer of the HMD).

[0164] At 1214, process 1200 can determine whether the HMD has moved to the portion corresponding to the next display space (e.g., whether the user has moved around barrier 1104). If process 1200 determines that the HMD has moved to the next display space ("yes" at 1214), process 1200 can cause content corresponding to the next display space to be presented at 1216 while the next display space is in the field of view of the wearer of the HMD.

[0165] At 1210, if process 1200 determines that the HMD has not moved to the last area corresponding to the first display space ("No" at 1210), or if process 1200 determines that the HMD has not moved to the next display space ("No" at 1214), process 1200 can move to 1218 and determine whether the HMD has moved back to the first portion associated with the first display space. Alternatively, when content for the next display space is presented at 1216, the user can move back to the portion associated with the first display space, and process 1200 can move to 1218.

[0166] If the process 1200 determines that the HMD has not moved back to the first portion corresponding to the first space ("No" at 1218), the process 1200 may return to 1208, where it may determine whether the HMD has moved to the last portion associated with the current display space (i.e., the next display space when the HMD is in the portion corresponding to the first display space). Otherwise, if the process 1200 determines that the HMD has moved back to the first portion corresponding to the first space ("Yes" at 1218), the process 1200 may access a portion of content preceding the currently displayed content at 1220 so that the HMD can display the content when the HMD moves back to the previous display space (which may be the same physical space as the next display space). In some embodiments, the process 1200 may ensure that when the user moves to the next or previous display space (e.g., on the other side of the barrier 1104 in either direction), the HMD is always ready to display content corresponding to that display space.

[0167] In some implementations, process 1200 can be performed by an HMD or a server. Additionally or alternatively, some portions of process 1200 can be performed by an HMD, while other portions are performed by one or more other devices (e.g., a server, a sensor, etc.).

[0168] 13A to 13D An example of a scene from a replay of a recorded session of an HMD for presenting content to be used is shown in accordance with some implementations of the disclosed subject matter. In some implementations, the HMD 100 and / or server 204 can record media (e.g., audio, video, and / or computer-generated graphics) depicting the actions of one or more users, and the media can be accessed for review (e.g., by a user, an instructor, etc.). Figure 13A An example of a first viewing mode according to some implementations of the disclosed subject matter is shown, wherein a first user represented by an avatar 1302 (HLA) and a second user represented by an avatar 1304 (HLB) interact with a virtual accident victim 1306 (AI). Figure 13A In the illustrated mode, text 1308 representing a transcript of the conversation can be displayed as the scene progresses. Additionally, windows 1310 and 1312 can display first-person views from the HMDs worn by first user 1302 and second user 1304, respectively. In some embodiments, different users can be represented by different colored avatars (and / or personalized avatars), and windows 1310 and 1312 can be color-coded (and / or otherwise labeled) to clarify which user is viewing which part of the scene. In some embodiments, the content being viewed can represent a virtual reality session and can include graphics and / or images of the scene presented to the user via the HMD worn by the user during the session being viewed. Additionally or alternatively, the content being viewed can represent an augmented reality (or other mixed reality) session and may not include image data of the user's broader environment beyond the image data captured by the HMD (e.g., as shown in windows 1310 and 1312).

[0169] In some embodiments, any suitable computing device can be used to review the content recorded during the session. For example, a personal computer, laptop computer, tablet computer, etc. can be used to display the review content as a two-dimensional image. As another example, the content can be displayed by an HMD (e.g., as an augmented reality or virtual reality experience), and the wearer of the HMD can navigate around the recorded content by physically moving through the scene (e.g., viewing the content from different viewpoints), and the content displayed by the HMD can be based on the user's current position and field of view. As another example, the wearer of the HMD can issue one or more commands to control which part of the session is being displayed (i.e., navigate through time), or enter different review modes (e.g., display a bird's-eye view), display video content recorded by the HMD (and / or playback content associated with the HMD's viewpoint that was not recorded by the HMD, e.g., for a virtual reality review session) in a fully immersive review mode (i.e., displaying images corresponding to the HMD wearer's field of view in first-person mode).

[0170] In some embodiments, while viewing content, the viewer can select a specific avatar (e.g., avatar 1302 or 1304) and / or a specific window (e.g., window 1310 or 1312) to enter a first-person viewing mode, in which a first-person view associated with a specific user corresponding to the content is displayed as the primary view (e.g., not the primary view). Figure 13A In such a first-person view, windows corresponding to other users may or may not be displayed.

[0171] Figure 13B shows at different points in time according to some embodiments of the disclosed subject matter Figure 13A An example of another view of the scene shown in . In some implementations, a user reviewing a recording can navigate to any portion of the recording, such as by fast-forwarding or rewinding, dragging user interface element 1314 along scrubber 1316, searching for keywords (e.g., from a transcript), and the like. Figure 13C Shown at yet another time according to some embodiments of the disclosed subject matter Figure 13A and Figure 13B. In some embodiments, a user can navigate to a specific point (time) in the content using any suitable technique or combination of techniques, and the available techniques may depend on the hardware used to display the content. For example, if the content is displayed by a 2D display (e.g., by a personal computer or tablet computer), the user can pause, play, fast forward, rewind, skip, etc. by selecting user interface elements (e.g., using a mouse, touchpad, touch screen, game controller, etc.) using user interface elements displayed in conjunction with the content (not shown). As another example, if the content is being displayed by a 2D display or HMD, the user can use touch commands (e.g., on a paired device's touchpad, on a paired device's touch screen, or touch-sensitive input on the HMD) to navigate through the content (e.g., tap play / pause, swipe one direction to fast forward, swipe another direction to rewind, etc.). As yet another example, if the content is being displayed by a 2D display or HMD, the user can use voice commands to navigate through the content.

[0172] Figure 13D An example of a replay of an augmented reality session according to some implementations of the disclosed subject matter is shown. Figure 13D As shown, video of the wider surrounding physical environment may not be available for playback, but may show the user's avatar, hologram 1318, and a window representing the video recorded by the HMD, with or without audio and / or text representing conversations during the recording of the video content.

[0173] In some embodiments, information that can be used to present content during a review can be received by the computing device at any suitable time using any suitable technique or combination of techniques. In some embodiments, the content presented during a session (e.g., a 3D model, one or more virtual environments or portions of virtual environments, etc.) can be received from any suitable source at any suitable time. For example, the content can be downloaded, uploaded, sideloaded, etc. to the computing device before, during, and / or after the presentation of the content. As another example, at least a portion of the content can already be stored by the computing device (e.g., in the case where the computing device is used to prepare a presentation for presentation during the review of the session, in the case where the computing device is an HMD configured to present content presented during the presentation, etc.). In some embodiments, information regarding the dynamic portion of the session (e.g., the position of the HMD, audio, supplemental content, etc.) can be received at any suitable time. For example, in response to a request to present a particular session (e.g., from a server such as server 204), the computing device can receive recorded information regarding the position of the HMD relative to the presented content, environment, and / or other HMDs. As another example, the computing device may receive information about the position of the HMD relative to the displayed content, the environment, and / or other HMDs during a session to be reviewed (e.g., the computing device may record information during the session). As yet another example, the computing device may receive information about the position of the HMD relative to the displayed content, the environment, and / or other HMDs during a review session (e.g., the content may be streamed or otherwise transmitted when the session is to be reviewed). In some embodiments, portions of the dynamic content may be transmitted using different formats. For example, information about the position of the HMD, displayed content, supplemental content, etc. may be transmitted as an XML file, and audio content and / or video content may be received as media files.

[0174] Figure 14 An example 1400 of a process for reviewing media recorded by one or more HMDs is shown in accordance with some implementations of the disclosed subject matter. Figure 14 As shown, at 1402, process 1400 can receive a selection of content to review. In some embodiments, the content to review can correspond to a particular session in which the HMD is used to view an augmented reality presentation. For example, the content to review can correspond to a lecture given to a group of students all using HMDs to follow the lecture. In some embodiments, the content can be organized in any suitable manner (e.g., by date, title, user, etc.).

[0175] At 1404, process 1400 can download, stream, load, and / or otherwise access the selected content. In some implementations, the recorded content can be stored in any suitable location via any suitable computing device. For example, the recorded content can be stored via a user computing device (e.g., user computing device 220), a server on a local network, a remote server, a network-attached storage, etc. In some implementations, at 1404, the selected content can be retrieved from storage for presentation.

[0176] At 1406, process 1400 can present the recorded content from the selected session in the selected or default (e.g., if no selection has been made) mode. For example, process 1400 can present the recorded content from a bird's-eye view, from a ground-level view, from the view of an HMD that recorded the session, etc. In some implementations, process 1400 can present the text of the transcript.

[0177] At 1408, process 1400 can determine whether the viewing mode and / or one or more settings have been updated. For example, the user can select a new viewing mode, can turn text on or off, can turn audio on or off, can turn the view from a specific HMD on or off, etc. If process 1400 determines that the viewing mode and / or one or more settings have been updated ("yes" at 1408), process 1400 can move to 1410 and can present updated content based on the new viewing mode and / or settings. Otherwise, if process 1400 determines that the viewing mode and / or one or more settings have not been updated ("no" at 1408), process 1400 can move to 1412.

[0178] At 1412, process 1400 can determine whether input has been received for navigating within the recorded content. For example, the user can search for a specific time or keyword, can select a fast-forward user interface element, can move a progress indicator along a clear bar, etc. If process 1400 determines that input has been received for navigating within the recorded content ("yes" at 1412), process 1400 can move to 1414 and can change the presentation of the content based on the received navigation input. Otherwise, if process 1400 determines that input has not been received for navigating within the recorded content ("no" at 1412), process 1400 can return to 1406 and continue presenting the content in the current viewing mode with the current settings.

[0179] In some embodiments, any suitable computer-readable medium may be used to store instructions for performing the functions and / or processes described herein. For example, in some embodiments, the computer-readable medium may be temporary or non-temporary. For example, non-temporary computer-readable media may include media such as magnetic media (e.g., hard disks, floppy disks, etc.), optical media (e.g., compact disks, digital video disks, Blu-ray discs, etc.), semiconductor media (e.g., RAM, flash memory, electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.), any suitable medium that is not transient or has no permanent resemblance during transmission, and / or any suitable tangible medium. As another example, a transient computer-readable medium may include a signal on a network, a wire, a conductor, an optical fiber, a circuit, any other suitable medium that is transient and has no permanent resemblance during transmission, and / or any suitable intangible medium.

[0180] Those skilled in the art will appreciate that although the disclosed subject matter has been described above in conjunction with particular embodiments and examples, the invention is not necessarily limited thereto, and many other embodiments, examples, uses, modifications, and deviations from the embodiments, examples, and uses are intended to be covered by the appended claims. The entire disclosure of each patent and publication cited herein is incorporated herein by reference, as if each such patent or publication were individually incorporated herein by reference.

[0181] Various features and advantages of the invention are set forth in the following claims.

Claims

1. A system for displaying an interactive augmented reality presentation, comprising: An augmented reality device comprising: Display; and at least one processor, wherein the at least one processor is programmed to: determining which of a plurality of predetermined content display locations within a physical environment associated with the augmented reality device are within a field of view of the augmented reality device; selecting a content display position closest to the augmented reality device from among a plurality of predetermined content display positions from a plurality of content display positions within the field of view of the augmented reality device; causing content to be displayed by the augmented reality device to be anchored at the selected content display position, receiving information regarding a location of a second augmented reality device during a first time period relative to a location in a physical environment of the second augmented reality device at which the second augmented reality device is displaying the content, wherein the second augmented reality device is remote from the augmented reality device; and Based on the information about the positioning of the second augmented reality device, using the display of the augmented reality device, show a representation of a user of the second augmented reality device located proximate to a content display location, such that the representation of the user of the second augmented reality device is positioned with respect to the content displayed by the augmented reality device to reflect the positioning of the second augmented reality device relative to a location in the physical environment of the second augmented reality device where the content was being displayed by the second augmented reality device during the first time period.

2. The system according to claim 1, wherein: The at least one processor is further programmed to: receiving first content including a first three-dimensional model; receiving second content including a second three-dimensional model; presenting, using the display, a first view of the first three-dimensional model anchored at the selected content display location at a first time; as well as The display is used to present a first view of the second three-dimensional model at a second time after the first time.

3. The system according to claim 2, wherein: The at least one processor is further programmed to: One or more instructions are received from the server at a third time after the first time and before the second time, wherein the one or more instructions include at least one instruction causing the augmented reality device to stop displaying the first content at the selected content display location and start displaying the second content anchored at the selected content display location.

4. The system according to claim 2, wherein: Prior to presenting the first content, the first content and the second content are received as part of a content library.

5. The system according to claim 4, wherein: A content library is preloaded before the at least one processor determines which of a plurality of predetermined content display locations within a physical environment associated with the augmented reality device are within the field of view of the augmented reality device.

6. The system according to claim 2, wherein: The first content is received before the first time, and the second content is received after the first time.

7. The system of claim 2, further comprising a device orientation tracking system, wherein the at least one processor is further programmed to: receiving an indication to present the first content at a first predetermined content display position among the plurality of predetermined content display positions, wherein the first predetermined content display position corresponds to the selected content display position; receiving an instruction to present the second content at a second predetermined content display position among the plurality of predetermined content display positions; determining a second field of view based on information output by the device orientation tracking system; determining which of a plurality of predetermined content display locations are located in the second field of view; selecting a second content display position closest to the augmented reality device from a plurality of predetermined content display positions within a second field of view of the augmented reality device, wherein the second content display position corresponds to the second predetermined content display position; and In response to selecting the second predetermined content display position, presenting the second three-dimensional model anchored at the second predetermined content display position.

8. The system of claim 1 , wherein the at least one processor is further programmed to: receiving information about the position of a hand of a user of the second augmented reality device relative to a location where content is being presented by the second augmented reality device, A representation of the hand is presented based on the information about the position of the hand using a display of the augmented reality device.

9. The system of claim 1 , wherein the at least one processor is further programmed to: receiving audio recorded by a second augmented reality device; and Present audio through speakers during presentation of content.

10. The system of claim 1, wherein the augmented reality device is a head-mounted display. The system of claim 10 , wherein the display is a transparent display.

12. A system for displaying interactive content, comprising: A first head-mounted display, wherein the first head-mounted display is configured to: receiving content including first content and second content; determining which of a plurality of predetermined content display locations in an environment are within a field of view of the first head-mounted display; selecting, from a plurality of content display positions within a field of view of the first head-mounted display, a content display position that is closest to the first head-mounted display among a plurality of predetermined content display positions; presenting, in the environment, first content anchored at the selected content display location during a first time period; receiving information indicating that a second head-mounted display is presenting the first content at a first location in a physical environment of the second head-mounted display during a first time period; receiving information regarding a position of a second head mounted display relative to the first location in a physical environment of the second head mounted display during a first time period; presenting a representation of a user of the second head-mounted display as being located proximate to the selected content display location in the environment based on the information regarding the position of the second head-mounted display during the first time period, such that the representation of the user of the second head-mounted display is positioned with respect to the first content in the environment to reflect the position of the second head-mounted display relative to the first location in the physical environment of the second head-mounted display during the first time period; as well as Second content is presented at a second location in the environment based on a proximity of the first head mounted display to the second location in the environment.

13. The system of claim 12, wherein the first head mounted display comprises a transparent display.

14. The system of claim 12, wherein the first head mounted display comprises a virtual reality display.

15. The system of claim 12, wherein the first head mounted display is further configured to: receiving information regarding a position of a hand of a user of a second head mounted display relative to a location where content is being presented on the second head mounted display; and Based on the information about the hand position, a representation of the hand is presented using a first head mounted display.

Images