Systems and methods for virtual and augmented reality

Abstract

Examples of the present disclosure describe systems and methods related to mobile computing. According to the example method, a first user location of a user of a mobile computing system is determined. Based on the first user location, a first communication device proximate to the first user location is identified. A first signal is transmitted to the first communication device. In response to the first communication device receiving the first signal, a first information payload based on the first user location is received from the first communication device. At a first time when the user is at the first user location, video or audio data based on the first information payload is presented to the user.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 62 / 823,619, filed on March 25, 2019, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present invention relates to connected mobile computing systems, methods and configurations, and more particularly, to mobile computing systems, methods and configurations featuring at least one wearable component that can be used for virtual and / or augmented reality operations. Background Art

[0004] It is desirable for mixed reality or augmented reality near-eye displays to be lightweight, low cost, small form factor, have a wide virtual image field of view, and be as transparent as possible. Additionally, it is desirable to have a configuration that presents virtual image information in multiple focal planes (e.g., two or more) so as to be suitable for a variety of use cases without exceeding acceptable tolerances for vergence-accommodation mismatch. Compact and continuously connected systems and components optimized for use in wearable computing systems are needed. Summary of the invention

[0005] Examples of the present disclosure describe systems and methods related to mobile computing. According to the example method, a first user location of a user of a mobile computing system is determined. Based on the first user location, a first communication device proximate to the first user location is identified. A first signal is transmitted to the first communication device. In response to the first communication device receiving the first signal, a first information payload based on the first user location is received from the first communication device. At a first time when the user is at the first user location, video or audio data based on the first information payload is presented to the user. A second user location of the user is determined. Based on the second user location, a second communication device proximate to the second user location is identified. A second signal is transmitted to a second communication device. In response to the second communication device receiving the second signal, a second information payload based on the second user location is received from the second communication device. At a second time while the user is at the second user location, video or audio data based on the second information payload is presented to the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 An exemplary mobile computing system according to examples of the present disclosure is shown.

[0007] Figure 2 An exemplary use case scenario for a mobile computing system according to examples of the present disclosure is shown.

[0008] Figure 3 An exemplary use case scenario for a mobile computing system according to examples of the present disclosure is shown.

[0009] Figure 4 An exemplary use case scenario for a mobile computing system according to examples of the present disclosure is shown.

[0010] Figure 5 An exemplary use case scenario for a mobile computing system according to examples of the present disclosure is shown.

[0011] Figure 6 Exemplary connected resources according to examples of the present disclosure are shown.

[0012] Figure 7 Exemplary connected resources according to examples of the present disclosure are shown.

[0013] Figure 8 Exemplary connected resources according to examples of the present disclosure are shown.

[0014] Fig. 9 Exemplary connected resources according to examples of the present disclosure are shown.

[0015] Fig.10 An exemplary method of using a mobile computing system according to examples of the present disclosure is shown.

[0016] Fig.11 An exemplary mobile computing system according to examples of the present disclosure is shown.

[0017] Fig.12 An exemplary method of using a mobile computing system according to examples of the present disclosure is shown.

[0018] Fig.13 An exemplary method of using a mobile computing system according to examples of the present disclosure is shown. DETAILED DESCRIPTION

[0019] In the following description of the examples, reference is made to the accompanying drawings which form a part of the description and in which is shown by way of illustration specific examples that can be practiced. It should be understood that other examples can be used and structural changes can be made without departing from the scope of the disclosed examples.

[0020] refer to Figure 1, shows a mobile computing system such as a wearable augmented reality system, characterized by a head-mounted viewing component (2), a handheld controller component (4), and an interconnected auxiliary computing or controller component (6) that can be configured to be worn on the user as a waist pack or the like. Each of these components can be operably coupled to each other or coupled (10, 12, 14, 16, 17, 18) to other connected resources (8) via a wired or wireless communication configuration (such as specified by IEEE 802.11, Bluetooth (RTM), and other connection standards and configurations). For example, in U.S. patent application serial numbers 14 / 555,585, 14 / 690,401, 14 / 331,218, 15 / 481,255 and 62 / 518,539, such as described in the following. Figure 1 The illustrated system, each of which is incorporated herein by reference in its entirety. The aforementioned U.S. patent applications describe various aspects of such a system, such as the two depicted optical elements (20) and various embodiments of associated system components and visual components with which a user can see the world around them (i.e., the user's local environment) in an augmented reality experience.

[0021] refer to Figure 2 , depicting an exemplary driving scenario (160) in which a mobile computing system (such as reference Figure 1 A user of a wearable computing system (described herein) moves in the world. Figure 2 The example shows a user's home (22) featuring at least one communication device (40), which may be a wireless communication device and configured to connect to the user's mobile computing system. As the user navigates the world around him, on a day in the illustrative example, the user travels (30) from home (22; point A 80) to work (24; points B 82, C 84, D 86, E 88); he then travels (32; points I 96, J 98) from work (24) to a park (26) for a walk (28; points K 100, L 102, M 104), and then returns (34; points N 106, O 108) to home (22) - along the way, his mobile computing system communicates with various wireless devices (40, 42, 44, 46, 48, 50, 52, 54, and other devices such as in FIG. 1 ). Figure 3 and Figure 4 In some examples, the mobile computing system is configured to utilize various wireless devices and information exchanged therewith to provide a relatively low latency and robust connection experience to the user, and such experience is generally influenced by user preferences (e.g., reflecting a preferred trade-off between content and quality of service) that can be selected by the user.

[0022] In one embodiment, the mobile computing system may be configured to allow the user to select certain aspects of his computing experience for the day. For example, through a graphical user interface, voice control, and / or gestures, the user may input to the mobile computing system that he will have a typical workday, take a regular route to and from work, and stop for a short walk in the park on the way home. Preferably, the mobile computing system may include certain artificial intelligence or machine learning aspects; for example, the system may integrate the user's electronic calendar to temporarily understand his schedule, requiring quick confirmation. For example, when the user leaves for work, the system may be configured to say or display: "Going to work; regular route and regular computing configuration", and the regular route may be obtained from previous GPS and / or mobile triangulation data by the user's mobile computing system. The "regular computing configuration" may be customized by the user and subject to regulations; for example, the system may be configured to present only certain non-occluded visuals while the user is driving, no advertisements, no shopping or other non-driving related information, and provide an audio version of a news program or a currently favorite audiobook while the user is driving to work. As the user navigates his way to work, he may disconnect from his home wireless device (40) and enter or remain connected to other wireless devices (42, 44, 46, 48). Each of these wireless devices may be configured to provide information relevant to the user's experience to the user's mobile computing system with relatively low latency (i.e., by locally storing certain information that may be relevant to the user at that location). Figure 6 and Figure 7 Certain aspects of a wireless device that may be used as described herein are shown; Figure 8 and Fig. 9 Embodiments of the invention feature non-storage beacon and / or tag configurations that can be utilized to connect directly to locally relevant cloud-based information without the benefit of local storage.

[0023] For example, Figure 2As depicted, at various times as the user travels from point A (80) to point B (82) to point C (84), the user's mobile computing system may identify a nearby communication device proximate to the user's location and communicate with the communication device. The communication device may include a wireless communication device. For example, if the user's location is near point C (84), the mobile computing system may determine that the communication device (44) is proximate to the user's location. The communication device (44) may be configured to receive a signal from the user's mobile computing system; and respond to the signal by transmitting an information payload to the user's mobile computing system. For example, the information payload transmitted to the user's mobile computing system may include spatial information (e.g., geographic or geometric information, such as a grid describing the geometry of a room) that may be used on the user's mobile computing system. The spatial information may be used by the mobile computing system, for example, to highlight where a ditch is being dug at the user's location so that the user can clearly visualize and / or understand the hazard when driving by, and the spatial information (which may, for example, feature a highlighted outline of the ditch or another graphical enhancement) may be stored locally on the local communication device (44) so ​​that it does not need to be pulled from more remote resources (which may involve greater delays in getting the information to the driver). In addition to reducing latency, local storage may also be used to reduce the overall computational load on the user's mobile computing system, since the mobile system may receive information that it would otherwise have to generate or construct itself based on sensors (which may, for example, include part of the local mobile hardware). The information payload provided by the communication device may also include non-spatial information, such as photos, video, audio, text, metadata, or other suitable information; and this information may be used by the user's mobile computing system as appropriate—for example, by presenting the information to the user visually (e.g., via a transmissive display) and / or audibly (e.g., via one or more speakers).

[0024] The information payload may be based on data sent by a user's mobile computing system to the communication device and / or data sent by another user's mobile computing system to the communication device. That is, for example, data (e.g., spatial coordinates, sensor outputs, telemetry data) provided by a first mobile computing system at a certain location may be reused to present an information payload to a second user's mobile computing system that subsequently arrives at the same location.

[0025] The user can communicate with other communication devices at other locations via his mobile computing system. Figure 2, if the user is at location E (88), the mobile computing system may determine that the communication device (48) is proximate to the user's location. The communication device (48) may be configured to receive a signal from the user's mobile computing system; and respond to the signal by transmitting an information payload to the user's mobile computing system, such as described above with respect to the communication device (44). In some examples, the communication devices may be in networked communication and share information with each other. For example, the communication device (48) may communicate with the communication device (44) to share information for use in generating an information payload to be sent to the user.

[0026] Once the user arrives at his workplace's parking lot (24), the system may be configured, for example, to detect walking speed as the user walks to the office via integration with his computerized calendar system and configured by the user to review his schedule for the day with the user. Certain additional information not resident on his local mobile computing system may be pulled from local resources (e.g., 48, 50) that may feature certain storage capacities, again facilitating less mobile overhead and lower latency than a direct cloud connection.

[0027] refer to Figure 3 Once in the office (24), the user may connect with various communication devices (50, 60, 62, 64, 66, 68, 70, 72, 74) (which may include wireless communication devices), each of which may be configured to provide location-based information. For example, when at point F (90), the user's mobile computing system may be configured to detect the location (such as through GPS, computer vision, tag or beacon recognition, and / or communication device (60, 62, 64) triangulation) and then quickly upload information related to the location (such as a dense triangular mesh of the room's geometry, or specific information related to whose office the room is, information about the person, or other information that may be considered relevant) from local storage (i.e., from communication devices 60, 62, 64) to his mobile computing system, such as by an artificial intelligence agent operating automatically on the user's mobile computing system. Various other communication devices (50, 66, 68, 70, 72, 74) may be located elsewhere in the office and configured to feature other location-based information to again provide low latency and robust mobile computing capabilities to local users without requiring sensor infrastructure local to the mobile computing system to do everything from scratch in real time (such as determining room geometry).

[0028] refer to Figure 4, similar communication device resources (40, 56, 58) may be utilized at home (22) to assist with location-based information as the user navigates (P 110, Q 112, R 114, S 116, T 118, U 120) his home with his mobile computing system. In the context of the office (24) or home (22), the mobile computing system may be configured to utilize external resources that are entirely different from driving. For example, the artificial intelligence component of the user's mobile computing system may know that the user likes to watch highlights of the previous week's nightly news while walking around between 7 and 8 a.m. on Saturday mornings (perhaps in a display format that is not normally acceptable while driving but is acceptable while walking, or perhaps automatically unfolding when the user stops walking around and moves while sitting or standing), so when walking speed is detected, the system may be configured to provide such highlights from local storage between these times, while also collecting other location-based information, such as the location of various objects or structures in relevant locations within the house (i.e., to reduce computer vision processing load).

[0029] Similarly, when a user navigates a walk (28) in a park (26), as Figure 5 As shown in the enlarged view in , local wireless device resources (54) can be used to provide location-based information, such as contextual information about a sculpture garden that the user may observe while walking; such information can be displayed or reproduced as audio as the user moves around in a manner that is customized and / or customizable for his park walking scenario (i.e., as opposed to driving, or walking around at home or work).

[0030] refer to Figure 6 In one embodiment, the aforementioned communication devices (40, 42, 44, 46, 48, 50, 52, 54, and Figure 3 and Figure 4 One or more of the other communication devices shown in the enlarged view of FIG. 1 may include Figure 6, wherein a local controller (134) (such as a processor) is operably coupled (138) to a power source (132) (such as a battery), a transceiver (130) (such as transmit and receive antennas configured to wirelessly communicate with a mobile computing system and other computing systems and resources, such as by using mobile telecommunications (i.e., GSM, EDGE, HSPA / +, 3G, 4G, 5G), WiFi (i.e., 802.11 standards such as 802.11a, 802.11b, 802.11g, 802.11n), WiMax, and / or Bluetooth (RTM; i.e., 1.x, 2.x, 3.x, 4.x) configurations), and a local storage device (136) (such as a mass storage device or memory device). The storage device (136) can be operably coupled (140) to an external storage resource (146), such as a cloud storage resource; the local power source (132) can be operably coupled (142) to an external energy source (148), such as for long-term charging or replenishment; the transceiver (130) can be operably coupled to an external connection resource (150) to provide, for example, access to an Internet backbone. All of these local and connected resources can be configured based on the location of such a device to provide local users with information that is customized for the local context, whether such information is related to transportation, shopping, weather, structure, culture, etc.

[0031] Figure 7 Shows something like Figure 6 Embodiments of, but without a local storage facility - its components are operably coupled (141) to remote storage resources (146) (such as cloud resources); Figure 7 This embodiment of the invention can be used in various configurations to replace, for example, Figure 6 Those embodiments in , but without the benefit of direct local storage (which, as described above, can be beneficial in reducing latency in providing information to mobile systems in the area).

[0032] refer to Figure 8In other scenarios where there is no local storage capability, a transmitter beacon (41) type device (e.g., featuring only a transmitter (131, not a bidirectional transceiver; such as a transmitting antenna configured to wirelessly communicate with a mobile computing system and other computing systems and resources, for example, using mobile telecommunications (i.e., GSM, EDGE, HSPA / +, 3G, 4G, 5G), WiFi (i.e., 802.11 standards such as 802.11a, 802.11b, 802.11g, 802.11n), WiMax, and / or Bluetooth (RTM; i.e., 1.x, 2.x, 3.x, 4.x)) and a relatively long term battery (132)) can be used to connect to a locally located mobile computing device to share location or beacon identification information that serves as a pointer to connect the mobile computing system to relevant cloud resources (i.e., bypassing local storage but providing information similar to a "you are here" pointer to relevant cloud resources).

[0033] refer to Fig. 9 In a very basic scenario, non-electronic tags (43) (such as aruco tags) can also be used as pointers to connect the mobile computing system to relevant cloud resources (i.e., bypassing local storage but providing information similar to a "you are here" pointer to the relevant cloud resources).

[0034] As described above, in order to reduce latency and generally increase useful access to relevant location-based information, communication devices (such as Figure 6 ) can be located throughout the interior of a structure (such as a home, business, etc.) and the exterior (such as a city district, outside a store or shop, etc.). Similarly, communication devices that do not have localized storage capacity but are operably coupled to or pointed to remote storage resources can also be located throughout the interior of a structure (such as a home, business, etc.) and the exterior (such as a city district, outside a store or shop, etc.).

[0035] In one embodiment, the mobile computing system may be customizable by the user to present information that is filtered based on time, such as by how old or "stale" such information is. For example, the user may be able to configure the system to only provide traffic information that is 10 minutes old or newer, etc. while he is driving (i.e., the time aspect may be customizable / configurable); or the user may be able to configure the system to only present building information (i.e., the location of walls within a building), etc. that is 1 year old or newer (i.e., the time aspect may be customizable / configurable).

[0036] refer to Figures 10 to 13In various embodiments, it is desirable that the system be configured such that the user's position and / or orientation (i.e., via determining the position and / or orientation of a coupled component (such as a head mounted viewing component 2 that may be coupled to the user's head)) may be used to provide the user with additional and / or enhanced content and / or information relevant to the user's particular view of the world as the user navigates the world.

[0037] For example, refer to Fig.10 In one embodiment, the user may wear a Figure 1 to navigate the world using the augmented reality system (200) depicted in FIG. 1 . A user may enter an area (such as a walkable area, or a functional volume inside or outside a building) in which certain features (such as intentionally visually detectable features and wireless connection-related features) have been preconfigured to be located and interpreted by the user's augmented reality system so that the system is configured to determine the user's position and / or orientation relative to the world immediately surrounding the user. Such relatively information-rich areas may be referred to as "positioning islands." For example, in one embodiment, certain connected resources (8) may include wireless connection devices (such as 802.11 devices) that may broadcast information such as an SSID and / or IP address, and relative signal strength for the wireless connection device may be determined and may be correlated to proximity; and further, other detectable features, such as Bluetooth, audio, and / or infrared beacons with known locations, and / or posters or other visual features with known locations. Combined detection and analysis of these inputs, such as from a plurality of sensors (which may include a plurality of components, such as a monochrome camera, a color camera, a Bluetooth detector, a microphone, a depth camera, a stereo camera, etc.) operably coupled to a head mounted component (2) of the subject system, may be used to determine the location and / or orientation of the user (202) based on analysis of information related to predetermined or known locations of such items, which may, for example, be included on a connected resource (8), such as, for example, a reference Figure 6 Those cloud storage systems described.

[0038] Reference again Fig.10Once the user's initial position and / or orientation has been determined, the sensors of the user's augmented reality system, along with certain features of the location island, may be used to maintain an updated determination of the user's position and / or orientation within the area or volume (204). Assuming that the user's position and / or orientation relative to the coordinate system of the place is updated as the user views and / or navigates around the place, certain specialized content and information may be presented to the user through the user's augmented reality system, including but not limited to content and information related to other remote places via a "passable world" configuration (such as that described in U.S. Patent Application Serial No. 13 / 663,466, which is incorporated herein by reference in its entirety), which "passable world" configuration may be configured, for example, to allow other users and objects to virtually "teleport" to a different location to see images related to the place and / or communicate with others who are there, either physically or virtually (206). The user interface of the user's augmented reality system may be configured to allow the user to ingest, utilize, view, and / or bypass certain information presented through the user's augmented reality system. For example, if a user is walking through a shopping area that is rich in particularly identifiable features (i.e., such as a "location island") and content, but does not want to see a virtual presentation of any information related to shopping at that time, the user can configure his or her system to not display such information, but only display information that has been selected for display, such as urgent personal messaging information.

[0039] refer to Fig.11 , showing the Figure 1 , but also shows a number of wirelessly connected resources that can be used to assist in positioning the user (i.e., determining the position and / or orientation of a component that can be coupled to the user's head in operation (such as the head-coupleable component 2)). For example, referring to Fig.11In addition to the major system components (2, 4, 6) being operably coupled to each other and to a connected resource (8) (such as cloud storage or cloud computing resources), these system components may also be wirelessly coupled to devices that can assist in locating a user, such as a Bluetooth device (222), such as a transmitter beacon with a known identity and / or location; an 802.11 device (218), such as a WiFi router with a specific SSID, IP address identifier, and / or signal strength or proximity sensing and / or transmission capabilities; a vehicle or component thereof (220), which may be configured to transmit information related to speed, location, and / or orientation (e.g., certain speedometer systems in certain motor vehicles may be configured to transmit instantaneous speed and approximate GPS location by coupling with a component having GPS tracking capabilities (such as an onboard GPS tracking device)); such speed, location, and / or orientation information related to the vehicle in which the user is located may be used, for example, to reduce display "jitter" and also to assist in presenting to the user real-world features that are visible through the vehicle window. related displayed images, such as a label of a hilltop being passed, or other features on the exterior of the vehicle; in certain embodiments involving a vehicle or other structure with a viewing portal on the exterior of such a vehicle or structure, information related to the geometry of such a vehicle, structure, and / or portal, such as from a cloud repository of connected resources 8, can be used to place virtual content appropriate for each user relative to the vehicle or structure); a mobile connected network transceiver (210), such as a transceiver configured for LTE connectivity that can not only connect to the user's system, but also provide triangulation and / or directional integration and integrated GPS information; a GPS transmitter and / or transceiver configured to provide location information to a connected device (212); an audio transmitter or transceiver beacon, such as those configured to assist in locating or directing nearby systems by using frequencies that are not normally audible (214; for example, in various embodiments, an audio transmitter or transceiver can be used to assist in minimally invasive "honing" of a mobile system (such as an augmented reality system) in upon) or locate (i.e., similar to how a first person in the dark can whistle to a second person in the dark to help the second person find the first person) not only an audio transmitter or transceiver, but also another adjacent or co-located location asset (such as a light, infrared, RF, or other beacon, transmitter and / or transceiver) (i.e., automatically through a sensor suite available on the augmented reality system (such as Figure 1 and Fig.11), or in other embodiments manually or semi-automatically so that the audio transmitter and / or transceiver are directionally represented in the user interface to the user, for example via a visual indicator (such as an arrow) in the user interface and / or an audio indicator through an integrated speaker in the head-mounted assembly); and / or an infrared beacon, which can be detected by the user's augmented reality system to similarly attract and / or identify information related to position and / or orientation.

[0040] refer to Fig.12 , showing the difference with Fig.11 An embodiment of operations related to the system depicted in . A user wearing an augmented reality system navigates a world (200). Within range of various wirelessly connected resources (such as mobile telecommunications transceivers (such as LTE), GPS devices, 802.11 devices, and various types of beacons (such as Bluetooth RF, audio, and / or infrared beacons)), the user's augmented reality system can be configured to determine the user's position and / or orientation relative to the world immediately surrounding the user (224). Once the user's initial position and / or orientation has been determined, sensors of the user's augmented reality system, along with particular wirelessly connected resources, can be used to maintain an updated determination of the user's position and / or orientation within an area or volume (226). Assuming that the position and / or orientation of the user's coordinate system relative to the place is updated as the user views and / or navigates around the place, certain specialized content and information may be presented to the user through the user's augmented reality system, including, but not limited to, content and information related to other remote places via a "passable world" configuration (such as that described in U.S. patent application Ser. No. 13 / 663,466, which is incorporated herein by reference in its entirety), which "connected world" configuration may be configured to, for example, allow other users and objects to virtually "teleport" to a different location to see images related to the place and / or communicate with others who are there, either physically or virtually (206). The user interface of the user's augmented reality system may be configured to allow the user to ingest, utilize, view, and / or bypass certain information presented through the user's augmented reality system, such as the content and information described above with respect to Fig.10 as described.

[0041] refer to Fig.13 In another embodiment, such as via computer vision and / or image or feature processing techniques, such as Figure 1 and Fig.11 The connected systems and resources described in the foregoing can be used to analyze other detectable resources (such as buildings, skylines, horizons, and / or different geometric structures of panoramas) to determine the user's location and / or orientation. Fig.13, a user wearing an augmented reality system navigates a world (200). In the vicinity of various structures or other detectable resources (such as one or more buildings, a skyline, a horizon line, and / or different geometric structures of a panorama), the user's augmented reality system can be configured to determine the user's position and / or orientation relative to the world immediately surrounding the user by processing, thresholding, and / or comparing aspects of such images to known images related to such scenes or resources (228). Once the user's initial position and / or orientation has been determined, sensors of the user's augmented reality system (such as color, monochrome, and / or infrared cameras) can be utilized to maintain an updated determination of the user's position and / or orientation within an area or volume (230). Assuming that the position and / or orientation of the user's coordinate system relative to the place is updated as the user views and / or navigates around the place, certain specialized content and information may be presented to the user through the user's augmented reality system, including, but not limited to, content and information related to other remote places via a "passable world" configuration (such as that described in U.S. patent application Ser. No. 13 / 663,466, which is incorporated herein by reference in its entirety), which "connected world" configuration may be configured to, for example, allow other users and objects to virtually "teleport" to a different location to see images related to the place and / or communicate with others who are there, either physically or virtually (206). The user interface of the user's augmented reality system may be configured to allow the user to ingest, utilize, view, and / or bypass certain information presented through the user's augmented reality system, such as the content and information described above with respect to Fig.10 as described.

[0042] Various exemplary embodiments of the present invention are described herein. Reference is made to these examples in a non-restrictive sense. They are provided to illustrate aspects of the present invention that are more widely applicable. Various changes may be made to the described present invention and equivalents may be substituted without departing from the true spirit and scope of the present invention. In addition, many modifications may be made to adapt specific situations, materials, material compositions, processes, (one or more) process actions or (one or more) steps to (one or more) purposes, spirit or scope of the present invention. In addition, as will be appreciated by those skilled in the art, each of the various variants described and illustrated herein has discrete components and features that can be easily separated or combined with the features of any one of the other several embodiments without departing from the scope or spirit of the present invention. All of these modifications are intended to fall within the scope of the claims associated with the present disclosure.

[0043] The present invention includes methods that can be performed using subject equipment. The method can include the action of providing such suitable equipment. This provision can be performed by the end user. In other words, the "providing" action only requires the end user to obtain, access, approach, locate, establish, activate, power on or otherwise act to provide the necessary equipment in the subject method. The methods described herein can be performed in any order of the listed events that is logically possible and in the listed order of events.

[0044] The exemplary aspects of the present invention and the details about material selection and manufacturing have been described above. As for other details of the present invention, these can be understood in conjunction with the patents and publications cited above and what is generally known or understood by those skilled in the art. This is also applicable to the method-based aspects of the present invention in terms of additional actions that are usually or logically employed.

[0045] In addition, although the invention has been described with reference to several examples optionally incorporating various features, the invention is not limited to what is described or indicated as contemplated with respect to each variant of the invention. Various changes may be made to the described invention and equivalents (whether described herein or not included for reasons of brevity) may be substituted without departing from the true spirit and scope of the invention. In addition, where a range of values ​​is provided, it is understood that each intermediate value between the upper and lower limits of the range and any other specified or intermediate value in the specified range are included in the invention.

[0046] In addition, it is contemplated that any optional features of the described variants of the invention may be independently set forth and claimed, or may be combined with any one or more features described herein. Reference to singular items includes the possibility of multiple identical items. More specifically, as used herein and in the claims associated therewith, the singular forms "one", "an", "said", and "the" include plural forms unless otherwise specifically stated. In other words, the use of articles allows for "at least one" subject item in the above description and in the claims associated with the present disclosure. It should also be noted that such claims may be drafted to exclude any optional elements. Therefore, this statement is intended to serve as a prior basis for the use of exclusive terms such as "only", "only" and the like in association with the narration of the claim elements, or for the use of "negative" restrictions.

[0047] In the absence of such exclusive terms, the term "comprising" in the claims associated with the present disclosure shall allow for the inclusion of any additional elements, regardless of whether a given number of elements are recited in such claims or the addition of features may be considered to transform the nature of the elements recited in such claims. Unless otherwise expressly defined herein, all technical and scientific terms used herein shall be given as much commonly understood meaning as possible while maintaining claim validity.

[0048] The scope of the present invention is not limited to the examples and / or subject description provided, but rather is limited by the scope of the claim language associated with this disclosure.

Claims

1. A method for virtual and augmented reality, comprising: determining a first user location of a user of the mobile computing system; Based on the first user location, identifying a first communication device proximate to the first user location; transmitting a first signal to the first communication device; receiving, in response to the first communication device receiving the first signal, a first information payload based on the first user location from the first communication device; presenting video or audio data based on the first information payload to the user at a first time when the user is at the first user location; determining a second user location of the user; Based on the second user location, identifying a second communication device proximate to the second user location; transmitting a second signal to the second communication device; receiving, in response to the second communication device receiving the second signal, a second information payload based on the second user location from the second communication device; as well as presenting video or audio data based on the second information payload to the user at a second time when the user is at the second user location, Wherein, the first information payload is also based on a signal received from a second mobile computing system.

2. The method according to claim 1, wherein: The mobile computing system includes a wearable head device, and presenting the video or audio data to the user includes presenting the video or audio data to the user via the wearable head device.

3. The method according to claim 1, wherein: The first communication device includes a first wireless communication device, and the second communication device includes a second wireless communication device.

4. The method according to claim 1, wherein: The second communication device communicates with the first communication device.

5. The method according to claim 1, wherein: The first communication device comprises a first local storage device, and In response to receiving the first signal, data based on the first signal is stored in the first local storage device.

6. The method according to claim 1, wherein: The first communication device comprises a first local storage device, and The first information payload is generated based on data stored in the first local storage device.

7. The method according to claim 1, wherein: In response to receiving the first signal, the first communication device transmits data based on the first signal to a remote server.

8. The method according to claim 1, wherein: The first information payload is generated based on data received from a remote server.

9. The method according to claim 1, wherein: The mobile computing system includes one or more sensors, and the first signal is based on an output of the one or more sensors.

10. The method according to claim 1, wherein: The mobile computing system includes one or more sensors, and determining the first user location includes determining the first user location based on outputs of the one or more sensors.

11. The method according to claim 1, wherein: Determining the first user location includes determining the first user location based on a schedule determined by one or more machine learning techniques.

12. The method according to claim 1, wherein: Determining the first user location includes determining the first user location based on a schedule input by the user.

13. A mobile computing system comprising: one or more sensors; as well as One or more processors configured to execute a method comprising: determining a first user location of a user of the mobile computing system based on outputs of the one or more sensors; Based on the first user location, identifying a first communication device proximate to the first user location; transmitting a first signal to the first communication device; receiving, in response to the first communication device receiving the first signal, a first information payload based on the first user location from the first communication device; presenting video or audio data based on the first information payload to the user at a first time when the user is at the first user location; determining a second user location of the user; Based on the second user location, identifying a second communication device proximate to the second user location; transmitting a second signal to the second communication device; receiving, in response to the second communication device receiving the second signal, a second information payload based on the second user location from the second communication device; and presenting video or audio data based on the second information payload to the user at a second time when the user is at the second user location, Wherein, the first information payload is also based on a signal received from a second mobile computing system.

14. The mobile computing system of claim 13, wherein: The mobile computing system includes a wearable head device, and presenting the video or audio data to the user includes presenting the video or audio data to the user via the wearable head device.

15. The mobile computing system of claim 13, wherein: The first communication device comprises a first local storage device, and In response to receiving the first signal, data based on the first signal is stored in the first local storage device.

16. The mobile computing system of claim 13, wherein: The first communication device comprises a first local storage device, and The first information payload is generated based on data stored in the first local storage device.

17. The mobile computing system of claim 13, wherein: In response to receiving the first signal, the first communication device transmits data based on the first signal to a remote server.

18. A non-transitory computer readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform a method comprising: determining a first user location of a user of the mobile computing system; Based on the first user location, identifying a first communication device proximate to the first user location; transmitting a first signal to the first communication device; receiving, in response to the first communication device receiving the first signal, a first information payload based on the first user location from the first communication device; presenting video or audio data based on the first information payload to the user at a first time when the user is at the first user location; determining a second user location of the user; Based on the second user location, identifying a second communication device proximate to the second user location; transmitting a second signal to the second communication device; receiving, in response to the second communication device receiving the second signal, a second information payload based on the second user location from the second communication device; as well as presenting video or audio data based on the second information payload to the user at a second time when the user is at the second user location, Wherein, the first information payload is also based on a signal received from a second mobile computing system.

Images