Compact head-mounted augmented reality system

Abstract

A head mounted device (HMD) may include a front portion, a rear portion, and one or more straps. The front side may include an optical display configured to output image light to an eye of a user. The rear portion may be disposed with the front portion to balance the weight of the HMD. One or more straps connect the front and rear portions such that the two portions rest on either side of the user's head.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application No. 63 / 356,472, filed on June 28, 2022, which is incorporated herein by reference. Technical Field

[0003] The present disclosure relates generally to augmented reality (AR), and in particular to compact AR devices. Background Art

[0004] Over the past ten to fifteen years, a variety of AR devices targeting consumer use cases have been developed. Widespread adoption of compact AR devices has been hampered by the challenge of providing a balance between practicality and a socially acceptable form factor. Technological limitations have been a barrier for achieving the desired level of functionality combined with a form factor acceptable to a wide range of users. When trying to fit the components of a useful system into the form factor of glasses or headphones, wearable devices become heavy and large, especially on the user's face, which can be inconvenient and uncomfortable. Conversely, smaller form factor devices that are more convenient and potentially more socially acceptable have fewer features and the application value becomes too low. Summary of the invention

[0005] The above and other challenges can be solved by a compact head-mounted augmented reality (AR) device that has integrated electronic components for providing AR content to the wearer. The various features, aspects, and advantages of the compact AR device can be better understood with reference to the following embodiments. The accompanying drawings illustrate specific embodiments and, together with the embodiments, help explain the various principles. However, the drawings should not be construed as limiting. The scope of protection should be determined by the claims. The following embodiments focus on a specific use case: glasses with AR functionality for outdoor gaming, but the disclosed principles can also be applied to other use cases and form factors.

[0006] In one aspect, the present disclosure provides a head mounted device (HMD). The HMD may include a front portion, a rear portion, and one or more straps. The front portion may include an optical display that outputs image light to the user's eyes. The rear portion is arranged together with the front portion to balance the weight of the HMD. One or more straps connect the front portion and the rear portion so that the two portions rest on both sides (e.g., front and rear) of the user's head.

[0007] On the other hand, the present disclosure provides a gaming device. The gaming device may include a head mounted device (HMD) and a game controller. The head mounted device may include a front portion, a rear portion, and one or more straps. The front portion may include an optical display that outputs image light to the user's eyes. The rear portion is arranged together with the front portion to balance the weight of the HMD device. One or more straps connect the front portion and the rear portion so that the two portions rest on both sides of the user's head. The game controller may receive input from the user and transmit the input to the HMD. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 is a block diagram of a network computing environment suitable for providing parallel reality games with augmented reality elements according to one embodiment.

[0009] Figure 2 The coordinate correspondence between the virtual world and the real world in a parallel reality game is illustrated according to one embodiment.

[0010] Figure 3 is a diagram of a system environment including a compact augmented reality (AR) device according to one embodiment.

[0011] Figure 4A A side view of a compact AR device is illustrated according to one embodiment.

[0012] Figure 4B A front view of a compact AR device is illustrated according to one embodiment.

[0013] Figure 5A An example AR device with a flexible strap is illustrated according to one embodiment.

[0014] Figure 5B An example AR device with a non-flexible strap is illustrated according to one embodiment.

[0015] Figure 6 An example AR device with an interchangeable forehead pad is illustrated, according to one embodiment.

[0016] Figure 7 An example AR device for storing a folded device is illustrated according to one embodiment.

[0017] Figure 8 An example AR device having a visor form factor is illustrated according to one embodiment.

[0018] Fig. 9 Illustrated is a block diagram of components of an example machine capable of reading instructions from a machine-readable medium and executing the instructions in a processor according to one embodiment. DETAILED DESCRIPTION

[0019] Reference will now be made in detail to various embodiments, one or more examples of which are illustrated in the accompanying drawings. Each example is provided as an explanation of the described embodiments and is not a limitation of the claims. In fact, it will be apparent to those skilled in the art that various modifications and variations may be made without departing from the principles described. For example, features illustrated or described as part of one embodiment may be used with another embodiment to produce further embodiments. Therefore, the present disclosure is intended to encompass such modifications and variations within the scope of the appended claims and their equivalents.

[0020] Described herein is an embodiment of a compact AR device having its components separated into two parts: a front portion and a back portion. During use, the front portion of the AR device is positioned in front of the user's face, while the back portion is positioned behind the user's head. The front and back portions may be connected by a flexible band so that the front and back portions may be folded together for compact storage when the device is not in use. The flexible band connecting the two portions may additionally be elastic to allow the AR device to accommodate various head sizes and shapes with small adjustments. In other embodiments, the compact AR device may have additional portions.

[0021] Exemplary Location-Based Parallel Reality Gaming System

[0022] In a parallel reality game, the geography of the virtual world is parallel to at least part of the geography of the real world, so that the player's movements and behaviors in the real world affect the behavior in the virtual world and vice versa. The virtual world may include augmented reality (AR) content, and players can view content superimposed on real-world images by going to corresponding real-world locations. A person of ordinary skill in the art, using the disclosure provided herein, will understand that the described subject matter is applicable to other situations in which AR images are displayed. In addition, a person of ordinary skill in the art, using the disclosure provided herein, will realize that, in view of the present disclosure, numerous game interface configurations and underlying functions will be apparent. The present disclosure is not intended to be limited to any one particular configuration.

[0023] Figure 1The network computing environment 100 is illustrated according to one or more embodiments. The network computing environment 100 provides interaction of players in a virtual world with a geographic location parallel to the real world. In particular, a geographic area in the real world can be directly linked or mapped to a corresponding area in the virtual world. Players can move in the virtual world by moving to various geographic locations in the real world. For example, the position of the player in the real world can be tracked and used to update the position of the player in the virtual world. Generally, the position of the player in the real world is determined by finding the location of the client device 110 used by the player to interact with the virtual world, and assuming that the player is located at the same (or approximately the same) location. For example, in various embodiments, if the player's location in the real world is within a threshold distance (e.g., ten meters, twenty meters, etc.) of the real world location corresponding to the virtual location of the virtual element in the virtual world, the player can interact with the virtual element. For convenience, players who are close enough to the virtual element and can interact with it in this way are referred to as being located at the real world location corresponding to the virtual element. Additionally, various embodiments are described with reference to a “player's location,” but those skilled in the art will appreciate that these references may refer to the location of the player's client device 110 .

[0024] Reference now Figure 2 According to one embodiment, Figure 2 A conceptual diagram of a virtual world 210 parallel to the real world 200 is depicted, which can serve as a game board for players of a parallel reality game. As illustrated, the virtual world 210 can include geographic locations parallel to the geographic locations of the real world 200. In particular, a coordinate range defining a geographic area or space in the real world 200 is mapped to a corresponding coordinate range defining a virtual space in the virtual world 210. The coordinate range in the real world 200 can be associated with a town, block, city, campus, region, country, continent, the entire earth, or other geographic area. Each geographic coordinate in the geographic coordinate range is mapped to a corresponding coordinate in the virtual space in the virtual world.

[0025] The position of the player in the virtual world 210 corresponds to the position in the real world 200. For example, player A, who is located at position 212 in the real world 200, has a corresponding position 222 in the virtual world 210. Similarly, player B, who is located at position 214 in the real world, has a corresponding position 224 in the virtual world. When the player moves in the geographic coordinate range of the real world, the player also moves in the coordinate range that defines the virtual space in the virtual world 210. In particular, a positioning system (e.g., a GPS system) associated with a mobile computing device carried by the player can be used to track the position of the player as the player navigates in the geographic coordinate range of the real world. Data associated with the position of the player in the real world 200 is used to update the position of the player in the corresponding coordinate range that defines the virtual space in the virtual world 210. In this way, the player can navigate along a continuous track within the coordinate range that defines the virtual space in the virtual world 210 by simply traveling within the corresponding geographic coordinate range in the real world 200, without having to check in at specific discrete locations in the real world 200 or regularly update location information.

[0026] Location-based games may include multiple game objectives, requiring players to travel to or interact with various virtual elements or virtual objects dispersed in various virtual locations in the virtual world. Some or all of the virtual objects may be AR objects displayed by the client device 110. AR objects may include animate objects and inanimate objects. Animated objects may be referred to as virtual characters. Virtual characters may represent characters in the game, such as non-player characters (NPCs). Players may travel to these virtual locations by traveling to corresponding locations of virtual elements or objects in the real world. For example, a positioning system may continuously track the position of the player so that when the player continuously navigates the real world, the player also continuously navigates a parallel virtual world. The player may then interact with various virtual elements or objects at a specific location to achieve or execute one or more game objectives.

[0027] For example, the goal of the game is to allow players to interact with virtual elements 230 located at various virtual locations in the virtual world 210. These virtual elements 230 can be linked to landmarks, geographical locations, or objects 240 in the real world 200. The real-world landmarks or objects 240 can be works of art, monuments, buildings, businesses, libraries, museums, or other suitable real-world landmarks or objects. Interactions include capturing, claiming ownership, using certain virtual items, spending some virtual currency, etc. In order to capture these virtual elements 230, the player goes to the landmarks or geographical locations 240 that are linked to the virtual elements 230 in the real world and interacts with the virtual elements 230 in the virtual world 210. For example, Figure 2Player A in the game may have to go to a landmark 240 in the real world 200 in order to interact with or capture a virtual element 230 linked to that particular landmark 240. Interaction with the virtual element 230 may require taking action in the real world, such as viewing an AR representation of the virtual element superimposed on a view of the real world location in an image collection (e.g., video) captured by a camera, and taking a "photo" of the virtual element in the real world. The image collection and photos may be captured by a camera mounted on a compact AR device (e.g., AR goggles or visor).

[0028] The game objectives may require the player to use one or more virtual items collected by the player in the location-based game. For example, the player may travel to the virtual world 210 to find virtual items (e.g., weapons, creatures, power-ups, or other items) that can be used to complete the game objectives. These virtual items may be found or collected by traveling to different locations in the real world 200 or by completing various actions in the virtual world 210 or the real world 200. Figure 2 In the example shown, a player uses virtual item 232 to capture one or more virtual elements 230. In particular, a player may deploy virtual item 232 at a location in virtual world 210 near virtual element 230 or at a location within virtual element 230. Deploying one or more virtual items 232 in this manner may result in a particular player or a particular player's team / faction capturing virtual element 230.

[0029] In one particular embodiment, players may have to collect virtual energy or other items as part of a parallel reality game. Figure 2 As shown, virtual energy 250 can be scattered at different locations in the virtual world 210. Players can collect virtual energy 250 and other items by going to corresponding locations in the real world 200. Virtual energy 250 can be used to power virtual items or perform various game goals in the game. Other items can perform any other suitable functions, depending on the game.

[0030] According to aspects of the present disclosure, parallel reality games can be large-scale multiplayer location-based games in which each participant in the game shares the same virtual world. Players can be divided into separate teams or factions and can work together to achieve one or more game goals, such as capturing or claiming virtual elements. In this way, parallel reality games can essentially be social games that encourage cooperation between players in the game. During parallel reality games, players from opposing teams can compete against each other (or sometimes cooperate to achieve common goals). Players can use virtual items to attack or hinder the progress of players of opposing teams. In some cases, players are encouraged to gather at locations in the real world to conduct cooperative or interactive activities in parallel reality games. In these cases, the game server seeks to ensure that the players are indeed present and there is no deception.

[0031] Parallel reality games can have various functions to enhance and encourage gameplay in parallel reality games. For example, players can accumulate virtual currency or other virtual rewards (e.g., virtual tokens, virtual points, virtual material resources, etc.), which can be used throughout the game (e.g., to purchase in-game items, exchange other items, make items, etc.). When players complete one or more game goals and gain experience in the game, players can advance to various levels. In some embodiments, players can communicate with each other through one or more communication interfaces provided in the game. Players can also obtain enhanced "abilities" or virtual items, which can be used to complete game goals in the game. It should be understood by those of ordinary skill in the art using the disclosure provided herein that various other game functions can be included in parallel reality games without departing from the scope of the present disclosure.

[0032] Reference back Figure 1 , the networked computing environment 100 uses a client-server architecture, where a game server 120 communicates with a client device 110 via a network 105 to provide a parallel reality game to a player on the client device 110. The networked computing environment 100 may also include other external systems, such as sponsor / advertiser systems or business systems. Figure 1 Only one client device 110 is shown, but any number of client devices 110 or other external systems may be connected to the game server 120 via the network 105. Furthermore, the networked computing environment 100 may include different or additional elements, and functionality may be distributed between the client devices 110 and the server 120 in a manner different from that described below.

[0033] The client device 110 may be any portable computing device that can be used by a player to interact with the game server 120. For example, the client device 110 may be a wireless device, a personal digital assistant (PDA), a portable gaming device, a cellular phone, a smart phone, a tablet computer, a navigation system, a handheld GPS system, a wearable computing device, a display with one or more processors, or other such device.

[0034] In one embodiment, the functions described below performed by the client device 110 are distributed between a mobile computing device (e.g., a tablet or smartphone) and a wearable device (e.g., AR goggles). For example, one or more cameras on the AR goggles worn by the player can capture images of the player's surroundings, the player's smartphone can determine a real-world location to display one or more AR elements, and one or more displays mounted on the AR goggles can display the AR elements at the determined location. In another embodiment, the AR goggles provide all of the functions described below performed by the client device 110.

[0035] In one embodiment, the client device 110 communicates with the game server 120 and provides sensory data of the physical environment to the game server 120. The client device 110 includes a camera component 125 that can capture two-dimensional image data of the scene of the client device 110 in the physical environment. Figure 1 In the illustrated embodiment, the client device 110 includes additional software components such as a game module 135, a positioning module 140, and an optical component 143. The client device 110 may include various other input / output devices for receiving information from a player or providing information to a player. Example input / output devices include a display screen, a touch screen, a touch pad, data entry keys, a speaker, and a microphone suitable for voice recognition. The client device 110 may also include various other sensors for recording data from the client device 110, including but not limited to motion sensors, accelerometers, gyroscopes, other inertial measurement units (IMUs), barometers, positioning systems, thermometers, optical sensors, etc. The client device 110 may also include a network interface for providing communication through the network 105. The network interface may include any suitable components for interacting with one or more networks, including, for example, a transmitter, a receiver, a port, a controller, an antenna, or other suitable components.

[0036] The camera assembly 125 captures image data of the environment scene in which the client device 110 is positioned. The camera assembly 125 can utilize a variety of different light sensors that have different color capture ranges at different capture rates. The camera assembly 125 may include a wide-angle lens or a telephoto lens. The camera assembly 125 may be configured to capture a single image or video as image data. The camera assembly 125 captures image data and shares it with a computing device on the client device 110. The image data may be attached with metadata that describes other details of the image data, including sensing data (e.g., temperature, ambient brightness) or capture data (e.g., exposure, color temperature, shutter speed, focal length, capture time, etc.). The camera assembly 125 may include one or more cameras that can capture image data. In one example, the camera assembly 125 includes a pair of cameras mounted on the left and right sides of the AR goggles, which are configured to capture stereoscopic image data from a position approximating the wearer's eyes. In another example, the camera assembly 125 includes a single camera mounted on the front approximate center of the AR goggles. In various other implementations, the camera assembly 125 includes a plurality of cameras, each camera being configured to capture image data.

[0037] The game module 135 provides an interface for players to participate in parallel reality games. The game server 120 transmits game data to the client device 110 via the network 105 for use by the game module 135 on the client device 110, thereby providing a local version of the game to players away from the game server 120. The game server 120 may include a network interface for providing communication via the network 105. The network interface may include any suitable components for interacting with one or more networks, such as a transmitter, a receiver, a port, a controller, an antenna, or other suitable components.

[0038] The game module 135 executed by the client device 110 provides an interface between the player and the parallel reality game. The game module 135 can present a user interface on a display device, such as an optical display mounted on an AR goggles and positioned in front of the wearer's eyes. The user interface may include virtual elements associated with the game, and enables the player to interact with the virtual elements to perform various game objectives. The game module 135 can generate virtual content or adjust virtual content based on information received from other components of the client device. For example, the game module 135 can adjust the virtual objects displayed on the user interface based on the scene depth information captured in the image data.

[0039] The game module 135 can also control various other outputs to allow the player to interact with the game. For example, the game module 135 can control various audio, vibration or other notifications. The game module 135 can access the received game data from the game server 120 to provide the player with an accurately represented game. The game module 135 can receive and process player input and provide updates to the game server 120 through the network 105. The game module 135 can also generate or adjust the game content displayed by the client device 110. For example, the game module 135 can generate virtual elements based on information describing the traversable space of the scene (e.g., determined by the traversable space estimation model 130). For example, the game module 135 can determine the path for the virtual element in the traversable space of the scene.

[0040] The positioning module 140 can be any device or circuit for monitoring the location of the client device 110. For example, the positioning module 140 can determine the actual or relative location by using a satellite navigation positioning system (e.g., GPS system, Galileo positioning system, Global Navigation Satellite System (GLONASS), Beidou satellite navigation positioning system), an inertial navigation system, a dead reckoning system, based on an IP address, by using triangulation or proximity to a cellular tower or Wi-Fi hotspot, or other appropriate techniques for determining positioning. The positioning module 140 can also include various other sensors that help to accurately determine the location of the client device 110. For example, the client device 110 can perform positioning by comparing images captured by one or more cameras in the client device with a 3D map.

[0041] When the player moves with the client device 110 in the real world, the positioning module 140 tracks the player's location and provides the player's location information to the game module 135. The game module 135 updates the player's location in the virtual world associated with the game according to the player's actual location in the real world. Therefore, the player only needs to carry or transport the client device 110 in the real world to interact with the virtual world. In particular, the player's location in the virtual world can correspond to the player's location in the real world. The game module 135 can provide the player's location information to the game server 120 through the network 105. In response, the game server 120 can formulate various technologies to verify the location of the client device 110 to prevent cheaters from forging the location of the client device 110. It should be understood that the location information related to the player will only be used if the player is notified that his location information will be accessed and how the location information will be used in the game environment (for example, to update the player's location in the virtual world) and with permission. In addition, any location information related to the player will be stored and maintained in a way that protects the player's privacy.

[0042] Light assembly 143 includes one or more lights that can illuminate the environment surrounding the client device. In one embodiment, light assembly 143 includes a flash associated with camera assembly 125. The flash can be illuminated briefly to help take a single photo or can be used to illuminate for a long time to act as a flashlight. In other embodiments, light assembly 143 can include different types of light, such as infrared light to illuminate objects and enable an infrared camera to capture scene images without using visible light to illuminate the scene.

[0043] The game server 120 can be any computing device and can include one or more processors and one or more computer-readable storage media. The computer-readable storage medium can store instructions that cause the processor to perform operations. The game server 120 can include or communicate with a game database 115. The game database 115 stores game data used in the parallel reality game, which is served or provided to (multiple) client devices 110 via the network 105.

[0044] The game data stored in the game database 115 may include: (1) data associated with the virtual world in the parallel reality game (for example, image data used to draw the virtual world on a display device, geographic coordinates of locations in the virtual world, etc.); (2) data associated with players of the parallel reality game (for example, player profiles, including but not limited to player information, player experience level, player currency, the player's current position in the virtual world / real world, player energy level, player preferences, team information, faction information, etc.); (3) data associated with game objectives (for example, data associated with the current game objective, the status of the game objective, past game objectives, future game objectives, desired game objectives, etc.); (4) data associated with virtual elements in the virtual world (for example, the virtual elements' The game database 115 may include data associated with the game world (e.g., location, type of virtual element, game objective associated with the virtual element, real-world location information corresponding to the virtual element, behavior of the virtual element, relevance of the virtual element, etc.); (5) data associated with real-world objects, landmarks, locations linked to virtual-world elements (e.g., location of real-world objects / landmarks, description of real-world objects / landmarks, relevance of virtual elements linked to real-world objects, etc.); (6) game state (e.g., current number of players, current state of game objectives, player leaderboards, etc.); (7) data associated with player actions / inputs (e.g., current player location, past player location, player movement, player input, player queries, player communications, etc.); and (8) any other data used, related to, or obtained during the implementation of the parallel reality game. The game data stored in the game database 115 may be populated by a system administrator offline or in real time, or may be populated by data received from users / players of the system 100, such as from client devices 110 via network 105.

[0045] The game server 120 may be configured to receive requests for game data from the client devices 110 (e.g., via remote procedure calls (RPCs)) and to respond to these requests via the network 105. For example, the game server 120 may encode the game data in one or more data files and provide the data files to the client devices 110. In addition, the game server 120 may be configured to receive game data (e.g., player positions, player actions, player input, etc.) from the client devices 110 via the network 105. For example, the client devices 110 may be configured to periodically send player input and other updates to the game server 120, which the game server 120 uses to update the game data in the game database 115 to reflect any or all changes to the game.

[0046] In the illustrated embodiment, the server 120 includes a general game module 145, a commercial game module 150, a data collection module 155, an event module 160, and a navigable space estimation training system 170. As mentioned above, the game server 120 interacts with a game database 115, which can be part of the game server 120 or accessed remotely (e.g., the game database 115 can be a distributed database accessed via the network 105). In other embodiments, the game server 120 includes different or additional elements. In addition, functions can be distributed between elements in a manner different from that described. For example, the game database 115 can be integrated into the game server 120.

[0047] The universal game module 145 hosts the parallel reality games for all players and serves as the authoritative source for the current state of the parallel reality games for all players. As a hosting party, the universal game module 145 generates game content for presenting to the player, such as via their corresponding client devices 110. The universal game module 145 can access the game database 115 to retrieve or store game data when hosting the parallel reality game. The universal game module 145 also receives game data (such as depth information, player input, player position, player action, landmark information, etc.) from the client device 110 and combines the received game data into the overall parallel reality game for all players of the parallel reality game. The universal game module 145 can also manage the delivery of game data to the client device 110 through the network 105. The universal game module 145 can also manage the security aspects of the client device 110, including but not limited to protecting the connection between the client device 110 and the game server 120, establishing the connection between each client device 110 and verifying the position of each client device 110.

[0048] In one embodiment including a business game module 150, it may be separate from or may be part of the general game module 145. The business game module 150 may manage various game features included in parallel reality games associated with business activities in the real world. For example, the business game module 150 may receive a request from an external system (such as a sponsor / advertiser, a business, or other entity) over the network 105 (via a network interface) to include game features associated with the business activities in the parallel reality game. The business game module 150 may then be deployed in the parallel reality game for including these game features.

[0049] The game server 120 may also include a data collection module 155. In one embodiment including the data collection module 155, it may be separate from the general game module 145 or may be part of it. The data collection module 155 may manage various game features contained within the parallel reality game associated with the data collection activities in the real world. For example, the data collection module 155 may modify the game data stored in the game database 115 to include game features associated with the data collection activities in the parallel reality game. The data collection module 155 may also analyze the data collected by the players according to the data collection activities and provide the data for access by various platforms.

[0050] The event module 160 manages players accessing events in parallel reality games. Although the term "event" is used for convenience, it should be understood that the term does not necessarily refer to a specific event at a specific time and place. Instead, it can refer to any game content that provides access control, where one or more access criteria are used to determine whether a player can access the content. Such content can be part of a larger parallel reality game that includes less or no access controlled game content or can be an independent, access-controlled parallel reality game.

[0051] The network 105 may be any type of communication network, such as a local area network (e.g., an intranet), a wide area network (e.g., the Internet), or some combination thereof. The network may also include a direct connection between the client device 110 and the game server 120. In general, communications between the game server 120 and the client device 110 may be carried using any type of wired or wireless connection via a network interface, using a variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML, JSON), or protection schemes (e.g., VPN, secure HTTP, SSL).

[0052] The technology discussed herein relates to servers, databases, software applications, and other computer-based systems, as well as the actions taken and information sent to and from such systems. Those of ordinary skill in the art will recognize that the inherent flexibility of computer-based systems allows for a wide variety of possible configurations, combinations, and division of tasks and functions between and among components. For example, the server processes discussed herein can be implemented using a single server or multiple servers working in combination. Databases and applications can be implemented on a single system, or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0053] In addition, in the case where the systems and methods discussed herein access and analyze personal information of users (e.g., players), or use personal information such as location information, users may be provided with the opportunity to control whether the program or feature collects the information, and whether or how to receive content from the system or other applications. No such information or data will be collected or used until the user is provided with effective notice of what information is to be collected and how the information is to be used. Unless the user provides permission, the information will not be collected or used, and the user can revoke or modify the permission at any time. Therefore, the user can control how information about the user is collected and how the information is used by the application or system. In addition, certain information or data may be processed in one or more ways before it is stored or used so that personally identifiable information is removed. For example, the user's identity may be processed so that personally identifiable information for the user cannot be determined.

[0054] Exemplary Augmented Reality System

[0055] In various embodiments, the augmented reality content can be provided by an augmented reality engine on the client device 110. For example, the augmented reality engine can be executed on a smart phone, a tablet computer, or a wearable device (e.g., AR goggles), or distributed between a combination of these elements. The augmented reality engine can receive a digital image stream generated by the camera component 125 of the client device 110. The digital image represents a near real-time view of the environment surrounding the client device 110.

[0056] The augmented reality engine may also receive a geo-location location from a geo-location system (e.g., positioning module 140). The augmented reality engine may use the geo-location location to identify and obtain (e.g., download from server 120) 3D map data (e.g., point cloud) for the environment surrounding the client device 110. The augmented reality engine may perform positioning by comparing the image of the client device 110 captured by the camera assembly 125 with the 3D map data to determine a more precise location of the client device 110. Using the determined location of the client device 110, the augmented reality engine may identify one or more augmented reality elements having a location in the virtual world that corresponds to a real-world location in one or more camera views in the camera assembly 125. The augmented reality engine may cause the augmented reality element to be displayed superimposed on the image captured by the camera assembly 125.

[0057] Exemplary Wearable Augmented Reality Devices

[0058] As described above, some or all of the functionality of the client device 110 may be provided by a wearable device. In one embodiment, the wearable device is an AR goggles. Alternatively, the wearable device may have other form factors, such as a visor or glasses.

[0059] Figure 3 A system environment diagram including a compact augmented reality (AR) device is illustrated according to one embodiment. The AR device may include an AR goggles 310. The system environment includes the AR goggles 310 (e.g., a head mounted device or HMD), a multi-function computing disk (VCP) 320, an outdoor game controller 330, and a server 340. The VCP 320 may adopt a variety of form factors.

[0060] In some embodiments, the HMD 310 is lightweight, small, and low-power, and the VCP 320 can provide application support or development functions to the HMD 310. In some embodiments, the VCP 320 can have stronger processing power, memory, and power storage than the HMD 310 to support the provision of more services and software functions. In one embodiment, the VCP 320 can be installed on a vehicle or the like, rather than worn, so there are fewer restrictions on size, weight, and shape. The VCP 320 unloads components from the HMD, such as cellular communications (5G), antennas, GPS, and batteries. In some embodiments, the VCP 320 can unload one or more computing processes from the HMD 310, such as content drawing in an application and communication with the back end. In this way, the VCP 320 can reduce the power consumption of the HMD 310.

[0061] The game controller 330 is a device configured to accept user input and transmit the input to the AR goggles 310. For example, a user can use the game controller 330 to select an item shown by the AR goggles 310 and interact with the item by pointing the game controller 330 at the item. In some embodiments, the game controller 330 can be used outdoors.

[0062] The components in the system environment (e.g., HMD 310, VCP 320, game controller 330, and server 340) can communicate via various communication technologies, such as Bluetooth, computer vision (CV), Wi-Fi, near field communication (NFC), Internet of Things (IoT), satellite communication, etc. This aspect and other aspects of the described embodiments can provide a socially acceptable and compact wearable product (e.g., AR goggles or AR visor) that still provides a high-performance, high-fidelity AR experience.

[0063] Figure 4A A side view of a compact AR device 400 is illustrated according to one embodiment. The AR device 400 may include two parts: for example, a front part 410 and a back part 420. Each part may encapsulate components for the AR device 400, such as sensors, cameras, batteries, optical displays, etc. These components are arranged in the two parts so that the weight of the AR device 400 is approximately balanced. For example, the weight of the front part 410 is approximately equal to the weight of the back part 420.

[0064] In one embodiment, the rear portion 420 includes one or more batteries and a processing unit. The processing unit may include a PCB and an antenna that allows the rear portion 420 to process data from the sensors of the front portion 410 and communicate with a game server or other external device. In some embodiments, the rear portion 420 may additionally house an accelerometer and a positioning unit (e.g., a GPS unit) to enable the AR device 400 to determine its location and movement within the real world.

[0065] In some embodiments, the rear portion 420 and the front portion 410 are connected via one or more straps 430 that rest on either side of the user's head. For example, a left strap may connect the front portion 410 near the user's left temple and connect the rear portion 420 behind the user's left ear. In one embodiment, the strap 430 is made of a flexible and elastic material that allows the AR device 400 to stretch to comfortably fit a variety of head sizes and shapes. Figure 7 As shown, the flexible strap 430 may additionally allow the AR device 400 to be foldable / collapsible for compact storage.

[0066] Figure 4BA front view of a compact AR device 400 is illustrated according to one embodiment. In one embodiment, the front includes one or more optical displays 412, speakers, microphones, cameras, and additional sensors 416. In some embodiments, the number of components located in the front 410 is kept low to prevent overheating near the user's eyes and face. However, this can be balanced by providing a substantially uniform weight distribution between the front and back of the AR device 400 (e.g., placing heavier components that generate relatively less heat in the front 410, and placing components that generate more heat in the back 420). The placement of the components of the front 410 can be determined by the width of the (multiple) optical displays 412. For example, only components that can fit the footprint of the optical display 412 can be included in the front 410, so that the front 410 does not exceed the width of the user's head. This allows the front 410 to have a small and socially acceptable volume. For example, the size of the front 410 can be similar to a pair of sunglasses. The compact shape of the front 410 allows the AR device 400 to adapt to people's heads of different widths without relying on adjustments.

[0067] The optical display 412 of the front portion 410 is positioned in front of the wearer's eyes. In one embodiment, the optical display 412 can emit image light to the user's eyes to generate visual content for augmented reality games. The optical display 412 can be transparent to allow light from the environment of the AR device 400 to pass through the optical display 412, so that the user can see the image light and the environment at the same time. In this way, the user can perceive the augmented reality objects as being superimposed on the environment around them. Although Figure 4B A single optical display 412 is illustrated, but in some embodiments, the AR device 400 may have a pair of optical displays 412 (eg, arranged in front of each eye of the user).

[0068] In some embodiments, the AR device 400 may include a space between the optical display(s) 412 and the user's eyes. The AR device 400 may include vision correction glasses or sunglasses that fit into the space. In some embodiments, the vision correction glasses or sunglasses may be incorporated into the AR goggles by temporarily or permanently connecting to the AR device 400 using one or more connectors. In some embodiments, the AR device 400 may include an interchangeable forehead pad 414.

[0069] In some embodiments, the AR device 400 may include a sunshade lens incorporated into the AR goggles. For example, the AR device 400 may be used for outdoor gaming, and the AR goggles preferably allow a portion of ambient light (e.g., sunlight) to pass through the lens so that the user can still see the AR content provided by the optical display 412. . In one embodiment, the sunshade lens is electronically adjustable (e.g., electrochromic) to change the level of sunshade according to ambient light conditions. For example, on a sunny day, the sunshade lens may allow 30% of ambient light to pass through, while on a cloudy day or at dusk, the sunshade lens may allow 60% of ambient light to pass through. In another embodiment, the sunshade lens may be removable so that the user can replace the lens to obtain the desired level of light blocking, or remove it together when the AR device 400 is used indoors.

[0070] In one example, the AR device 400 may include an electronic component that includes an environmental sensor 416, such as a camera, a sensor, and / or a communication component (e.g., Bluetooth, Wi-Fi, cellular network, 5G, etc.), a computing unit (CPU, GPU), a memory, and a circuit, etc. In another example, the environmental sensor 416 includes a camera, a depth sensor, and a microphone. In some embodiments, the speaker is positioned just above the wearer's ear. In another embodiment, the speaker is positioned near the user's temple in the front.

[0071] In some embodiments, the AR device 400 may include a set of antennas mounted on the AR device 400. These antennas allow for better communication with cellular tower antennas. It should be understood that any number of antennas may be mounted on the AR device 400, the only limitation being the size of the antennas and the available space on (and within) the AR device 400.

[0072] In some embodiments, the AR device 400 includes a socket for connecting the AR goggles to a standalone device. The connector of the auxiliary device can be a port for cable attachment or a permanently attached cable to connect a separate device to the AR goggles. In one embodiment, this standalone device is a mobile phone running an application that controls the AR goggles. For example, the wearer can connect the AR goggles to their mobile phone to pass images and sensor data captured by the AR goggles to the mobile phone for processing, or have the mobile phone provide AR data to display AR content on the optical display 412. In another embodiment, the standalone device is an external battery pack that powers the AR goggles. In this embodiment, the external battery can reduce the weight of the goggles by replacing the battery of the AR goggles, or it can increase the battery life of the AR goggles by supplying additional power. In any embodiment, the auxiliary device can have one or more connectors so that the AR goggles can be connected to multiple devices at once. In one embodiment, the VCP or outdoor game controller is connected to the AR goggles via a cable plugged into the socket.

[0073] The AR device 400 may also include one or more batteries to provide power to other components. Additionally or alternatively, power may be provided through a connector from an auxiliary device. The shape of the battery may be customized to fit the rounded shape of the rear portion, such as flush with the back of the user's head.

[0074] The rear portion 420 of the AR device 400 may have an identifier such as a QR code or another visual encoder on its surface. The identifier may be used to identify devices in the same environment. For example, another compact AR device or client device may capture an image of the identifier on the AR device 400 and process the image to create an identity for the user wearing the AR device 400 in the image. In this way, the device interacts in a multiplayer setting by identifying other game players nearby and establishing contact with them. In some embodiments, the connection between devices may also be established through a message stream or a shared location between devices.

[0075] Figure 5A An example AR device 500 having a flexible strap is illustrated according to one embodiment. Figure 5A As shown, the front portion 510 and the rear portion 520 are connected by a flexible strap 530. In some embodiments, the AR device 500 may also include an adjustment region 532 through which the flexible strap 530 may be pulled to increase or decrease its length.

[0076] Figure 5B An example AR device 500 with a non-flexible strap is illustrated, according to one embodiment. Figure 5AAs shown, the front portion 510 and the back portion 520 are connected by an inflexible strap 530. In some embodiments, the AR device 500 includes an adjuster 534 that can allow the inflexible strap 530 to slide to a desired length to fit the user's head.

[0077] In some embodiments, the band 530 (flexible or non-flexible) can house cables that electronically couple components in the front 510 with components in the rear 520. In some embodiments, the cables can be braided to allow the cables to be safely bent and manipulated. In some embodiments, 30 to 50 cables (e.g., coaxial cables) can be passed through each band 530 to connect the front and rear components while keeping heat dissipation to a minimum. In some embodiments, the band 530 can be customized for aesthetic value. For example, the band 530 can be interchangeable so that the user can switch the color or pattern of the band 530 based on their aesthetic preferences.

[0078] Figure 6 An example AR device 600 with an interchangeable forehead pad is illustrated, according to one embodiment. Figure 6 As shown, the AR device 600 can include a forehead pad 610 for providing stability and comfort of the device. For example, the forehead pad 610 can distribute the weight of the front components across the user's forehead, rather than having the AR goggles rest primarily on the user's nose bridge or cheekbones. To accommodate different head shapes, the forehead pad 610 can be interchangeable and customizable into certain shapes to fit the user. Figure 6 Two example embodiments of the forehead pad 610 are shown, one of which is tall (e.g., tall forehead pad 612) and the other is short (e.g., short forehead pad 614). Additional customization of the forehead pad 610 is possible. For example, the forehead pad 610 can vary in width to accommodate user preferences. In some embodiments, the material of the forehead pad 610 can be customizable. For example, the forehead pad 610 can be plastic, but a terry cloth cover can be customized to act as a sweatband. In some embodiments, the forehead pad 610 is coupled to the AR goggles via a pair of connectors. In one embodiment, the forehead pad 610 has a male connector 622 that can fit into female connectors 624 on both sides of the AR goggles. In some embodiments, the AR device 600 can include additional connections. In some embodiments, the forehead pad 610 can be interchangeable with a visor form factor having a sunshade edge that extends forward beyond the optical display, thereby providing a visor form factor for the AR device 600.

[0079] Figure 7 An example AR device 700 for storing a folded device is shown in FIG. Figure 7As shown, the AR device 700 has flexible side straps 730, and when not in use, the AR device 700 can be folded / closed for compact storage. The flexible side straps allow the rear portion 720 to meet the front portion 710. In some embodiments, the folded AR device 700 can be shaped to fit into a traditional sunglass case for easy transportation. In the embodiment shown, the rear portion 720 is curved to closely fit the surface of the front portion 710 resting on the user's forehead.

[0080] Figure 8 An example AR device 800 having a visor form factor is illustrated, according to one embodiment. Figure 8 As shown, the AR device 800 includes a front portion 810, a rear portion 820, and a strap 830 connecting the two portions. The front portion 810 of the AR device 800 includes a visor edge, rather than the visor edge of FIG. Figure 7 The goggle form factor shown in . In some embodiments, the visor edge can be used as a heat sink and allow more components to be located in the front 810, so that the back 820 can be smaller or have more room for a larger battery.

[0081] Compute device architecture examples

[0082] Fig. 9 is a block diagram illustrating components of an example machine capable of reading instructions from a machine-readable medium and executing the instructions in a processor. Specifically, Fig. 9 A diagrammatic representation of a machine is shown in the example form of a computer system 900. The computer system 900 may be used to execute instructions 924 (e.g., program code or software) for causing the machine to perform any one or more of the methods (or processes) described herein, including those methods associated with and described in connection with components (or modules) of a game server 110 or a client device 110.

[0083] The machine may be a server computer, a client computer, a personal computer (PC), a tablet computer, a set-top box (STB), a smart phone, a network router, a switch or bridge, a telephone signal tower, or any device capable of executing instructions 924 (either sequentially or otherwise) to specify actions to be taken by the machine. Further, while only a single machine is illustrated, the term "machine" should also be taken to include any group of machines that individually or collectively execute instructions 924 to perform any one or more of the methodologies discussed herein.

[0084] The example computer system 900 includes one or more processing units (typically one or more processors 902). For example, the processor 902 is a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a controller, a state machine, one or more application specific integrated circuits (ASICs), one or more radio frequency integrated circuits (RFICs), or any combination of these. Any processor 902 referenced herein may refer to a single processor or multiple processors. The computer system 900 also includes a main memory 904. The computer system may include a storage unit 916. The processor 902, the memory 904, and the storage unit 916 communicate via a bus 908.

[0085] In addition, the computer system 900 may include a static memory 906, a display driver 910 (e.g., driving a plasma display screen (PDP), a liquid crystal display screen (LCD), or a projector). The computer system 900 may also include an alphanumeric input device 914 (e.g., a keyboard), a cursor control device 914 (e.g., a mouse, a trackball, a joystick, a motion sensor, or other pointing instrument), a signal generating device 918 (e.g., a speaker), and a network interface device 920, which are also configured to communicate via the bus 908.

[0086] The storage unit 916 includes a machine-readable medium 922 on which are stored instructions 924 (e.g., software) embodying any one or more of the methods or functions described herein. The instructions 924 may also reside completely or at least partially in the main memory 904 or the processor 902 (e.g., in a cache memory of the processor) during execution of the instructions 924 by the computer system 900, the main memory 904 and the processor 902 also constituting machine-readable media. The instructions 924 may be transmitted or received over a network 970 (e.g., network 105) via the network interface device 920.

[0087] Although the machine-readable medium 922 is shown as a single medium in the example embodiment, the term "machine-readable medium" should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) that can store instructions 924. The term "machine-readable medium" should also be taken to include any medium that can store instructions 924 for execution by a machine and cause the machine to perform any one or more of the methods disclosed herein. "Machine-readable media" includes, but is not limited to, data repositories in the form of solid-state memories, optical media, and magnetic media.

[0088] Additional considerations

[0089] Some parts of the above description of the embodiments describe the embodiments in the form of algorithmic processes or operations. These algorithmic implementations and representations are usually used by technicians in the computing field to effectively convey their main work content to technicians in other fields. Although these operations are described from the perspective of function, calculation or logic, it should be understood that they are implemented by a computer program including instructions for execution by a processor or can be implemented by equivalent circuits, microcodes, etc. In addition, it is sometimes convenient to refer to the arrangement of these functional operations as modules without losing generality.

[0090] As used herein, any reference to "one embodiment" or "an embodiment" means that a particular element, feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The phrase "in one embodiment" appearing in different places in the specification does not necessarily all refer to the same embodiment. Similarly, the use of "one" or "an" before an element or component is merely for convenience. Unless it is obvious that it has another meaning, the embodiment should be understood to mean that one or more of the elements or components are present.

[0091] When numerical values ​​are described as "approximately" or "substantially" (or derivatives thereof), unless the context clearly indicates otherwise, these numerical values ​​should be interpreted as plus or minus 10% of the exact value. For example, "approximately ten" should be understood to mean "in the range of nine to eleven".

[0092] As used herein, the terms "comprises," "comprising," "including," "including," "having," "having," or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, article, or apparatus that comprises a series of elements is not necessarily limited to only those elements, but may also include other elements not expressly listed or elements inherent to the process, method, article, or apparatus. Further, unless expressly provided otherwise, "or" refers to an inclusive or and not an exclusive or. For example, condition A or B satisfies any of the following: A is true (or exists) and B is false (or does not exist); A is false (or does not exist) and B is true (or exists); both A and B are true (or exist).

[0093] Although the subject matter has been described in detail with respect to specific exemplary embodiments and methods thereof, it should be understood that those skilled in the art, after obtaining an understanding of the above, can easily produce changes, variations and equivalents to these embodiments. Accordingly, the scope of the present disclosure is exemplary rather than restrictive, and the present disclosure does not exclude modifications, variations or supplements to the subject matter as would be apparent to a person of ordinary skill in the art.

Claims

1. A head mounted device (HMD), comprising: a front portion comprising an optical display configured to output image light toward an eye of a user; a rear portion, the rear portion being arranged with the front portion to balance the weight of the HMD; as well as One or more straps connecting the front portion and the back portion such that the two portions rest on either side of the user's head.

2. The HMD of claim 1, wherein the front portion further comprises one or more of a speaker, a microphone, a camera, or a sensor.

3. The HMD of claim 1, wherein the rear portion comprises one or more batteries or processing units.

4. The HMD of claim 1, wherein the optical display is transparent to allow light from an environment of the HMD to pass through the optical display such that the user sees the image light and the environment at the same time.

5. The HMD according to claim 1, further comprising: One or more vision correction glasses or sunglasses incorporated into the HMD. 6 . The HMD according to claim 1 , wherein the rear portion includes an identifier for identifying the HMD to establish a connection with a second HMD.

7. The HMD of claim 1, wherein the one or more straps house a cable that electronically couples the front portion and the rear portion.

8. The HMD according to claim 1, further comprising: Interchangeable forehead pad attachment.

9. The HMD of claim 8, wherein the forehead pad attachment is configured to distribute the weight of the front portion across the forehead of the user.

10. The HMD of claim 1, wherein the HMD is foldable.

11. The HMD of claim 1, wherein the optical display outputs image light to generate visual content for an augmented reality (AR) game.

12. A gaming device comprising: A head mounted device (HMD), the HMD comprising: a front portion, the front portion comprising an optical display configured to output image light to generate visual content to a user's eyes for an augmented reality (AR) game; a rear portion, the rear portion being arranged with the front portion to balance the weight of the HMD; and one or more straps connecting the front portion and the back portion so that the two portions rest on either side of the user's head; and A game controller is configured to receive input from the user and transmit the input to the HMD.

13. The device of claim 12, wherein the front portion further comprises one or more of a speaker, a microphone, a camera, and a sensor.

14. The device of claim 12, wherein the rear portion comprises one or more of a battery and a processing unit.

15. The device of claim 12, wherein the optical display is transparent to allow light from the environment of the HMD to pass through the optical display so that the user sees the image light and the environment at the same time.

16. The device of claim 12, wherein the HMD further comprises: One or more vision correction glasses or sunglasses incorporated into the HMD.

17. The device of claim 12, wherein the rear portion includes an identifier for identifying the HMD to establish a connection with a second HMD.

18. The apparatus of claim 12, wherein the one or more straps house cables that electronically couple the front portion and the rear portion.

19. The device of claim 12, wherein the HMD comprises an interchangeable forehead pad attachment configured to distribute the weight of the front portion across the forehead of the user.

20. The apparatus of claim 12, further comprising: A versatile computing disk (VCP) that provides power, memory, and power storage for the HMD.

Images