Compact Head-Mounted Augmented Reality System

Abstract

A head-mounted device (HMD) may include a front portion, a rear portion, and one or more bands. The front portion may include an optical display configured to output image light to the user's eyes. The rear portion may be arranged with the front portion to balance the weight of the HMD. The one or more bands connect the front portion and the rear portion so that the two portions rest on both sides of the user's head.

Description

Technical Field

[0001] The present invention generally relates to augmented reality, and more particularly to compact AR devices.

Background Art

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 356,472, filed Jun. 28, 2022, which is incorporated by reference.

[0003] A variety of AR devices for consumer use cases have been developed over the past 10 to 15 years. The widespread adoption of compact AR devices has been hampered by challenges in providing a balance between usefulness and a socially acceptable form factor. Technical limitations have been a barrier to achieving a desirable level of functionality in conjunction with a form factor acceptable to a wide range of users. Attempting to fit the components of a useful system into the form factor of glasses or a headset makes the wearable device heavy and large, particularly for the user's face, which is inconvenient and uncomfortable. In contrast, devices with a smaller form factor that may be considered more convenient and socially acceptable have low functionality and low practical value.

Summary of the Invention

[0004] The above and other problems can be addressed by a compact head-mounted augmented reality (AR) device that integrates electronic components for providing AR content to a wearer. Various features, aspects, and advantages of this compact AR device can be better understood by reference to the following description. The accompanying drawings illustrate specific embodiments and serve to explain various principles in conjunction with the description. However, the drawings should not be considered limiting. Rather, the scope of protection should be determined from the claims. The following description focuses on a specific use case - AR-enabled goggles for outdoor games - but the disclosed principles can also be applied to other use cases and form factors.

[0005] In one aspect, the present disclosure provides a head-mounted device (HMD). The HMD may include a front portion, a rear portion, and one or more bands. The front portion may include an optical display that outputs image light to the user's eyes. The rear portion may be arranged with the front portion to balance the weight of the HMD. The one or more bands connect the front portion and the rear portion so that the two portions rest on both sides of the user's head (e.g., the front and the back).

[0006] In another aspect, the present disclosure provides a game device. The game device may include a head-mounted device (HMD) and a game controller. The HMD may include a front portion, a rear portion, and one or more bands. The front portion may include an optical display that outputs image light to the user's eyes. The rear portion may be arranged with the front portion to balance the weight of the HMD. The one or more bands connect the front portion and the rear portion so that the two portions rest on both sides of the user's head. The game controller may receive user input and transmit the input to the HMD. BRIEF DESCRIPTION OF THE DRAWINGS

[0007]

Figure 1

Figure 2

Figure 3

Figure 4A

Figure 4B

Figure 5A

Figure 5B

Figure 6

Figure 7

Figure 8

Figure 9

[0008] Here, various embodiments including one or more of the examples shown in the drawings will be referred to in detail. Each example is provided for the purpose of explaining the described embodiments and is not intended to limit the scope of the claims. In fact, it will be apparent to those skilled in the art that various modifications and changes can be made without departing from the described principles. For example, features illustrated or described as part of one embodiment can be used with another embodiment to create yet another embodiment. Accordingly, the present disclosure is intended to cover modifications and changes within the scope of the appended claims and their equivalents.

[0009] This specification describes an embodiment of a compact AR device having components divided into two parts, a front part and a rear part. The front part of the AR device is placed in front of the user's face during use, and the rear part is placed behind the user's head. The front part and the rear part are connected by a flexible strap, and when the device is not in use, the front part and the rear part can be folded together for compact storage. The flexible strap connecting the two parts is further stretchable and can fit various head sizes and shapes with little adjustment of the AR device. In other embodiments, the compact AR device may have additional parts.

[0010] Exemplary Location - Based Parallel Reality Game System In a parallel reality game, the geography of the virtual world is parallel to at least a part of the geography of the real world, and the movements and actions of the player in the real world affect the actions in the virtual world and vice versa. The virtual world may include augmented reality (AR) content that can be displayed overlaid on the real - world image as the player moves to the corresponding real - world location. Those skilled in the art will understand, using the disclosure provided herein, that the described subject matter is applicable to other situations where AR images are displayed. Further, those skilled in the art will understand, using the disclosure provided herein, that a number of game interface configurations and underlying functions will become apparent in light of this disclosure. This disclosure is not intended to be limited to any one particular configuration.

[0011] FIG. 1 shows a networked computing environment 100 according to one or more embodiments. The networked computing environment 100 provides player interaction in a virtual world having a geography parallel to the real world. In particular, a geographic area of the real world can be directly linked or mapped to a corresponding area of the virtual world. A player can move within the virtual world by moving to various geographic locations in the real world. For example, the position of a player in the real world can be tracked and used to update the position of the player in the virtual world. Typically, the position of a player in the real world is determined by finding the position of the client device 110 with which the player is interacting with the virtual world and assuming that the player is at the same (or approximately the same) position. For example, in various embodiments, a player can interact with a virtual element if the position of the player in the real world is within a threshold distance (e.g., 10 meters, 20 meters, etc.) from the real-world position corresponding to the virtual position of the virtual element in the virtual world. For convenience, a player who is close enough to a virtual element to interact with it in this way is said to be at the real-world position corresponding to the virtual element. Further, while various embodiments are described with reference to "the position of the player", one of ordinary skill in the art will understand that such reference may refer to the position of the player's client device 110.

[0012] Referring now to FIG. 2, FIG. 2 depicts a conceptual diagram of a virtual world 210 parallel to the real world 200 that can function as a game board for a player of a parallel reality game according to one embodiment. As shown, the virtual world 210 can include a geography parallel to the geography of the real world 200. In particular, the coordinate range that defines a geographic area or space of the real world 200 is mapped to a corresponding coordinate range that defines the virtual space of the virtual world 210. The coordinate range of the real world 200 can be associated with a town, neighborhood, city, campus, location, country, continent, the entire earth, or other geographic area. Each geographic coordinate within the geographic coordinate range is mapped to a corresponding coordinate within the virtual space of the virtual world.

[0013] The position of a player in the virtual world 210 corresponds to the position of the player in the real world 200. For example, player A located at position 212 in the real world 200 has a corresponding position 222 in the virtual world 210. Similarly, player B located at position 214 in the real world has a corresponding position 224 in the virtual world. When a player moves within the geographical coordinate range of the real world, the player also moves within the range of coordinates that define the virtual space of the virtual world 210. In particular, a positioning system (e.g., a GPS system) associated with the mobile computing device carried by the player can be used to track the player's position as the player moves within the geographical coordinate range of the real world. Data associated with the position of a player in the real world 200 is used to update the position of the player within the corresponding coordinate range that defines the virtual space of the virtual world 210. In this way, a player can move along a continuous track within the coordinate range that defines the virtual space of the virtual world 210 simply by moving within the corresponding geographical coordinate range of the real world 200 without checking in at a specific individual position in the real world 200 or periodically updating position information.

[0014] A location-based game can include a plurality of game objectives that require a player to move to or interact with various virtual elements or virtual objects scattered at various virtual locations in a virtual world. Some or all of the virtual objects can be AR objects displayed by the client device 110. The AR objects can include biological and inanimate objects. The biological objects can be referred to as virtual characters. The virtual characters can represent game characters such as non-player characters (NPCs). The player can move to these virtual locations by moving to the corresponding locations of the virtual elements or objects in the real world. For example, a positioning system can continuously track the player's position, and as the player continuously moves in the real world, the player also continuously moves in the parallel virtual world. The player can interact with various virtual elements or objects at a specific location to achieve or execute one or more game objectives.

[0015] For example, the game objective includes the player interacting with a virtual element 230 at a virtual location in the virtual world 210. These virtual elements 230 can be linked to landmarks, geographical locations, or objects 240 in the real world 200. Real-world landmarks or objects 240 are artworks, monuments, buildings, companies, libraries, museums, or other suitable real-world landmarks or objects. Interactions include capturing virtual items, claiming ownership, using virtual items, using virtual currency, etc. To capture these virtual elements 230, the player moves to the landmark or geographical location 240 linked to the virtual element 230 in the real world and interacts with the virtual element 230 in the virtual world 210. For example, player A in FIG. 2 may have to move to a landmark 240 in the real world 200 to interact with or capture a virtual element 230 linked to a specific landmark 240. Interactions with the virtual element 230 may require actions in the real world, such as displaying an AR representation of the virtual element overlaid on a view of the real-world location within a series of images (e.g., video) captured by a camera, or taking a "photo" of the virtual element in the real world. The series of images and photos can be captured by a camera attached to a compact AR device (e.g., AR goggles, visors, etc.).

[0016] The game objective may require the player to use one or more virtual items collected in a location-based game. For example, the player may move through the virtual world 210 in search of virtual items (such as weapons, creatures, power-ups, or other items) that are useful for achieving the game objective. These virtual items can be found or collected by moving to various locations in the real world 200 or by completing various actions either in the virtual world 210 or the real world 200. In the example shown in FIG. 2, the player uses virtual item 232 to capture one or more virtual elements 230. In particular, the player can place virtual item 232 at a location near or within virtual element 230 in virtual world 210. By placing one or more virtual items 232 in this way, the virtual elements 230 of a particular player or a particular player's team / faction can be captured.

[0017] In a particular embodiment, the player may have to collect virtual energy or other items as part of a parallel reality game. As depicted in FIG. 2, virtual energy 250 may be scattered at various locations in virtual world 210. The player can collect virtual energy 250 and other items by moving to the corresponding locations in the real world 200. Virtual energy 250 can be used to power virtual items or to perform various game objectives within the game. Other items may perform other appropriate functions depending on the game.

[0018] According to aspects of the present disclosure, a parallel reality game can be a large-scale multiplayer location-based game in which all participants in the game share the same virtual world. Players can be divided into separate teams or factions and cooperate to achieve one or more game goals, such as obtaining virtual elements or claiming ownership. Thus, a parallel reality game can essentially be a social game that promotes cooperation among players within the game. Players on opposing teams can compete against each other (or sometimes cooperate to achieve a common goal) during a parallel reality game. Players can use virtual items to attack or disrupt the progress of players on opposing teams. In some cases, players are encouraged to gather at real-world locations for cooperative events or interaction events in the parallel reality game. In these cases, the game server attempts to confirm that the players are physically present and not cheating.

[0019] A parallel reality game can include various features to enhance and facilitate gameplay within the parallel reality game. For example, players can accumulate virtual currency or other virtual rewards (such as virtual tokens, virtual points, virtual material resources, etc.) that can be used throughout the game (e.g., to purchase in-game items, exchange for other items, create items, etc.). Players can progress through various levels as they achieve one or more game goals and gain experience within the game. In some embodiments, players can communicate with each other via one or more communication interfaces provided within the game. Players can also obtain enhanced "powers" or virtual items that can be used to achieve game goals within the game. Those skilled in the art should understand that, using the disclosure provided herein, various other game features can be included in a parallel reality game without departing from the scope of the present disclosure.

[0020] Returning to FIG. 1, the networked computing environment 100 uses a client-server architecture in which a game server 120 communicates with client devices 110 via a network 105 to provide parallel reality games to players of the client devices 110. The networked computing environment 100 also includes other external systems such as a sponsor / advertiser system and a business system. Although only one client device 110 is shown in FIG. 1, any number of client devices 110 or other external systems can be connected to the game server 120 via the network 105. Further, the networked computing environment 100 may include different or additional elements, and the functionality may be distributed between the client devices 110 and the server 120 in a manner different from the methods described below.

[0021] The client device 110 can be any portable computing device that a player can use to interact with the game server 120. For example, the client device 110 can be a wireless device, a personal digital assistant (PDA), a portable game device, a cellular phone, a smartphone, a tablet, a navigation system, a handheld GPS system, a wearable computing device, a display with one or more processors, or other such devices.

[0022] In one embodiment, the functions described below that are performed by the client device 110 are distributed between a mobile computing device such as a tablet or smartphone and a wearable device such as AR goggles. For example, one or more cameras of the AR goggles worn by the player capture an image of the environment around the player, the player's smartphone determines the real-world positions at which to display one or more AR elements, and one or more displays attached to the AR goggles can display the AR elements at the determined positions. In another embodiment, the AR goggles provide all of the functions described below that are performed by the client device 110.

[0023] In one embodiment, client sensory data is provided. The client device 110 includes a camera assembly 125 that captures two-dimensional image data of a scene of the physical environment in which the client device 110 is present. In the embodiment shown in FIG. 1, the client device 110 includes additional software components such as a game module 135, a positioning module 140, and a light assembly 143. The client device 110 can include various other input / output devices for receiving information from or providing information to the player. Examples of input / output devices include a display screen, a touch screen, a touch pad, data input keys, a speaker, and a microphone suitable for voice recognition. The client device 110 can also include various other sensors for recording data from the client device 110, including but not limited to a motion sensor, an accelerometer, a gyroscope, other inertial measurement units (IMUs), a barometer, a positioning system, a thermometer, a light sensor, and the like. The client device 110 can further include a network interface for providing communication via the network 105. The network interface can include any suitable components for interacting with one or more networks, including, for example, a transmitter, a receiver, a port, a controller, an antenna, or other suitable components.

[0024] The camera assembly 125 captures image data of a scene in the environment where the client device 110 is located. The camera assembly 125 can utilize various photosensors with a color capture range at various capture rates. The camera assembly 125 can include a fisheye lens or a telephoto lens. The camera assembly 125 can be configured to capture a single image or video as image data. The camera assembly 125 captures image data and shares the image data with a computing device on the client device 110. The image data can be attached with metadata that describes other details of the image data, including sensory data (e.g., temperature, ambient brightness, etc.) and capture data (e.g., exposure, warmth, shutter speed, focal length, capture time, etc.). The camera assembly 125 can include one or more cameras capable of capturing image data. In one example, the camera assembly 125 includes a pair of cameras attached to the left and right sides of the AR goggles and is configured to capture stereoscopic image data from a position close to the position of the wearer's eyes. In another example, the camera assembly 125 includes one camera attached approximately in the center of the front of the AR goggles. In various other embodiments, the camera assembly 125 includes multiple cameras each configured to capture image data.

[0025] The game module 135 provides an interface for players to participate in the parallel reality game. The game server 120 transmits game data to the client device 110 via the network 105, and the game module 135 of the client device 110 uses this to provide a local version of the game to players located away from the game server 120. The game server 120 can include a network interface for providing communication via the network 105. The network interface can include any suitable components for interacting with one or more networks, for example, transmitters, receivers, ports, controllers, antennas, or other suitable components.

[0026] The game module 135 executed by the client device 110 provides an interface between the player and the parallel reality game. The game module 135 can be attached to AR goggles and display a user interface on a display device such as an optical display placed in front of the wearer's eyes. The user interface includes virtual elements associated with the game, enabling the player to interact with the virtual elements to execute various game objectives. The game module 135 can generate virtual content or adjust virtual content according to other information received from other components of the client device. For example, the game module 135 can adjust virtual objects displayed in the user interface according to the depth information of the scene captured with image data.

[0027] The game module 135 can also control various other outputs so that players can interact with the game. For example, the game module 135 can control various audio, vibration, or other notifications. The game module 135 can access the game data received from the game server 120 to provide players with an accurate representation of the game. The game module 135 can receive and process player input and provide updates to the game server 120 via the network 105. The game module 135 can also generate or adjust game content displayed by the client device 110. For example, the game module 135 can generate virtual elements based on information describing the passable space of a scene (e.g., determined by the passable space estimation model 130). For example, the game module 135 can determine the path of virtual elements within the scene of the passable space.

[0028] The positioning module 140 can be any device or circuit for monitoring the position of the client device 110. For example, the positioning module 140 can use techniques such as triangulation or proximity to a cellular phone base station or Wi-Fi hotspot, or other suitable techniques for determining position, based on the IP address, using a satellite navigation positioning system (e.g., GPS system, Galileo positioning system, Global Navigation Satellite System (GLONASS), Beidou satellite navigation positioning system), an inertial navigation system, a dead reckoning system, to determine the actual or relative position. The positioning module 140 can further include various other sensors that can help accurately position the client device 110. For example, the client device 110 can perform location identification by comparing an image captured by one or more cameras of the client device with a 3D map.

[0029] When a player walks around in the real world with the client device 110, the positioning module 140 tracks the player's position and provides the player's position information to the game module 135. Based on the player's actual position in the real world, the game module 135 updates the position of the player in the virtual world associated with the game. Therefore, the player can interact with the virtual world just by carrying or moving the client device 110 in the real world. In particular, the position of the player in the virtual world can correspond to the position of the player in the real world. The game module 135 can provide the player's position information to the game server 120 via the network 105. In response, the game server 120 can execute various techniques to verify the position of the client device 110 to prevent cheaters from faking the position of the client device 110. It should be understood that the position information associated with the player is used only after the player has been notified of the access to the player's position information and given permission as to how the position information is used in the context of the game (e.g., updating the position of the player in the virtual world). Further, the position information associated with the player is stored and maintained in a way that protects the player's privacy.

[0030] The light assembly 143 includes one or more lights that can illuminate the environment around the client device. In one embodiment, the light assembly 143 includes a flash associated with the camera assembly 125. The flash can illuminate for a short time to assist in taking one photo or can illuminate for a long time to function as a flashlight. In other embodiments, the light assembly 143 can include various types of lights, such as infrared lights that illuminate an object and enable an infrared camera to capture an image of the scene without illuminating the scene with visible light.

[0031] The game server 120 is an arbitrary computing device and can include one or more processors and one or more computer-readable storage media. The computer-readable storage media can store instructions that cause the processor to execute operations. The game server 120 can include the game database 115 or can communicate with the game database 115. The game database 115 stores game data used in a parallel reality game provided to the client 110 via the network 105.

[0032] The game data stored in the game database 115 may include: (1) data associated with the virtual world in the parallel reality game (e.g., image data used to render the virtual world on a display device, geographical coordinates of the positions in the virtual world, etc.), (2) data associated with the players of the parallel reality game (e.g., player profiles including, but not limited to, player information, player experience levels, player currency, current player position in the virtual world / real world, player energy levels, player preferences, team information, faction information, etc.), (3) data associated with the game objectives (e.g., current game objectives, states of the game objectives, past game objectives, future game objectives, data associated with desired game objectives, etc.), (4) data associated with the virtual elements of the virtual world (e.g., positions of the virtual elements, types of the virtual elements, game objectives associated with the virtual elements, real-world position information corresponding to the virtual elements, behaviors of the virtual elements, relevance of the virtual elements, etc.), (5) data associated with the positions linked to real-world objects, landmarks, and elements of the virtual world (e.g., positions of real-world objects / landmarks, descriptions of real-world objects / landmarks, relevance of the virtual elements linked to real-world objects, etc.), (6) the state of the game (e.g., current number of players, current state of the game objectives, player leaderboards, etc.), (7) data associated with player actions / inputs (e.g., current player position, past player positions, player movements, player inputs, player queries, player communications, etc.), and (8) other relevant or acquired data used during the implementation of the parallel reality game. The game data stored in the game database 115 can be input either offline or in real time by the system administrator or by data received from users / players of the system 100, such as from the client device 110 via the network 105.

[0033] The game server 120 can be configured to receive requests for game data from the client device 110 (e.g., via remote procedure calls (RPCs)) and respond to those requests via the network 105. For example, the game server 120 can encode game data into one or more data files and provide those data files to the client device 110. Further, the game server 120 can be configured to receive game data (e.g., player position, player actions, player input, etc.) from the client device 110 via the network 105. For example, the client device 110 can be configured to periodically send player input and other updates to the game server 120, and the game server 120 can use these updates to update the game data in the game database 115 to reflect all changed conditions of the game.

[0034] In the illustrated embodiment, the server 120 includes a universal game module 145, a commercial game module 150, a data collection module 155, an event module 160, and a passable space estimation training system 170. As described above, the game server 120 interacts with a game database 115 that can be part of the game server 120 or remotely accessed (e.g., the game database 115 can be a distributed database accessed via the network 105). In other embodiments, the game server 120 includes different or additional elements. Further, the functionality can be distributed among the elements in a different manner than described. For example, the game database 115 can be integrated into the game server 120.

[0035] The Universal Game Module 145 hosts parallel reality games for all players and serves as a reliable source for the current state of the parallel reality games of all players. As the host, the Universal Game Module 145 generates game content for presentation to players, for example, via respective client devices 110. The Universal Game Module 145 may access the game database 115 to obtain or save game data when hosting parallel reality games. The Universal Game Module 145 also receives game data (such as depth information, player input, player position, player actions, landmark information, etc.) from the client devices 110 and incorporates the received game data into the overall parallel reality game for all players of the parallel reality game. The Universal Game Module 145 can also manage the distribution of game data to the client devices 110 via the network 105. The Universal Game Module 145 can also manage the security aspects of the client devices 110, including, but not limited to, securing the connection between the client devices 110 and the game server 120, establishing connections between various client devices 110, and verifying the location of various client devices 110.

[0036] In embodiments in which it is included, the Commercial Game Module 150 may be separate from or part of the Universal Game Module 145. The Commercial Game Module 150 can manage the inclusion of various game functions linked to real-world commercial activities within the parallel reality game. For example, the Commercial Game Module 150 can receive requests via the network 105 (via the network interface) from external systems such as sponsors / advertisers, enterprises, or other entities to include game functions linked to commercial activities in the parallel reality game. The Commercial Game Module 150 can then arrange to include these game functions in the parallel reality game.

[0037] The game server 120 can further include a data collection module 155. In embodiments in which it is included, the data collection module 155 can be separate from or a part of the universal game module 145. The data collection module 155 can manage including various game functions linked to real-world data collection activities in the parallel reality game. For example, the data collection module 155 can modify the game data stored in the game database 115 to include game functions linked to data collection activities within the parallel reality game. The data collection module 155 can also analyze the data collected by the player according to the data collection activities and provide the data so that various platforms can access it.

[0038] The event module 160 manages a player's access to events within the parallel reality game. For convenience, the term "event" is used, but it should be understood that this term does not necessarily refer to a specific event at a specific location or time. Rather, it can refer to any provision of access-controlled game content for which one or more access criteria are used to determine whether a player can access that content. Such content can be part of a larger parallel reality game that includes game content with little or no access control, or it can be a stand-alone access-controlled parallel reality game.

[0039] Network 105 can be any type of communication network, such as a local area network (e.g., intranet), a wide area network (e.g., Internet), or a combination thereof. The network can also include a direct connection between the client device 110 and the game server 120. Generally, communication between the game server 120 and the client device 110 can be performed via a network interface using any type of wired or wireless connection, various communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encoding or formats (e.g., HTML, XML, JSON), or protection schemes (e.g., VPN, secure HTTP, SSL).

[0040] The techniques described herein refer to servers, databases, software applications, and other computer-based systems, as well as actions performed in these systems and information transmitted between these systems. One of ordinary skill in the art can understand that the inherent flexibility of computer-based systems enables various possible configurations, combinations, and divisions of tasks and functions between components. For example, the server processing described herein can be implemented using a single server or a combination of multiple servers. Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0041] Furthermore, when the systems and methods described herein access and analyze personal information about a user (e.g., a player), or utilize personal information such as location information, the user may be provided with an opportunity to control whether the program or function collects information, whether the system or other application receives content, or the method of receiving. Such information or data is not collected or used until the user is provided with a meaningful notice about the information being collected and how it will be used. The information is not collected or used without the user's consent, and the consent can be revoked or changed by the user at any time. Thus, the user can control how information about the user is collected and used by the application or system. Additionally, certain information or data is processed in one or more ways before being stored or used, and information that can identify an individual can be deleted. For example, a user's ID can be processed so that information that can identify an individual about the user is not identified.

[0042] Exemplary Augmented Reality System In various embodiments, augmented reality content can be provided by an augmented reality engine on the client device 110. For example, the augmented reality engine can be executed on a smartphone, tablet, or wearable device (e.g., AR goggles), or can be distributed among a combination of such elements. The augmented reality engine can receive a stream of digital images generated by the camera assembly 125 of the client device 110. The digital images represent a nearly real-time view of the environment around the client device 110.

[0043] The augmented reality engine may also receive geographical location information from a geographical location information positioning system (e.g., positioning module 140). The augmented reality engine may use the geographical location information to identify and obtain (e.g., download from server 120) 3D map data (e.g., point cloud) of the environment around client device 110. The augmented reality engine may perform positioning to determine a more accurate position of client device 110 by comparing the image captured by camera assembly 125 of client device 110 with the 3D map data. The augmented reality engine may use the determined position of client device 110 to identify one or more augmented reality elements having locations in the virtual world corresponding to real-world locations within the field of view of one or more cameras of camera assembly 125. The augmented reality engine may superimpose and display the augmented reality elements on the image captured by camera assembly 125.

[0044] Exemplary Wearable Augmented Reality Device As described above, some or all of the functions of client device 110 may be provided by a wearable device. In one embodiment, the wearable device is AR goggles. Alternatively, the wearable device may have other form factors such as visors or glasses.

[0045] FIG. 3 is a system environment diagram including a compact augmented reality (AR) device according to one embodiment. The AR device may include a pair of AR goggles 310. The system environment includes AR goggles 310 (e.g., head-mounted device or HMD), a versatile computing pack (VCP) 320, an outdoor game controller 330, and a server 340. The VCP 320 can take on multiple form factors.

[0046] In some embodiments, the HMD 310 is lightweight, small-sized, and low-power, and the VCP 320 can provide application support or development capabilities to the HMD 310. In some embodiments, the VCP 320 has higher processing power, memory, and power storage than the HMD 310 and can support the provision of more services and software functions. In one embodiment, the VCP 320 can be carried or mounted on a vehicle or the like rather than worn, with fewer restrictions on size, weight, and shape. The VCP 320 reduces the load from HMD components such as cellular communication (5G), antennas, GPS, and batteries. In some embodiments, the VCP 320 can reduce the load of one or more computing processes such as content rendering in an application and communication with the backend from the HMD 310. In this way, the VCP 320 can reduce the power consumption of the HMD 310.

[0047] The game controller 330 is a device configured to receive user input and transmit that input to the AR goggles 310. For example, a user can use the game controller 330 to select an item displayed on the AR goggles 310 and interact with the item by pointing the game controller 330 at the item. In some embodiments, the game controller 330 can be used outdoors.

[0048] Components within the system environment, such as the HMD 310, VCP 320, game controller 330, and server 340, can communicate via various communication technologies such as Bluetooth®, computer vision (CV), Wi-Fi, near-field communication (NFC), Internet of Things (IoT), and satellite communication. This aspect and other aspects of the described embodiments can provide socially acceptable compact wearable products (e.g., AR goggles or AR visors) and can provide a high-performance and high-fidelity AR experience.

[0049] FIG. 4A shows a side view of a compact AR device 400 according to one embodiment. The AR device 400 may include two parts, such as a front part 410 and a rear part 420. Each part may house components of the AR device 400, such as sensors, cameras, batteries, optical displays, etc. The components are arranged in two parts so that the weight of the AR device 400 is substantially evenly distributed. For example, the weight of the front part 410 is approximately equal to the weight of the rear part 420.

[0050] In one embodiment, the rear part 420 includes one or more batteries and a processing unit. The processing unit may include a PCB and an antenna that enable the rear part 420 to process data from the sensors of the front part 410 and communicate with a game server or other external devices. In some embodiments, the rear part 420 further houses an accelerometer and a positioning unit (e.g., a GPS unit), enabling the AR device 400 to determine its position and movement within the real world.

[0051] In some embodiments, the rear part 420 and the front part 410 are connected via one or more bands 430 that rest on both sides of the user's head. For example, the left band may connect to the front part 410 near the user's left temple and to the rear part 420 behind the user's left ear. In one embodiment, the band 430 is made of a flexible and stretchable material so that the AR device 400 can stretch and comfortably fit various head sizes and shapes. Further, the flexible band 430 enables the AR device 400 to be collapsible / foldable so that it can be compactly stored as seen in FIG. 7.

[0052] Figure 4B shows a front view of a compact AR device 400 according to one embodiment. In one embodiment, the front portion includes one or more optical displays 412, speakers, microphones, cameras, and additional sensors 416. In some embodiments, the number of components disposed in the front portion 410 is kept low so that excessive heat does not come close to the user's eyes or face. However, this allows the weight distribution between the front and back of the AR device 400 to be balanced such that, for example, heavy components that generate relatively little heat are placed in the front portion 410 and components that generate more heat are placed in the rear portion 420. The arrangement of the components in the front portion 410 can be determined by the width of the optical display 412. For example, by including only components that fit within the footprint of the optical display 412 in the front portion 410, the front portion 410 can be prevented from extending beyond the width of the user's head. This can reduce the footprint of the front portion 410 and make it socially acceptable in size. For example, the size of the front portion 410 can be comparable to that of a pair of sunglasses. The compact shape of the front portion 410 allows the AR device 400 to accommodate various head widths of humans without relying on adjustment.

[0053] The optical display 412 of the front portion 410 is disposed in front of the wearer's eyes. In one embodiment, the optical display 412 emits image light towards the user's eyes to generate a video of an augmented reality game. The optical display 412 is transparent so that light from the environment of the AR device 400 can pass through the optical display 412, and the user can view the image light and the environment simultaneously. In this way, the user can perceive that the augmented reality objects are overlaid on the surrounding environment. Figure 4B shows a single optical display 412, but the AR device 400 can have a pair of optical displays 412 in some embodiments (e.g., disposed in front of each of the user's eyes).

[0054] In some embodiments, the AR device 400 may include a space between the optical display 412 and the user's eyes. The AR device 400 may include vision correction glasses or sunglasses that fit into that space. In some embodiments, the vision correction glasses or sunglasses may be incorporated into the AR goggles by temporarily or permanently connecting them to the AR device 400 using one or more connectors. In some embodiments, the AR device 400 may include a replaceable forehead pad 414.

[0055] In some embodiments, the AR device 400 may include a light-blocking lens incorporated into the AR goggles. For example, the AR device 400 is used for outdoor games, and it is preferable that the AR goggles transmit a portion of the ambient light (e.g., sunlight) through the lens so that the user can still see the AR content provided by the optical display 412. In one embodiment, the light-blocking lens is electronically adaptable (e.g., electrochromic) and changes the light-blocking level according to the conditions of the ambient light. For example, on a bright sunny day, the light-blocking lens transmits 30% of the ambient light, and on a cloudy day or at dusk, the light-blocking lens transmits 60% of the ambient light. In another embodiment, the light-blocking lens may be removable, and the user can replace the lens to achieve a preferred light-blocking level or remove all the lenses for using the AR device 400 indoors.

[0056] In one example, the AR device 400 may include electronic components including environmental sensors 416 such as a camera, sensors, and / or communication components (e.g., Bluetooth®, Wi-Fi, cellular, 5G, etc.), a computing unit (CPU, GPU), memory, and wiring. In another example, the environmental sensors 416 include a camera, a depth sensor, and a microphone. In some embodiments, the speaker is disposed directly above the wearer's ear. In another embodiment, the speaker is disposed in the front portion near the user's temple.

[0057] In some embodiments, the AR device 400 may include a set of antennas attached to the AR device 400. The antennas improve communication with the antennas of the cellular base stations. It should be understood that any number of antennas can be attached to the AR device 400, limited only by the size of the antennas and the available space on (and inside) the AR device 400.

[0058] In some embodiments, the AR device 400 includes a socket for connecting the AR goggles to another device. The connector to the auxiliary device can be a cable attachment for connecting another device to the AR goggles or a port for a permanently attached cable. In one embodiment, this other device is a mobile phone that runs an application for controlling the AR goggles. For example, the wearer can connect the AR goggles to the mobile phone and pass the images and sensor data captured by the AR goggles to the mobile phone for processing, or the mobile phone can provide AR data to display AR content on the optical display 412. In another embodiment, the other device is an external battery pack that powers the AR goggles. In this embodiment, the external battery can reduce the weight of the AR goggles by replacing the goggles' battery, or can extend the battery life of the AR goggles by providing additional power. In any of the embodiments, there is one or more connectors to the auxiliary device, and the AR goggles can be connected to multiple devices at once. In one embodiment, a VCP or outdoor game controller is connected to the AR goggles via a cable plugged into the socket.

[0059] The AR device 400 may also include one or more batteries for powering other components. In addition to, or instead of, this, power can be supplied from an auxiliary device via a connector. The battery can be made in a custom shape that fits snugly behind the user's head, conforming to the rounded shape of the back portion.

[0060] The surface of the rear portion 420 of the AR device 400 may have an identifier such as a QR code (registered trademark) or other visual encoder. The identifier can be used to identify devices in the same environment. For example, another compact AR device or client device captures an image of the identifier on the AR device 400, processes the image, and creates an identity of the user wearing the AR device 400 within the image. In this way, the devices interact in a multiplayer game setting by identifying other players of the game nearby and establishing contact with them. In some embodiments, the connection between devices can be further established through a message stream or sharing of positions between devices.

[0061] FIG. 5A shows an exemplary AR device 500 with a flexible band according to one embodiment. As shown in FIG. 5A, the front portion 510 and the rear portion 520 are connected by a flexible band. In some embodiments, the AR device 500 may also include an adjustment area 532 where the flexible band 530 can be pulled to increase or decrease their length.

[0062] FIG. 5B shows an exemplary AR device 500 with a non - flexible band according to one embodiment. As shown in FIG. 5A, the front portion 510 and the rear portion 520 are connected by a non - flexible band. In some embodiments, the AR device 500 includes an adjuster 534 that slides the non - flexible band 530 to a desired length to fit the user's head.

[0063] In some embodiments, the band 530 (e.g., flexible or non - flexible) may house a cable that electronically couples components of the front portion 510 to components of the rear portion 520. In some embodiments, the cable may be braided to allow the cable to be safely bent and manipulated. In some embodiments, 30 - 50 cables (e.g., coaxial cables) may pass through each band 530 to connect the front and rear components while keeping heat dissipation at a minimal level. In some embodiments, the band 530 may be customizable for aesthetic value. For example, the band 530 may be interchangeable and the user may be able to switch the color or pattern of the band 530 based on aesthetic preferences.

[0064] Figure 6 shows an exemplary AR device 600 with a replaceable forehead pad according to one embodiment. As shown in Figure 6, the AR device 600 may include a forehead pad 610 to provide stability and comfort for the device. For example, instead of the AR goggles mainly resting on the user's nasal bridge or cheekbones, the forehead pad 610 may distribute the weight of the front portion component across the user's entire forehead. To accommodate different head shapes, the forehead pad 610 may be replaceable and customizable to a specific shape to fit the user. Figure 6 shows two exemplary embodiments of the forehead pad 610, one embodiment being tall (e.g., tall forehead pad 612) and the other embodiment being short (e.g., short forehead pad 614). Further customization of the forehead pad 610 is possible. For example, the width of the forehead pad 610 may vary according to the user's preference. In some embodiments, the material of the forehead pad 610 may be customizable. For example, the forehead pad 610 may be made of plastic or may be customizable with a terry cloth cover that functions as a sweatband. In some embodiments, the forehead pad 610 is coupled to the AR goggles via a pair of connectors. In one embodiment, the forehead pad 610 has male connectors 622 that fit into female connectors 624 on both sides of the AR goggles. In some embodiments, the AR device 600 may include additional connection portions. In some embodiments, the forehead pad 610 is replaceable with a visor form having a light-blocking flange that extends forward beyond the optical display, giving the AR device 600 a visor form factor.

[0065] FIG. 7 shows an exemplary AR device 700 folded for storage, according to one embodiment. As shown in FIG. 7, the AR device 700 has a flexible sideband 730, and the AR device 700 can be folded / collapsed to be compactly stored when not in use. Due to the flexible side, the rear portion 720 can contact the front portion 710. In some embodiments, the folded AR device 700 can be in a shape that fits within a conventional sunglass case for ease of carrying. In the illustrated embodiment, the rear portion 720 is curved to fit snugly against the surface of the front portion 710 that rests on the user's forehead.

[0066] FIG. 8 shows an exemplary AR device 800 having a visor form factor, according to one embodiment. As shown in FIG. 8, the AR device 800 includes a front portion 810, a rear portion 820, and a band 830 connecting the two portions. The front portion 810 of the AR device 800 includes a visor rim, rather than the goggle form factor shown in FIGS. 4-7. In some embodiments, the visor rim can be used as a heat sink, allowing more components to be placed in the front portion 810, reducing the rear portion 820, or providing more space for a larger battery.

[0067] Exemplary Computing Device Architecture FIG. 9 is a block diagram showing exemplary components of a machine that can read instructions from a machine-readable medium and execute them with a processor. Specifically, FIG. 9 shows a graphical representation of a machine in the form of an example of a computer system 900. The computer system 900 is associated with the components (or modules) of the game server 110 or the client device 110 and can be used to execute instructions 924 (e.g., program code or software, etc.) for causing the machine to execute any one or more of the methodologies (or processes) described herein, including those described.

[0068] The machine can be a server computer, a client computer, a personal computer (PC), a tablet PC, a set-top box (STB), a smartphone, a network router, a switch or bridge, a cell phone tower, or any machine capable of executing instructions 924 (sequential or otherwise) that specify actions to be performed by the machine. Further, although only a single machine is illustrated, the term "machine" shall also include any collection of machines that individually or jointly execute instructions 924 to perform any one or more of the methodologies described herein.

[0069] Exemplary computer system 900 includes one or more processing units (generally, one or more processors 902). Processor 902 can be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a controller, a state machine, one or more application specific integrated circuits (ASICs), one or more radio frequency integrated circuits (RFICs), or any combination thereof. Reference to processor 902 herein can refer to a single processor or multiple processors. Computer system 900 also includes main memory 904. The computer system can include storage unit 916. Processor 902, memory 904, and storage unit 916 communicate via bus 908.

[0070] Furthermore, computer system 900 can include static memory 906, display driver 910 (e.g., for driving a plasma display panel (PDP), liquid crystal display (LCD), or projector). Computer system 900 can also include alphanumeric input device 914 (e.g., a keyboard), cursor control device 914 (e.g., a mouse, trackball, joystick, motion sensor, or other pointing device), signal generation device 918 (e.g., a speaker), and network interface device 920, which are also configured to communicate via bus 908.

[0071] Storage unit 916 includes machine-readable medium 922 storing instructions 924 (e.g., software) embodying any one or more of the methodologies or functions described herein. Instructions 924 may also reside, completely or at least partially, within main memory 904 or within processor 902 (e.g., within a cache memory of the processor) while being executed by computer system 900, and main memory 904 and processor 902 also constitute machine-readable media. Instructions 924 may be transmitted or received via network interface device 920 over network 970 (e.g., network 105).

[0072] Machine-readable medium 922 is shown as a single medium in the exemplary embodiment, but the term "machine-readable medium" should be interpreted to include a single medium or multiple media (e.g., a centralized or distributed database, or associated caches and servers) that can store instructions 924. The term "machine-readable medium" should also be interpreted to include any medium that can store instructions 924 executable by a machine to cause the machine to perform any one or more of the methodologies disclosed herein. The term "machine-readable medium" includes, but is not limited to, data repositories in the form of solid state memories, optical media, and magnetic media.

[0073] Additional Considerations

[0074] In part of the above description, the embodiments are described from the perspective of the processes or operations of the algorithms. These descriptions and expressions of the algorithms are often used by experts in computing technology to effectively communicate the content of their work to other experts. Although these operations are described functionally, it is understood that computationally or logically, they are implemented by a computer program including instructions executed by a processor or equivalent electrical circuit, microcode, etc. Further, it is proven that it may be convenient to refer to these arrangements of functional operations as modules without loss of generality.

[0075] Any reference to "one embodiment" or "an embodiment" as used herein means that the particular element, feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearances of the phrase "in one embodiment" in various places in this specification do not necessarily all refer to the same embodiment. Similarly, the use of "a" or "an" before an element or component is merely for convenience. It should be understood that the foregoing description means that there is one or more elements or components unless the contrary is clearly indicated by the context.

[0076] When a value is described as "approximate" or "substantially" (or derivatives thereof), such a value should be construed as being accurate to ±10% unless another meaning is clear from the context. For example, "approximately 10" should be understood to mean "in the range of 9 to 11".

[0077] As used herein, the terms "comprises," "comprising," "includes," "including," "has," "having" or any variation thereof are intended to cover a non-exclusive inclusion. For example, a process, method, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements, but may include other elements not expressly listed or inherent to the above process, method, article, or apparatus. Further, unless expressly stated to the contrary, "or" refers to an inclusive "or" and not to an exclusive "or". For example, the condition A or B is satisfied by any one of the following: A is true (or present) and B is false (or absent), A is false (or absent) and B is true (or present), and both A and B are true (or present).

[0078] While the subject matter of the present invention has been described in detail with respect to specific exemplary embodiments and methods thereof, it will be understood by those of ordinary skill in the art that, having understood the foregoing, they can readily make alterations, modifications, and equivalents to such embodiments. Accordingly, the scope of the present disclosure is illustrative and not restrictive, and the disclosure is not intended to exclude modifications, variations, or additions to the subject matter as would be readily apparent to those of ordinary skill in the art.

Claims

1. A front portion comprising an optical display configured to output image light to a user's eyes, A rear portion arranged with the front portion to balance the weight of the HMD, One or more bands connecting the front portion and the rear portion such that the two portions rest on both sides of the user's head, A head-mounted device (HMD) comprising the above.

2. The front portion further comprises one or more of a speaker, a microphone, a camera, or a sensor, The head-mounted device (HMD) according to Claim 1.

3. The rear portion comprises one or more batteries or processing units, The head-mounted device (HMD) according to Claim 1.

4. The optical display is transparent such that light from the environment of the HMD can pass through the optical display, and the user can view the image light and the environment simultaneously, The head-mounted device (HMD) according to Claim 1.

5. The HMD further comprises one or more vision correction glasses or sunglasses incorporated therein, The head-mounted device (HMD) according to Claim 1.

6. The rear portion includes an identifier for identifying the HMD and establishes a connection with a second HMD, The head-mounted device (HMD) according to Claim 1.

7. The one or more bands accommodate a cable that electronically couples the front portion and the rear portion, The head-mounted device (HMD) according to Claim 1.

8. The HMD further comprises a replaceable forehead pad attachment, The head-mounted device (HMD) according to Claim 1.

9. The forehead pad attachment is configured to distribute the weight of the front portion over the entire forehead of the user, The head-mounted device (HMD) according to Claim 8.

10. The HMD is foldable, The head-mounted device (HMD) according to Claim 1.

11. The optical display outputs image light to generate images of an augmented reality (AR) game, The head-mounted device (HMD) according to Claim 1.

12. A head-mounted device (HMD), A front portion comprising an optical display configured to output image light for generating an image for the user's eyes for an augmented reality (AR) game, A rear portion arranged with the front portion to balance the weight of the HMD, One or more bands connecting the front portion and the rear portion such that the two portions rest on both sides of the user's head, A head-mounted device (HMD) comprising: A game controller configured to receive an input from the user and transmit the input to the HMD, A game device comprising: **Claim 13** The front portion further comprises one or more of a speaker, a microphone, a camera, and a sensor, The device according to claim 12. **Claim 14** The rear portion comprises one or more of a battery and a processing unit, The device according to claim 12. **Claim 15** The optical display is transparent such that light from the environment of the HMD can pass through the optical display, and the user can view the image light and the environment simultaneously, The device according to claim 12. **Claim 16** The HMD further comprises one or more pairs of vision correction glasses or sunglasses incorporated into the HMD, The device according to claim 12. **Claim 17** The rear portion includes an identifier for identifying the HMD and establishing a connection with a second HMD, The device according to claim 12. **Claim 18** The one or more bands accommodate a cable that electronically couples the front portion and the rear portion, The device according to claim 12. **Claim 19** The HMD comprises a replaceable forehead pad attachment configured to distribute the weight of the front portion over the entire forehead of the user, The device according to claim 12. **Claim 20** The head-mounted device (HMD) further comprises a versatile computing pack (VCP) that provides power, memory, and power storage to the HMD, The device according to claim 12.

Images