Mapping traversable space in a scene using a 3D mesh

Abstract

The parallel reality game uses a virtual game board having tiles arranged across an identified traversable space that corresponds to a flat region of the scene. The game board generation module receives one or more images of the scene captured by a camera of the mobile device. The game board generation module obtains a terrain mesh of the scene based on the received one or more images. The game board generation module then identifies a traversable space in the scene based on the obtained terrain mesh. The game board generation module determines a location for each of a set of polygon tiles in the identified traversable space. The game board generation module also enables queries to identify portions of the game board that meet one or more provided criteria.

Description

[Technical field]

[0001] The present subject matter relates generally to augmented reality, and more particularly to building game boards that correspond to real-world geography. [Background technology]

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 316,325, filed March 3, 2022, and U.S. Provisional Patent Application No. 63 / 345,420, filed May 24, 2022, which are incorporated herein by reference.

[0003] Location-based games use the real world as their geography. Parallel reality games are a type of location-based game that uses a virtual world that is parallel to the geography of the real world. The parallel virtual world may span the entire real world or may span a bounded area around the mobile device used to play the game. Parallel reality games that are played in a bounded area around the mobile device may set the bounded area by asking the player to manually define the bounded area. For example, the game may ask the player to walk around the desired boundary of the game so that the game can identify where to place the edges of the play area. However, it would be advantageous to allow parallel reality games to automatically detect and set the play area. [Brief description of the drawings]

[0004] [Figure 1] FIG. 1 illustrates a networked computing environment in accordance with one or more embodiments. [Diagram 2] FIG. 1 illustrates a representation of a virtual world having a parallel geography to the real world, according to one or more embodiments. [Diagram 3] 1 is a block diagram of a gaming module 135 according to one or more embodiments. [Figure 4]FIG. 1 illustrates an exemplary game interface for a parallel reality game that may be presented on a client device's display as part of an interface between a player and a virtual or parallel reality world, according to one or more embodiments. [Diagram 5] 1 is a flowchart illustrating a process for generating an augmented reality game board according to one or more embodiments. [Figure 6A] FIG. 6 illustrates a process for placing tiles in a walkable space 610 according to one or more embodiments. [Figure 6B] FIG. 6 illustrates a process for placing tiles in a walkable space 610 according to one or more embodiments. [Figure 6C] FIG. 6 illustrates a process for placing tiles in a walkable space 610 according to one or more embodiments. [Figure 7A] FIG. 1 illustrates an exemplary game interface for a parallel reality game, according to one or more embodiments. [Figure 7B] FIG. 1 illustrates an example game interface for a parallel reality game having a game board with optimized tile placement, in accordance with one or more embodiments. [Figure 7C] FIG. 1 illustrates an example game interface for a parallel reality game having a game board with tiles having different surface characteristics, according to one or more embodiments. [Figure 8] FIG. 1 illustrates an example game interface for a parallel reality game with a procedurally generated game board, in accordance with one or more embodiments. [Figure 9] 1 is a flowchart illustrating a method for dynamically generating an augmented reality game board according to one or more embodiments. [Figure 10] FIG. 1 illustrates an example computer system suitable for use in generating an augmented reality game board, according to one or more embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0005] The figures and the following description set forth specific embodiments by way of example only. Those skilled in the art will readily recognize from the following description that alternative embodiments of the structure and method may be employed without departing from the principles described. Reference will now be made to several embodiments, examples of which are illustrated in the accompanying figures.

[0006] Exemplary Location-Based Parallel Reality Gaming System Various embodiments are described in the context of a parallel reality game that includes augmented reality content in a virtual world geography that parallels at least a portion of the real world geography, such that player movements and actions in the real world affect actions in the virtual world and vice versa. With the disclosure provided herein, one skilled in the art will appreciate that the subject matter described is applicable to other situations in which it is desirable to determine a substantially flat, usable surface area in an image. Furthermore, the inherent flexibility of computer-based systems allows for a wide variety of possible configurations, combinations, and divisions of tasks and functionality among components of the system. For example, systems and methods according to aspects of the present disclosure may be implemented using a single computing device or across multiple computing devices (e.g., connected by a computer network).

[0007] FIG. 1 illustrates a networked computing environment 100 according to one or more embodiments. The networked computing environment 100 provides for player interaction in a virtual world having a parallel geography to the real world. In particular, geographic areas in the real world may be directly linked or mapped to corresponding areas in the virtual world. A player may move around in the virtual world by traveling to various geographic locations in the real world. For example, the player's location in the real world may be tracked and used to update the player's location in the virtual world. Typically, the player's location in the real world is determined by finding the location of the client device 110 at which the player is interacting with the virtual world and assuming that the player is in the same (or approximately the same) location. For example, in various embodiments, a player may interact with a virtual element if the player's location in the real world is within a threshold distance (e.g., 10 meters, 20 meters, etc.) of a real-world location that corresponds to the virtual location of the virtual element in the virtual world. For convenience, various embodiments are described with reference to the "location of the player," although one skilled in the art will understand that such reference may refer to the location of the player's client device 110.

[0008] Reference is now made to Figure 2, which illustrates a conceptual diagram of a virtual world 210 parallel to a real world 200 that can serve as a game board for players of a parallel reality game, according to one or more embodiments. As illustrated, the virtual world 210 can include a geography that parallels the geography of the real world 200. In particular, a range of coordinates that defines a geographic area or space in the real world 200 is mapped to a corresponding range of coordinates that defines a virtual space in the virtual world 210. A range of coordinates in the real world 200 may be associated with a town, neighborhood, city, campus, locale, country, continent, the entire globe, or other geographic area. Each geographic coordinate within the geographic coordinate range is mapped to a corresponding coordinate in a virtual space in the virtual world.

[0009] The positions of the players in the virtual world 210 correspond to the positions of the players in the real world 200. For example, player A, who is located at position 212 in the real world 200, has a corresponding position 222 in the virtual world 210. Similarly, player B, who is located at position 214 in the real world, has a corresponding position 224 in the virtual world. As the players move around within a range of geographic coordinates in the real world, they also move around within a range of coordinates that define a virtual space in the virtual world 210. In particular, a positioning system (e.g., a GPS system) associated with a mobile computing device carried by the player may be used to track the position of the player as the player navigates the range of geographic coordinates in the real world. Data associated with the player's position in the real world 200 is used to update the player's position in the corresponding range of coordinates that define a virtual space in the virtual world 210. In this manner, a player can navigate along a continuous track within a range of coordinates defining a virtual space in the virtual world 210 by simply moving between corresponding ranges of geographic coordinates in the real world 200, without having to check in at specific individual locations in the real world 200 or periodically update location information.

[0010] A location-based game may include multiple game objectives that require a player to move and / or interact with various virtual elements and / or objects scattered across various virtual locations in a virtual world. A player may travel to these virtual locations by moving to the corresponding locations of the virtual elements or objects in the real world. For example, a positioning system may continuously track the player's position such that the player continually navigates the parallel virtual world as the player continually navigates the real world. The player may then interact with the various virtual elements and / or objects at a particular location to accomplish or perform one or more game objectives.

[0011] For example, in the game objective, the player interacts with virtual elements 230 located at various virtual locations in the virtual world 210. These virtual elements 230 may be linked to landmarks, geographic locations, or objects 240 in the real world 200. The real world landmarks or objects 240 may be works of art, monuments, buildings, businesses, libraries, museums, or other suitable real world landmarks or objects. The interactions include acquiring some virtual items, claiming ownership of them, using them, using some virtual currency, etc. To acquire these virtual elements 230, the player must travel to the landmarks or geographic locations 240 linked to the virtual elements 230 in the real world and perform the necessary interactions with the virtual elements 230 in the virtual world 210. For example, player A in FIG. 2 may need to travel to the landmarks 240 in the real world 200 to interact with or acquire the virtual elements 230 linked to that particular landmark 240. Interaction with the virtual element 230 may require a real-world action, such as taking a photograph and / or viewing, obtaining, or capturing other information about a landmark or object 240 associated with the virtual element 230.

[0012] A game objective may require a player to use one or more virtual items that are collected by the player in a location-based game. For example, a player may travel through the virtual world 210 in search of virtual items (e.g., weapons, creatures, power-ups, or other items) that may help achieve a game objective. These virtual items may be found or collected by traveling to various locations in the real world 200 or by completing various actions in either the virtual world 210 or the real world 200. In the example shown in FIG. 2, a player uses a virtual item 232 to acquire one or more virtual elements 230. In particular, a player may deploy a virtual item 232 to a location in the virtual world 210 in proximity to or within the virtual element 230. Deploying one or more virtual items 232 in this manner may result in the acquisition of the virtual element 230 for a particular player or for a particular player's team / faction.

[0013] In one implementation, a player may have to collect virtual energy as part of a parallel reality game. As shown in FIG. 2, virtual energy 250 may be scattered at various locations in the virtual world 210. A player may collect virtual energy 250 by traveling to the corresponding location of the virtual energy 250 in the real world 200. The virtual energy 250 may be used to power virtual items and / or to perform various game objectives in the game. A player who loses all virtual energy 250 may be disconnected from the game.

[0014] According to aspects of the present disclosure, a parallel reality game can be a massively multiplayer location-based game in which all participants in the game share the same virtual world. Players can be divided into separate teams or factions and cooperate to achieve one or more game goals, such as gaining or claiming ownership of virtual elements. In this way, a parallel reality game can essentially be a social game that promotes cooperation between players in the game. Players from opposing teams can play against each other during the parallel reality game (or in some cases cooperate to achieve mutual goals). Players may use virtual items to attack or impede the progress of players from opposing teams. In some cases, players are encouraged to meet at real-world locations for cooperative or interactive events in the parallel reality game. In these cases, the game server attempts to ensure that players are actually physically present and are not impersonating each other.

[0015] A parallel reality game may have various features to enhance and facilitate gameplay within the parallel reality game. For example, a player may accumulate virtual currency or another virtual reward (e.g., virtual tokens, virtual points, virtual material resources, etc.) that may be used throughout the game (e.g., to purchase in-game items, exchange for other items, craft items, etc.). A player may go through various levels as he completes one or more game objectives and gains in-game experience. In some embodiments, players may communicate with each other through one or more communication interfaces provided in the game. Players may also obtain enhanced "powers" or virtual items that may be used to accomplish game objectives within the game. Using the disclosure provided herein, one skilled in the art would understand that various other game features may be included in a parallel reality game without departing from the scope of the present disclosure.

[0016] Referring again to FIG. 1, the networked computing environment 100 uses a client-server architecture, where a game server 120 communicates with the client devices 110 over a network 105 to provide a parallel reality game to players at the client devices 110. The networked computing environment 100 may also include other external systems, such as sponsor / advertiser systems or business systems. Although only one client device 110 is shown in FIG. 1, any number of clients 110 or other external systems may be connected to the game server 120 over the network 105. Additionally, the networked computing environment 100 may include different or additional elements, and functionality may be distributed between the client devices 110 and the server 120 in a manner different from that described below.

[0017] The client device 110 may be any portable computing device that can be used by a player to interface with the game server 120. For example, the client device 110 may be a wireless device, a personal digital assistant (PDA), a portable gaming device, a mobile phone, a smartphone, a tablet, a navigation system, a handheld GPS system, a wearable computing device, a display having one or more processors, or other such devices. In another example, the client device 110 includes a conventional computer system such as a desktop computer or a laptop computer. Furthermore, the client device 110 may be a vehicle equipped with a computing device. In short, the client device 110 may be any computing device or system that can enable a player to interact with the game server 120. As a computing device, the client device 110 may include one or more processors and one or more computer-readable storage media. The computer-readable storage media may store instructions that cause the processor to perform operations. The client device 110 is preferably a portable computing device, such as a smartphone or tablet, that can be easily carried or otherwise transported with the player.

[0018] The client devices 110 communicate with the game server 120, which provides the game server 120 with sensory data of the physical environment. The client devices 110 include a camera assembly 125 that captures image data in two dimensions of a scene in the physical environment in which the client devices 110 reside. In the embodiment shown in FIG. 1, each client device 110 includes software components such as a gaming module 135 and a positioning module 140. The client devices 110 may include various other input / output devices for receiving information from a player and / or providing information to a player. Exemplary input / output devices include a display screen, a touch screen, a touch pad, data entry keys, a speaker, and a microphone suitable for voice recognition. The client devices 110 may include various other sensors for recording data from the client devices 110, including, but not limited to, motion sensors, accelerometers, gyroscopes, other inertial measurement units (IMUs), barometers, positioning systems, thermometers, light sensors, and the like. The client devices 110 may further include a network interface for providing communication over the network 105. A network interface may include any suitable components for interfacing with one or more networks, including, for example, a transmitter, a receiver, a port, a controller, an antenna, or other suitable components.

[0019] The camera assembly 125 captures image data of a scene of an environment in which the client device 110 resides. The camera assembly 125 may utilize a variety of photosensors having various color capture ranges at various capture rates. The camera assembly 125 may include a wide-angle lens or a telephoto lens. The camera assembly 125 may be configured to capture a single image or video as the image data. In addition, the orientation of the camera assembly 125 may be parallel to the ground with the camera assembly 125 pointed toward the horizon. The camera assembly 125 captures and shares the image data with a computing device on the client device 110. The image data may be augmented with metadata that describes other details of the image data, including sensory data (e.g., temperature, brightness of the environment, etc.) or capture data (e.g., exposure, warmth, shutter speed, focal length, capture time, etc.). The camera assembly 125 may include one or more cameras capable of capturing image data. In one example, the camera assembly 125 includes one camera and is configured to capture monocular image data. In another example, camera assembly 125 includes two cameras and is configured to capture stereoscopic image data. In various other implementations, camera assembly 125 includes multiple cameras, each configured to capture image data.

[0020] The gaming module 135 provides an interface to players for participating in the parallel reality game. The game server 120 transmits game data to the client device 110 over the network 105 for use by the game module 135 in the client device 110 to provide a local version of the game to players at locations remote from the game server 120. The game server 120 may include a network interface for providing communications over the network 105. The network interface may include any suitable components for interfacing with one or more networks, including, for example, a transmitter, a receiver, a port, a controller, an antenna, or other suitable components.

[0021] The gaming module 135 executed by the client device 110 provides an interface between the player and the parallel reality game. The gaming module 135 can present a user interface on a display device associated with the client device 110 that displays a virtual world associated with the game (e.g., renders images in the virtual world) and allows the user to interact in the virtual world to perform various game objectives. In some other embodiments, the gaming module 135 presents image data from the real world (e.g., captured by the camera assembly 125) augmented with virtual elements from the parallel reality game. In these embodiments, the gaming module 135 can generate and / or adjust the virtual content according to other information received from other components of the client device 110. For example, the gaming module 135 can adjust the virtual objects displayed on the user interface according to a depth map of the scene captured in the image data. In other embodiments, such as when a headset is used, the gaming module 135 can display only the virtual elements on a transparent or semi-transparent display such that the user perceives the virtual elements as overlaid on the real world view.

[0022] The gaming module 135 may also control various other outputs to allow the player to interact with the game without the player needing to see a display screen. For example, the gaming module 135 may control various sounds, vibrations, or other notifications that allow the player to play the game without seeing a display screen. The gaming module 135 may access game data received from the game server 120 to provide an accurate representation of the game to the user. The gaming module 135 may receive and process player input and provide updates to the game server 120 via the network 105. The gaming module 135 may generate and / or adjust game content displayed by the client device 110. For example, the gaming module 135 may generate virtual elements based on a comparison of the depth information to one or more terrain meshes that represent the real-world environment surrounding the client device 110.

[0023] The positioning module 140 can be any device or circuitry for monitoring the location of the client device 110. For example, the positioning module 140 can determine an actual or relative location by using a satellite navigation positioning system (e.g., GPS system, Galileo positioning system, Global Navigation Satellite System (GLONASS), Beidou satellite navigation and positioning system), an inertial navigation system, a dead reckoning system, based on an IP address, by using triangulation and / or proximity to cellular towers or Wi-Fi hotspots, and / or other suitable techniques for determining location. The positioning module 140 can further include various other sensors that can help precisely locate the client device 110 location.

[0024] As the player moves around with the client device 110 in the real world, the positioning module 140 tracks the player's location and provides the player's location information to the gaming module 135. The gaming module 135 updates the player's location in the virtual world associated with the game based on the player's actual location in the real world. Thus, the player can interact with the virtual world simply by carrying or transporting the client device 110 in the real world. In particular, the player's location in the virtual world can correspond to the player's location in the real world. The gaming module 135 can provide the player location information to the game server 120 via the network 105. In response, the game server 120 can implement various techniques to verify the location of the client device 110 to prevent cheaters from spoofing the location of the client device 110. It should be understood that location information associated with a player will only be utilized if permission is given and after the player has been informed that their location information will be accessed and how the location information will be utilized in the context of the game (e.g., to update the player's position in the virtual world). Additionally, location information associated with a player will be stored and maintained in a manner that protects the player's privacy.

[0025] The game server 120 may be any computing device and may include one or more processors and one or more computer-readable storage media. The computer-readable storage media may store instructions that cause the processor to perform operations. The game server 120 may include or be in communication with a game database 115. The game database 115 stores game data used in the parallel reality game that is serviced or provided to the clients 120 over the network 105.

[0026] The game data stored in the game database 115 may include (1) data associated with a virtual world in a parallel reality game (e.g., image data used to render the virtual world on a display device, geographic coordinates of locations in the virtual world, etc.); (2) data associated with a player of the parallel reality game (e.g., player profile including, but not limited to, player information, player experience level, player currency, current player location in the virtual / real world, player energy level, player preferences, team information, faction information, etc.); (3) data associated with game goals (e.g., data associated with current game goals, game goal status, past game goals, future game goals, desired game goals, etc.); and (4) data associated with virtual elements in the virtual world (e.g., virtual elements (5) data associated with real world objects, landmarks, locations linked to virtual world elements (e.g., real world object / landmark location, real world object / landmark description, virtual element linked to real world object association, etc.), (6) game status (e.g., current number of players, current status of game objectives, player leaderboard, etc.), (7) data associated with player actions / inputs (e.g., current player positions, past player positions, player movements, player inputs, player queries, player communications, etc.), and (8) any other data used, related to, or obtained during implementation of the parallel reality game. The game data stored in the game database 115 may be populated either offline or in real time by a system administrator and / or by data received from users / players of the system 100, such as from client devices 110 over the network 105.

[0027] The game server 120 may be configured to receive requests for game data from the client devices 110 (e.g., via remote procedure calls (RPCs)) and respond to those requests over the network 105. For example, the game server 120 may encode the game data into one or more data files and provide the data files to the client devices 110. Additionally, the game server 120 may be configured to receive game data (e.g., player positions, player actions, player inputs, etc.) from the client devices 110 over the network 105. For example, the client devices 110 may be configured to periodically send player inputs and other updates to the game server 120, which uses them to update the game data in the game database 115 to reflect any changed conditions for the game.

[0028] In the illustrated embodiment, the server 120 includes a universal game module 145, a commercial game module 150, a data collection module 155, and an event module 160. As described above, the game server 120 interacts with a game database 115, which may be part of the game server 120 or may be accessed remotely (e.g., the game database 115 may be a distributed database accessed via the network 105). In other embodiments, the game server 120 includes different and / or additional elements. For example, the game server 120 may include a terrain mesh management module 320, a semantic segmentation module 340, and / or a game board generation module 350. Furthermore, functionality may be distributed among the elements in a manner different than described. For example, the game database 115 may be integrated into the game server 120.

[0029] The universal game module 145, in embodiments in which it is included, hosts the parallel reality game for all players and serves as an authoritative source for the current status of the parallel reality game for all players. As a host, the universal game module 145 generates game content for presentation to players, for example, via their respective client devices 110. In hosting the parallel reality game, the universal game module 145 may access the game database 115 to retrieve and / or store game data. The universal game module 145 also receives game data (e.g., depth information, player input, player position, player actions, landmark information, etc.) from the client devices 110 and incorporates the received game data into the overall parallel reality game for all players of the parallel reality game. The universal game module 145 may also manage the distribution of game data to the client devices 110 over the network 105. The universal game module 145 may also manage security aspects of the client device 110, including, but not limited to, securing connections between the client device 110 and the game server 120, establishing connections between various client devices 110, and verifying the location of various client devices 110.

[0030] Commercial games module 150, in embodiments where it is included, may be separate from universal games module 145 or may be part of universal games module 145. Commercial games module 150 may manage the inclusion of various game features in the parallel reality game that are linked to commercial activities in the real world. For example, commercial games module 150 may receive requests over network 105 (via a network interface) from external systems, such as sponsors / advertisers, businesses, or other entities, to include game features linked to commercial activities in the parallel reality game. Commercial games module 150 may arrange for the inclusion of these game features in the parallel reality game.

[0031] The game server 120 may further include a data collection module 155. The data collection module 155, in embodiments where it is included, may be separate from the universal game module 145 or may be part of the universal game module 145. The data collection module 155 may manage the inclusion of various game features in the parallel reality game linked with the data collection activities in the real world. For example, the data collection module 155 may modify the game data stored in the game database 115 to include game features linked with the data collection activities in the parallel reality game. The data collection module 155 may also analyze data collected by players pursuant to data collection activities and provide the data for access by various platforms.

[0032] The event module 160 manages player access to events in a parallel reality game. While the term "event" is used for convenience, it should be understood that the term does not necessarily refer to a specific event at a specific location or time. Rather, it may refer to any offering of access-controlled game content in which one or more access criteria are used to determine whether a player may access that content. Such content may be part of a larger parallel reality game that includes game content with little or no access control, or may be a stand-alone access-controlled parallel reality game.

[0033] The network 105 can be any type of communications network, such as a local area network (e.g., an intranet, etc.), a wide area network (e.g., the Internet, etc.), or some combination thereof. The network can also include direct connections between the client devices 110 and the game server 120. In general, communications between the game server 120 and the client devices 110 can be carried over a network interface using any type of wired and / or wireless connection, using a variety of communications protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML, JSON), and / or protection schemes (e.g., VPN, Secure HTTP, SSL).

[0034] The described techniques discussed herein refer to servers, databases, software applications, and other computer-based systems, as well as actions taken and information sent to and from such systems. Those skilled in the art will recognize that the inherent flexibility of computer-based systems allows for a wide variety of possible configurations, combinations, and divisions of tasks and functionality among components. For example, the server processes discussed herein may be implemented using a single server or multiple servers working in combination. Databases and applications may be implemented on a single system or distributed across multiple systems. Distributed components may operate sequentially or in parallel.

[0035] 3 illustrates one embodiment of the gaming module 135. In the illustrated embodiment, the gaming module 135 includes a terrain mesh management module 320, a local terrain mesh store 325, a semantic segmentation module 340, and a game board generation module 350. In other embodiments, the gaming module 135 includes additional or different components. Furthermore, the described functionality may be distributed differently among the components. For example, one or more components of the gaming module 135 illustrated in FIG. 3 may instead be included in the gaming server 120.

[0036] The terrain mesh management module 320 manages the terrain meshes of the gaming module 135. A terrain mesh is a three-dimensional representation of an environment in the real world and may include a combination of geometry (e.g., a polygon mesh), color (e.g., RGB data), texture (e.g., texture maps, bump maps, etc.), other material properties (reflectivity, friction, density, etc.), and other information that describes the real-world environment. Alternatively, the terrain mesh may be purely geometric, and other properties (e.g., segmentation data) may be stored separately and mapped to the polygons of the terrain mesh.

[0037] Depending on the embodiment, the terrain meshes may be generated (e.g., by the client device 110 or the game server 120) using processes requiring varying degrees of computational complexity, each of which may create terrain meshes that may be displayed with varying degrees of visual accuracy (e.g., similarity to the real-world environment). For example, the geometry of the terrain mesh may be represented using a high-density polygon mesh (e.g., high polygon count) that achieves a one-to-one or near-one correspondence with the real-world environment. Additionally, the terrain meshes used by the client device 110 may include photorealistic or near-photorealistic geometry and textures that represent the real-world environment.

[0038] In some embodiments, the terrain mesh management module 320 uses scan information describing the real-world environment surrounding the client device 110 (e.g., provided by the camera assembly 125 or the sensor module) to generate a terrain mesh, or uses the location of the client device 110 to retrieve a previously generated terrain mesh for the real-world environment. In embodiments in which the terrain mesh management module 320 generates a mesh, the terrain mesh management module 320 uses scan information captured from scanning the real-world environment using one or more sensors managed by the sensor module 310. In embodiments in which the terrain mesh management module 320 retrieves a previously generated terrain mesh, the terrain management module 310 uses the geographic location of the client device 110, the scan information, or other information to identify and retrieve the previously generated terrain mesh (e.g., from the local terrain mesh database 325 or from the game server 120).

[0039] The terrain mesh management module 320 can locally store terrain meshes generated by the client device 110 or obtained in other ways in a local terrain mesh store 325. The client device 110 can use the terrain meshes obtained by the terrain mesh management module 320 for associated processes of the client device 110. For example, the terrain mesh management module 320 can determine that a terrain mesh of a real-world environment was previously stored locally based on a scan of the real-world environment or the geographic location of the client device 110. In some embodiments, the terrain mesh management module 320 can provide the terrain mesh to the game server 120 for storage. In some embodiments, the terrain mesh management module 320 can obtain a terrain mesh or other information describing a real-world environment from the game server 120, such as a terrain mesh stored on the game server 120 that corresponds to a real-world environment in which the client device 110 is located.

[0040] In some embodiments, the terrain mesh management module 320 coordinates the collection of scan information about the real-world environment using the client device 110 by a player associated with the client device 110. For example, the terrain mesh management module 320 can provide a user interface for display on the client device 110 that displays scan-related information (i.e., a scan interface), such as an image of the real-world environment captured by the camera assembly 125. The scan interface can include one or more interactable objects (e.g., virtual buttons) configured to control a state of the scan process, such as an interactable object to start the scan, pause the scan, end the scan (e.g., to generate a terrain mesh using the collected scan-related information), etc. The scan interface may further include various visualizations of the scan-related information, such as a visualization of a portion of the terrain mesh (e.g., the geometry of the terrain mesh) and a visualization of depth information generated during the scan. Additionally, the scan interface can guide the player through the scan process, such as by displaying messages or visual indicators that describe portions of the real-world environment to scan and to what extent the real-world environment to scan. In some embodiments, multiple client devices 110 simultaneously coordinate the collection of associated terrain meshes by multiple respective users. For example, the game server 120 may instruct multiple client devices to collect terrain meshes. The terrain meshes collected by the multiple client devices 110 may be combined into a single terrain mesh. Thus, the multiple client devices 110 may generate respective terrain meshes to collectively map the real-world environment.

[0041] In some embodiments, the terrain mesh management module 320 generates the terrain mesh using one or more mesh generation techniques. In particular, the terrain mesh generation module 320 can perform one or more mesh generation techniques using the scan information. The scan information can be obtained directly by the client device 110 (e.g., one or more images captured by the camera assembly 125) or retrieved from other devices or remote systems (e.g., the game server 120). The terrain mesh management module 320 can obtain the scan information or generate the terrain mesh using various computer vision techniques, such as geometric computer vision techniques or machine learning based computer vision techniques. For example, using computer vision techniques, the terrain mesh management module 320 can determine depth information (e.g., point clouds or dense depth images) that describe the real-world environment. Furthermore, the terrain mesh management module 320 can process the scan information using various mesh generation techniques, such as Delaunay triangulation, Rupert algorithm, Advancing Front algorithm, Poisson reconstruction, etc.

[0042] In some embodiments, the terrain mesh generation process used by the terrain mesh management module 320 generates the terrain mesh by combining new scan information (e.g., collected by the client device 110) with previously collected scan information (e.g., stored on the client device 110 or gaming server 120). In some embodiments, the terrain mesh management module 320 generates the terrain mesh in real-time or near real-time. For example, the terrain mesh management module 320 may generate a terrain mesh corresponding to a portion of the real-world environment within milliseconds or seconds after the client device 110 captures or receives scan information of the real-world environment.

[0043] In some embodiments, the terrain mesh management module 320 determines location information of the real-world environment in which the client device 110 is located. For example, the terrain mesh management module 320 can determine the geographic location of the real-world environment based on the geographic location of the client device 110. Alternatively or additionally, the terrain mesh management module 320 may determine the location of the client device 110 using data captured by one or more sensors and comparing the captured data to previously captured sensor data for the real-world environment. For example, the terrain mesh management module 320 may use various image recognition techniques to predict the location of the client device 110 from one or more images captured by a camera of the client device 110.

[0044] In embodiments, the terrain mesh management module 320 uses the location information describing the real-world environment to determine whether a previously generated terrain mesh for the real-world environment is stored locally or on the game server 120. For example, a terrain mesh stored in the local terrain mesh store 325 or on the game server 120 may be stored in association with a geographic location and other location-related metadata (e.g., a description of the location, such as an address or location name). In some embodiments, the terrain mesh management module 320 uses the location information describing the real-world environment to retrieve other data associated with the real-world location, such as scan information stored by the client device 110 or the game server 120.

[0045] When the terrain mesh management module 320 requests a terrain mesh from the game server 120 based on the location information, the game server 120 can determine whether the client device 110 is authorized to access some or all of the stored terrain meshes associated with the location information. For example, some terrain meshes stored by the game server 120 that represent public environments (e.g., parks, entertainment venues, etc.) may be publicly accessible to any of the client devices 110. Other terrain meshes stored on the game server 120, such as terrain meshes that represent private environments (e.g., a player's home) or terrain meshes designated as private by the client device 110 when providing the terrain mesh to the game server 120, may be accessible only by authorized client devices 110.

[0046] In some embodiments, the terrain mesh management module 320 dynamically combines some or all of multiple terrain meshes for the associated process of the client device 110. In these embodiments, the terrain mesh management module 320 can combine terrain meshes obtained from one or more sources, such as those generated by the client device 110, retrieved from a local terrain mesh store 325, or retrieved from the game server 120. For example, the terrain mesh management module 320 can generate a terrain mesh representing an indoor space, such as a house in which the client device 110 is located, by retrieving one or more previously generated terrain meshes representing a first portion of the indoor space (e.g., living room, dining room, etc.), generating a new terrain mesh for a second portion of the indoor space, and generating one or more other terrain meshes representing portions of the indoor space. The terrain meshes retrieved (e.g., for combination) by the terrain mesh management module 320 can be generated simultaneously or within a time interval by the client device 110 or other client devices 110.

[0047] The terrain mesh management module 320 may combine (e.g., stitch) one or more retrieved or generated terrain meshes and provide the combined terrain mesh to other components of the gaming module 135 for use in displaying the AR content. The combined terrain mesh or meshes may be generated entirely by the client device 110 or crowd-sourced from multiple client devices 110, such as via the network 130. In some embodiments, the terrain mesh management module 320 generates or retrieves terrain meshes in response to changes in the location of the client device 110. For example, the terrain mesh management module 320 may retrieve or generate a terrain mesh representing a room of an indoor space after determining that the client device 110 has entered the room.

[0048] The local terrain mesh store 325 includes one or more computer-readable media configured to store information describing a terrain mesh. The information stored by the local terrain mesh store 325 may be retrieved or generated by the terrain mesh management module 320, or otherwise obtained or determined by the client device 110. The local terrain mesh data store 325 may additionally or alternatively store other information associated with and describing the real-world environment, such as scan or location information obtained by the client device 110.

[0049] The semantic segmentation module 340 uses the semantic segmentation model 345 to match regions of the image with objects or semantics that the semantic segmentation model 345 is trained to recognize. Once trained, the semantic segmentation model 345 is configured to predict characteristics associated with an object represented by one or more pixels of the image. For example, the semantic segmentation model 345 is a machine learning model trained to recognize a type of surface of an object represented in an image based on a training dataset having images of various surfaces with different characteristics. In some embodiments, the semantic segmentation model 345 is trained to recognize a material associated with a surface. For example, the semantic segmentation model 345 is trained to recognize whether a surface is made of concrete, wood, grass, water, snow, etc.

[0050] In some embodiments, the semantic segmentation module 340 receives the terrain mesh from the terrain mesh management module 320 and predicts one or more properties for each polygon cell in the terrain mesh. For example, the semantic segmentation module 340 predicts a material associated with each polygon cell of the received terrain mesh. The semantic segmentation module 340 may predict the properties associated with each polygon cell using the images used to generate the terrain mesh. For example, the semantic segmentation module 340 may additionally receive one or more images used by the terrain mesh management module 320 to generate the terrain mesh and predict the properties associated with each polygon cell in the terrain mesh.

[0051] The game board generation module 350 receives a terrain mesh and determines a placement of a set of tiles within the terrain mesh. In some embodiments, the game board generation module 350 places polygon tiles within the received terrain mesh. The polygon tiles placed by the game board generation module 350 may all be the same shape and size. For example, the game board generation module 350 places square or hexagonal tiles with set dimensions within the received terrain mesh.

[0052] In some embodiments, the game board generation module 350 identifies traversable spaces within the received mesh and determines the placement of polygon tiles within the identified traversable spaces. A detailed description of an example process for determining the placement of polygon tiles within the traversable spaces is provided below in connection with FIG.

[0053] The gaming module 135 can generate AR objects that interact with (e.g., are positioned by or move according to) a terrain mesh of the real-world environment surrounding the client device 110 and / or tiles placed by the game board generation module 350. In some embodiments, the display module 360 ​​simulates a custom view of the real-world environment by providing an AR interface for display by the client device 110. The AR interface can include virtual objects in a three-dimensional space, and the location of the virtual objects (e.g., AR objects) in the three-dimensional space is determined using the generated terrain mesh of the three-dimensional space. For example, the AR objects can be displayed interacting with real-world objects (e.g., tables, chairs, floors, ceilings, walls, etc.) represented by one or more terrain meshes. The display module 360 ​​can display the AR objects placed over an image captured from the camera assembly 125, or can display only the AR objects.

[0054] In some embodiments, the AR interface is associated with a parallel reality game hosted by the game server 120. In this case, the displayed AR objects may be interactive game objects (e.g., game buttons, game items, etc.), game boards, game characters (e.g., game characters controllable by a player), game effects, or other AR content of the parallel reality game. In other embodiments, the virtual objects may be displayed with some transparency or selectivity to allow the user to see the real world through the virtual objects. In one embodiment, the display module 360 ​​may display a representation of a terrain mesh to allow the user to view a game board generated for a location other than the user's current location.

[0055] Furthermore, in situations where the systems and methods discussed herein access and analyze a user's personal information or utilize personal information such as location information, the user may be provided with the opportunity to control whether a program or function collects information and whether and / or how to receive content from the system or other applications. Such information or data will not be collected or used until the user is provided with meaningful notice of what information will be collected and how such information will be used. Information will not be collected or used unless the user has given consent, which may be revoked or modified by the user at any time. Thus, the user can control how information about the user is collected and used by the application or system. Furthermore, certain information or data may be treated in one or more ways such that any personally identifiable information is removed before it is stored or used. For example, the user's identity may be treated such that any personally identifiable information cannot be determined.

[0056] Exemplary Game Interface FIG. 4 illustrates an exemplary game interface 400 of a parallel reality game that may be presented on the display of the client device 110 as part of an interface between a player and a virtual world (such as the virtual world 210 of FIG. 2) or a parallel reality world, according to one or more embodiments. The game interface includes a display window 410 that may be used to display the virtual world and various aspects of the game, such as the locations of virtual items. For example, the display window 410 displays a game board 420 having a number of tiles 425. In the example of FIG. 4, the tiles are polygonal tiles. Specifically, the tiles 425 of the game board 420 are square tiles. However, tiles of other shapes (such as triangular tiles or hexagonal tiles) may be used. In some embodiments, only shapes that can be tiled on a flat surface may be used to generate the game board 420. Additionally, the display window 410 displays one or more virtual objects or characters 430. The virtual objects or characters 430 may be positioned within one or more tiles 425 of the game board 420.

[0057] The user interface 400 may also display other information, such as game data information, game communications, player information, client location verification instructions, and other information associated with the game. For example, the user interface 400 may display player information 415, such as player name, experience level, and other information. The user interface 400 may include menus 450 for accessing various game settings and other information associated with the game. The user interface 400 may also include a communication interface that allows communication between the game system and the player and between one or more players of a parallel reality game.

[0058] According to aspects of the present disclosure, a player can interact with a parallel reality game by simply carrying a client device 110 in the real world. For example, a player can play the game by simply accessing an application associated with the parallel reality game on a smartphone and moving around the real world using the smartphone. In this regard, a player does not need to keep looking at a visual representation of the virtual world on a display screen to play a location-based game. As a result, the user interface 400 can include multiple non-visual elements that allow the user / player to interact with the game. For example, the game interface can provide audio notifications to the player when the player is approaching a virtual element or object in the game or when an important event occurs in the parallel reality game. The player can control these audio notifications using the audio control 440. Depending on the type of virtual element or event, various types of audio notifications can be provided to the user / player. The audio notifications can increase or decrease in frequency or volume depending on the player's proximity to the virtual element or object. Other non-visual notifications and signals can also be provided to the user / player, such as vibration notifications or other suitable notifications or signals.

[0059] Using the disclosure provided herein, one of ordinary skill in the art will appreciate that numerous game interface configurations and underlying functionality will become apparent in light of the present disclosure, which is not intended to be limited to any particular configuration.

[0060] Exemplary Methods 5 is a flow chart illustrating a method 500 for generating an augmented reality game board, according to one or more embodiments. The method 500 results in a game board that can be overlaid on an image or video of a scene to enable an augmented reality game. The steps of FIG. 5 may be performed by the gaming module 135 of the client device 110. Alternatively, one or more steps of the method 500 may be performed by the game server 120 or by other components of the client device 110. Additionally, in some embodiments, steps may be performed in parallel, steps may be performed in a different order, or different steps may be performed.

[0061] In some embodiments, method 500 begins with the gaming module 135 of the client device 110 receiving (510) one or more images of a scene. For example, the gaming module 135 of the client device 110 receives one or more images (such as a sequence of images of a video) captured using the camera assembly 125 of the client device 110.

[0062] The gaming module 135 receives (520) a mesh (e.g., a terrain mesh) based on the one or more images. In some embodiments, the mesh is generated by the terrain mesh management module 320 based on the received one or more images. Alternatively, the mesh is retrieved from a local terrain mesh store 325. In some embodiments, the gaming module 135 sends the one or more images to the game server 120 and receives from the game server 120 a mesh for the scene depicted in the one or more images. That is, the game server can generate or retrieve a pre-generated mesh based on one or more images received from the gaming module 135 of the client device 110 over the network 105.

[0063] In some embodiments, the mesh is retrieved based on the location of the location of the client device 110 (as determined by the positioning module 140). The terrain mesh management module 320 or the game server 120 may identify a mesh that corresponds to a scene located at the current location of the client device and provide the identified mesh to the gaming module 135.

[0064] The game board generation module 350 identifies passable space in the scene based on the received mesh. In one embodiment, whether a mesh cell is passable is semantic information determined by applying a classifier to data defining the mesh cell (e.g., by the semantic segmentation module 340). In other embodiments, the game board generation module 350 identifies passable space by identifying mesh cells that are parallel to a horizontal plane (e.g., a floor plane). The game board generation module 350 determines the angle between each mesh cell of the received mesh and the horizontal plane. The horizontal plane may be identified with various techniques, such as identifying a floor plane using a classifier or measuring a gravity vector using a force sensor of the client device 110. The game board generation module 350 then classifies the mesh cell as passable if the angle between the mesh cell and the horizontal plane is less than a threshold angle. In some embodiments, the game board generation module 350 may evaluate multiple mesh cells that are connected to each other and treat them as a single mesh cell if the standard deviation of the normal vectors of the mesh cells in the group is less than a threshold. This can prevent small anomalies in a mesh from causing the game board generation module 350 to designate a portion of a mesh as impassable when it is actually passable to a virtual character. The game board generation module 350 can then add meshes classified as passable to the passable space.

[0065] In some embodiments, in addition to or instead of comparing the angle between the mesh cell and the horizontal plane to the threshold angle, the game board generation module 350 determines other measures, such as the average or median distance between the mesh cell and the horizontal plane, and compares the calculated measure or measures to a corresponding threshold. In some embodiments, the game board generation module 350 determines the average distance between the plane defined by the mesh cell and the horizontal plane, and compares the average distance to the threshold distance. Alternatively, the game board generation module 350 determines whether any portion of the mesh cell is greater than or equal to a threshold distance from a specified horizontal plane. In other embodiments, the game board generation module 350 may determine the distance between the plane defined by the mesh cell and the horizontal plane through methods such as random forests, support vector machines (SVM), or winsorization. The game board generation module 350 then classifies the mesh cell as passable if the angle between the mesh cell and the horizontal plane is less than the threshold angle and the distance between the mesh cell and the horizontal plane is less than the threshold distance. In other embodiments, the distance between the mesh cell and other planes may be used, for example a vertical plane corresponding to the wall if the game board is placed on a wall, or a plane at an angle to the horizontal if the ground in the environment shown in one or more images is determined to be sloped.

[0066] The game board generation module 350 places polygon tiles in the identified traversable space (540). In some embodiments, the game board generation module 350 places tiles having preset shapes and sizes. For example, the game board generation module 350 places square tiles having a perimeter of 1 foot (30 cm) by 1 foot (30 cm). In some embodiments, the tile shapes and sizes are selected based on the game being played by the user of the client device 110. Furthermore, the tile shapes may be limited to shapes that can be tiled on a flat surface. In some embodiments, the game board generation module 350 places tiles having multiple shapes or sizes in a preset pattern.

[0067] In some embodiments, the game board generation module 350 may use virtual character configurations to identify traversable surfaces. As described above, the game board generation module 350 divides the terrain mesh into tiles. The game board generation module 350 determines a height measurement for each tile (e.g., the average or median height of the portion of the terrain mesh in which the tile is placed) and identifies contiguous groups of tiles using agents or sliding windows. Agents may be virtual characters selected by the player or may be virtual characters used to determine connected tiles.

[0068] The game board generation module 350 may use the characteristics of the virtual character to identify surfaces that are traversable by the virtual character selected by the player. A surface is a set of contiguous tiles in which the virtual character can reach all of the tiles by crossing between adjacent tiles that do not have a height difference greater than the virtual character's step or jump height. For example, a virtual character with a large step height (e.g., height change) may be able to climb a staircase of a given step height, while a virtual character with a smaller step height may not be able to. In another example, the virtual character configuration (e.g., jump height / distance) may enable the virtual character to jump over a gap between disconnected surfaces. In some embodiments, the virtual character configuration may determine the size of the tile. Additionally, the game board generation module 350 may identify a multi-level surface. For example, a first surface (e.g., a floor surface) and a second surface (e.g., a tabletop surface) disposed on the first surface.

[0069] 6A-6C illustrate a process for placing tiles in a traversable space 610, according to one or more embodiments. As shown in FIG. 6A, to place a polygon tile, the game board generation module 350 determines the placement of an initial tile 620. The game board generation module 350 may determine the location of the initial tile by identifying an edge or boundary of the traversable space 610 and place the initial tile 620 a set distance from the boundary of the traversable space 610.

[0070] The game board generation module 350 identifies the location of the additional tiles based on the placement of the initial tiles 620. For example, as shown in FIG. 6B, for each edge of the initial tiles 620, the game board generation module 350 determines whether an additional tile placed adjacent to the initial tile along the edge of the initial tile overlaps the impassable space 615. If the additional tile does not overlap the impassable space 615, the game board generation module 350 adds the additional tile to the game board. In contrast, if the additional tile overlaps the impassable space 615, the game board generation module 350 removes the additional tile from the game board. For example, in the diagram of FIG. 6B, the additional tile 630 does not overlap the impassable space 615 (i.e., is entirely contained within the passable space 610), and the additional tile 635 overlaps the impassable space. Thus, the game board generation module 350 adds the additional tile 630, which does not overlap the impassable space 615, to the game board. In other embodiments, the game board generation module 350 adds tiles to the game board that overlap with less than a threshold amount of impassable area (eg, less than 5%).

[0071] The game board generation module 350 recursively adds further additional tiles by repeating the process for each additional tile added to the game board. For example, as shown in the diagram of FIG. 6C, the game board generation module 350 determines whether an additional tile 640 adjacent to a tile 630 overlaps an impassable space 615, and adds the additional tile 640 if the additional tile 640 does not overlap the impassable space 615 (or overlaps by less than a threshold). Similarly, for each tile 640 added to the game board, the game board generation module 350 determines whether the additional tile 650 overlaps the impassable space 615, and adds the additional tile 650 if the additional tile 650 does not overlap the impassable space 615 (or overlaps by less than a threshold). This process is repeated until no additional tiles can be added to the game board. In some embodiments, the game board generation module 350 builds the game board during an initialization phase of the parallel reality game. Alternatively or additionally, the game board generation module 350 can continuously update the game board as the user / player moves. The game board generation module 350 can take an updated terrain mesh (e.g., a terrain mesh of a new scene accessed by a player as the player moves around the play area) and expand the game board by adding additional tiles to the game board to cover the passable space of the new scene accessed by the player. That is, the game board generation module 350 identifies the passable area of ​​the new scene and adds tiles to the edges of the board game to expand the board game in the direction of the new scene.

[0072] In some embodiments, the game board generation module 350 uses other algorithms to determine the location of polygon tiles within the traversable space. For example, the game board generation module 350 can overlay a grid of tiles over the traversable space and add to the game board tiles from the grid that are completely enclosed within the traversable space. Alternatively, the game board generation module 350 may generate a game board that completely covers the traversable space and remove tiles from the game board that overlap the non-traversable space.

[0073] In some embodiments, the efficiency of the tile placement or the efficiency of the game board (i.e., the ratio between the area covered by the game board and the area of ​​the traversable space) depends on the placement of the initial tiles. In some embodiments, the placement of the initial tiles is determined based on the initial viewpoint when the process for generating the game board is started. This may result in a non-optimized placement of the tiles.

[0074] 7A illustrates an example game interface 700A for a parallel reality game having a game board with a non-optimized tile arrangement, according to one or more embodiments. As shown in the game interface 700A of FIG. 7A, the tiles 425 of the game board 420A are at an angle to the boundaries of the traversable area 610. Thus, the efficiency of the game board 420A is reduced.

[0075] The game board generation module 350 may optimize the placement of tiles 425 of the game board 420 to increase the efficiency of the game board. For example, the game board generation module 350 determines the angle and placement of the initial tiles 620 to increase the efficiency of the game board 420. In some embodiments, the game board generation module 350 generates multiple game boards with initial tiles each having a different angle or location. The game board generation module 350 then calculates the efficiency of each of the generated game boards and selects the game board with the maximum efficiency. In other embodiments, the tile positions and sizes, tile angles, distances, and constraints (e.g., properties and overlapping surfaces) may be modeled as a discrete optimization problem (e.g., a mixed integer problem). To solve the discrete optimization problem, an algorithm such as a branch and bound algorithm may be used.

[0076] FIG. 7B illustrates an example game interface 700B of a parallel reality game having a game board with an optimized tile arrangement, according to one or more embodiments. As illustrated in the game interface 700B of FIG. 7B, the portion of the passable space 610 covered by the game board 420B of FIG. 7B is increased compared to the portion of the passable space 610 covered by the game board 420a of FIG. 7A. For example, to increase the efficiency of the game board 420B, the game board generation module 350 rotates the tiles 425 of the game board. By rotating the tiles 425 of the game board, the tiles of the game board 420B of FIG. 4B are aligned with one or more boundaries of the passable space 610.

[0077] 5, the game board generation module 350 may assign (550) properties to each polygonal tile of the game board. For example, the game board generation module 350 may determine the properties of the tile based on the surface material of the location corresponding to the tile. The game board generation module 350 may assign one or more properties to the tile based on the output of the semantic segmentation module 340 for the mesh cell that corresponds to the location associated with the tile.

[0078] FIG. 7C illustrates an example game interface 700C for a parallel reality game having a game board with tiles having different surface properties, according to one or more embodiments. The game interface 700C of FIG. 7C illustrates a game board 710 having a first section including a first set of tiles 725 having a first property and a second section including a second set of tiles 735 having a second property. The first set of tiles 725 are placed on a first surface 720 made of a first material, and the second set of tiles 735 are placed on a second surface 730 made of a second material. Specifically, in the example of FIG. 7C, the first set of tiles 725 are assigned a first property corresponding to the first material of the first surface 720, and the second set of tiles 735 are assigned a second property corresponding to the second material of the second surface 730. Additionally or alternatively, potential tiles that would have a particular property if placed may not be placed. For example, potential tiles that overlap a first material (e.g., water) may not be placed such that those portions of the image (and thus the physical environment) are unavailable for placement of an AR character or other object.

[0079] For example, the first surface 720 is a concrete surface and the second surface 730 is a grass surface. The semantic segmentation module 340 can analyze an image depicting the first surface 720 and assign to mesh cells that overlap the first surface 720 a characteristic indicating that the mesh cells correspond to a concrete surface. Similarly, the semantic segmentation module 340 can analyze an image depicting the second surface 730 and assign to mesh cells that overlap the second surface 730 a characteristic indicating that the mesh cells correspond to a grass surface.

[0080] The game board generation module 350 then assigns each of the tiles of the game board a tile property based on the mesh cells that overlap the tile. In some embodiments, the tile property is determined based on the game being played by the user of the client device 110. For example, the game being played by the user of the client device may configure the game board generation module 350 to assign tiles that overlap a concrete surface a tile property corresponding to ice and tiles that overlap a grass surface a tile property corresponding to water. Thus, in a parallel reality game using the generated game board, tiles that overlap the first surface 720 will appear to be made of ice and tiles that overlap the second surface 730 will appear to be made of water.

[0081] In some embodiments, one or more tiles overlap multiple surfaces, each associated with a different property. In this case, the game board generation module 350 can assign properties to the tiles based on the amount of overlap between the tile and each surface. For example, the game board generation module 350 determines the amount of overlap between the tile and each surface and assigns the property associated with the surface with the greatest amount of overlap with the tile. For example, the game board 710 includes a tile 750 having a first portion that overlaps with the first surface 720 and a second portion that overlaps with the second surface 730. The game board generation module 350 determines an area of ​​the first portion and an area of ​​the second portion, and compares the area of ​​the first portion and the area of ​​the second portion. If the area of ​​the first portion is greater than the area of ​​the second portion, the game board generation module 350 assigns the property corresponding to the first surface 720 to the tile 750. Alternatively, if the area of ​​the second portion is greater than the area of ​​the third portion, the game board generation module 350 assigns the tile 750 the properties corresponding to the second surface 730 .

[0082] In the example of FIG. 7C , a second portion of the tile 750 that overlaps with the second surface 730 is larger than a first portion of the tile 750 that overlaps with the first surface 720. Thus, the tile 750 is assigned a property corresponding to the second surface 730. Additionally, the game board 710 includes a tile 755 having a first portion that overlaps with the first surface 720 and a second portion that overlaps with the second surface 730. Here, the first portion of the tile 755 that overlaps with the first surface 720 is larger than a second portion of the tile 755 that overlaps with the second surface 730. Thus, the tile 755 is assigned a property corresponding to the first surface 720.

[0083] Procedurally generated augmented reality game board Figure 8 illustrates an exemplary game interface 800 for a parallel reality game that may be presented on the display of a client device 110 as part of an interface between a player and a virtual world (such as virtual world 210 of Figure 2) or a parallel reality world, according to one or more embodiments. The game interface 800 of Figure 8 uses a procedurally generated or dynamic game board that expands when a player takes certain actions, when a player moves around the game board, or when events occur in the virtual world.

[0084] In some embodiments, the game board generation module 350 generates a game board around the area 810 close to the player. The portion of the game board around the area 810 may be generated in response to a request for the game board (e.g., generated by a player requesting to perform a particular action or activate a particular mode in the game) or automatically (e.g., when the client device 110 is in a position where the game board has not yet been generated). In one embodiment, the game board generated around the area 810 close to the player is generated based on one or more images captured using the camera assembly 125 of the player's client device 110. For example, a height field or mesh is generated based on one or more images captured using the camera assembly 125 of the player's client device 110 to identify traversability in the area 810 close to the player. Based on the identified traversable areas, the game board generation module generates the game board. Alternatively, the location data (e.g., GPS data) of the client device 110 may be used to retrieve (e.g., from the game server 120) a pre-generated height field or terrain mesh for the immediate vicinity of the client device.

[0085] The game board generation module 350 may expand the game board to include regions 820 that are far from the user's location (e.g., above a threshold). In some embodiments, the regions 820 include regions that are visible from the user's location. However, the regions 820 may be at a distance that causes the accuracy or confidence level in identifying traversable areas using height fields and terrain meshes generated from camera data to fall below a certain threshold. The generation of the game board in the regions 820 may be performed based on a combination of information collected from the player's client device 110 (e.g., one or more images captured by the camera assembly 125 of the client device 110, depth information captured by a depth sensor of the client device, etc.) and information received from an external source (e.g., 3D map information received from the game server 120).

[0086] The game board generation module 350 may extend the game board to include an area 830 that is not visible to the client device 110. For example, the area 830 may include an area blocked by buildings or other large objects. In the example of Figure 8, the area 830 includes a road behind a set of buildings. In one embodiment, the game board generation module 350 generates the game board for the area 830 based on 2D map data (e.g., retrieved from the game server 120).

[0087] FIG. 9 is a flow chart illustrating a method 900 for dynamically generating an augmented reality game board, according to one or more embodiments. The method 900 results in a game board that can be used to place virtual objects or characters in a virtual world of an augmented reality game. The steps of FIG. 9 may be performed by the gaming module 135 of the client device 110. Alternatively, one or more steps of the method 900 may be performed by the game server 120 or by other components of the client device 110. Additionally, in some embodiments, steps may be performed in parallel, steps may be performed in a different order, or different steps may be performed.

[0088] In some embodiments, method 900 begins with the gaming module 135 of the client device 110 identifying a location of a player. In some embodiments, the gaming module 135 determines the location of the player by determining the location of the player's client device. For example, the gaming module 135 may determine the location of the player's client device 110 using a Global Positioning System (GPS) signal, using Video Positioning System (VPS) analysis, or using a combination thereof.

[0089] The gaming module 135 determines (920) a topology of the player's surroundings and generates (930) a game board based on the determined topology of the player's surroundings. The gaming module 135 may determine the topology of the player's surroundings and generate the game board using the process described in Figure 5. For example, the gaming module 135 may generate a mesh or determine a height field based on one or more images (or footage) captured using the camera assembly 125 of the player's client device 110 and generate the game board based on the mesh or height field.

[0090] The gaming module 135 may identify one or more regions outside the existing game board to expand the game board beyond the current game board boundaries (940). For example, the game logic of a parallel reality game may choose to generate objects or characters at locations outside the current extent of the game board. In some embodiments, the gaming module 135 receives an indication of a location to expand the game board.

[0091] The gaming module 135 receives 950 information regarding the identified areas outside the current game board (950). Specifically, the gaming module 135 receives information regarding the areas that will be used to extend the current game board. In some embodiments, the gaming module 135 receives information regarding the topology of the areas outside the current game board from the game server 120. Alternatively or additionally, the gaming module 135 receives information regarding the topology of the areas outside the current game board from a third-party system. The gaming module 135 may receive different or additional information regarding the areas outside the game board, such as semantic characteristics. In some embodiments, the gaming module 135 sends a request to identify one or more areas and receives information regarding those areas. That is, the gaming module 135 sends a request to identify areas that will be used to extend the current game board and receives information regarding those areas (e.g., topology and other geographic information).

[0092] In some embodiments, the topology information includes height information of points within the identified region or regions. In other embodiments, the topology information includes mesh or 3D map information for the identified region or regions. In some embodiments, the gaming module 135 receives different topology information for different regions. For example, as a player explores the world using the parallel reality game, the game server 120 analyzes the topology of locations where the player has utilized the parallel reality game and stores topology information for those locations. When the gaming module 135 of the player's client device requests topology information for a region that has been previously analyzed, the game server 120 transmits the stored topology information of the region that was generated based on the previous analysis to the gaming module 135 of the client device. However, if the gaming module 135 of the player's client device requests topology information for a region that has not been previously analyzed (or has not been analyzed within a threshold amount of time) by the game server 120, the game server 120 transmits less accurate topology information for the requested region to the gaming module 135 of the client device 120. For example, the gaming module 135 may transmit 3D map information or satellite imagery information (or information derived therefrom) regarding the requested area to the gaming module 135 of the player's client device.

[0093] Based on the received topology information, the gaming module 135 expands the game board (960). For example, the gaming module 135 identifies a traversable area within the identified region for expanding the game board and adds the identified traversable area to the traversable area of ​​the current game board.

[0094] In some embodiments, the characteristics of the traversable area in the area identified for extending the game board are determined based on the topology information received by the gaming module 135. Furthermore, the characteristics of the traversable area are additionally determined based on the distance between the area identified for extending the game board and the location of the client device. Specifically, the algorithm for determining the characteristics of the traversable area may depend on the distance between the area identified for extending the game board and the location of the client device. Alternatively, the characteristics of the traversable area are determined based on a confidence level of the accuracy of the received topology information. For example, the algorithm for determining the characteristics of the traversable area depends on a confidence level of the accuracy of the received topology information.

[0095] In some embodiments, characteristics of the traversable area (such as height values ​​of points in the traversable area and semantics of points in the traversable area) are determined based on a weighted average or weighted combination of multiple sources. Each source may be weighted based on the confidence level of the corresponding source. In some embodiments, the confidence level of the topology information of the area received from a particular source depends on the distance between the player's client device and the area. For example, the topology information of the area determined based on images captured by the camera assembly 125 of the client device 110 or sensor data captured by the client device 110 depends on the distance between the client device 100 and the area. As the distance between the area and the client device 110 increases, the confidence level of the topology information determined based on images captured by the camera assembly 125 of the client device 110 or sensor data captured by the client device 110 decreases.

[0096] In other embodiments, the gaming module 135 periodically updates the topology information and updates the characteristics of the traversable area based on the updated topology information. For example, as the player moves around the world, the gaming module 135 of the player's client device 110 captures images or sensor data of the client device's surroundings. When the player moves to a second location, the client device 110 captures images or sensor data of the second location's surroundings. The gaming module 135 updates the topology information of the area around the second location based on the captured images or sensor data of the second location's surroundings. Based on the determined topology information, the gaming module 135 updates the characteristics of the game board that correspond to the area around the second location.

[0097] Thus, the gaming module 135 can expand the game board for a player beyond the player's immediate vicinity by using topology information from a less accurate source than the topology information determined based on imagery or sensor data captured by the client device. As the player approaches areas corresponding to locations where the game board was generated using the less accurate source, characteristics of the game board at those locations can be updated based on newly captured information by the camera assembly or sensors of the client device 110.

[0098] In some embodiments, this further allows the gaming module 135 to extend the game board to locations that are not visible to the camera assembly or sensors of the client device. For example, the gaming module 135 can extend the game board to locations behind buildings or other large objects. The gaming module 135 can use topology information from a less accurate source to determine characteristics of the game board in locations that are not visible by the client device, and can update the characteristics of the game board in those locations when the player moves to a second location where the locations that were not visible by the client device become visible.

[0099] In various embodiments, the gaming module 135 determines the game board at multiple distances with corresponding levels of detail / granularity. The exemplary embodiment described above has three regions of the game board with corresponding levels of detail, but it will be understood that any number of regions may be used. Each region is determined from corresponding geographic data, such as camera images, 3D map data, 2D map data, and other geographic location data, e.g., weather forecasts, population density, number of users in the preceding time period, etc. Image segmentation may be used to apply characteristics to the game board based on images of corresponding portions of the real-world environment. For example, portions of the terrain mesh corresponding to buildings may be assigned tags such as "building," "concrete," "office," etc., based on segmentation classifiers applied to images showing buildings captured by the client device 110.

[0100] A parallel reality game may dynamically place virtual elements (e.g., objects or characters) on a procedurally generated game board according to semantic properties of the game board. The game may specify one or more properties of a desired location for the virtual elements and may query the game board for locations that meet the desired properties. In some embodiments, multiple possible locations may be returned along with a matching score indicating how closely the identified location matches the desired properties and / or a confidence level of the location's suitability. The search for a suitable location may begin in an area immediately around the client device (e.g., extending to a distance of about 5 meters from the client device) where a semantic topology mesh may be generated using the client device's sensors. If no suitable location is found, the search may be extended to a next area that is further away from the client device but still visible to the device's camera. This area may be modeled using a combination of images captured by the client device, previous images captured by other devices, 3D map data, 2D map data, and / or other available information. If this second area also does not have a suitable location, a third area may be searched that is sufficiently far away to be invisible and / or substantially invisible to the client device. The suitability of a location in this third region may be determined from 2D map data and other geographic information available to the client device.

[0101] To take a specific example, a virtual bird may be intended to be placed in a tree. To find the location of the virtual bird, the client device may first search a segmentation map generated from a camera image captured by the client device to identify potential trees. If a suitable tree is found, the bird may be placed in the tree. Conversely, if the tree is not found, the search may be extended to more distant areas visible to the client device's camera. The client device may identify portions of the image corresponding to a forest in the background where individual trees cannot be accurately segmented, and place the bird in the forest. As the player moves toward the forest, a new terrain map may be generated with semantic information capable of resolving individual trees, and the bird may be placed in the appropriate tree. Finally, if no trees are visible to the camera at all, the client device may search for a forested area in the 2D map and place the bird in the forested area. The client device may present an arrow or other indicator to guide the player toward the bird. As described above, as the player gets closer to the bird, the game board may be updated with more precision, and eventually the bird may be placed in a particular tree.

[0102] In one embodiment, the virtual item may be located by searching for a location that has a particular object, material, or other characteristic. Examples include placing a virtual object on a physical object (e.g., a table, bench, chair, bed, etc.), placing an object on a surface of a particular material (e.g., water, grass, concrete, sand, etc.), or placing an object in a particular type of location (e.g., indoor public space, park, road, field, etc.). More complex characteristics may be specified for the desired location. For example, a game may query for an object that is large enough for the virtual character to hide in. If the virtual character is a rabbit, a nearby small bush in a terrain map generated by the client device may be a suitable location. Because the location and shape of the bush can be determined relatively accurately from the camera image, a realistic animation of the virtual rabbit running behind the bush and even peeking out from behind the bush may be provided. In contrast, if the virtual character is Godzilla, the location may be behind a building in the background of the image captured by the device's camera. In this case, the building is far away, so its location does not need to be known with a high degree of precision. A believable experience can be created by scaling the virtual character based on a relatively inaccurate estimate of the distance to the building.

[0103] As another example, when a player approaches a timid virtual character, the character may try to run and hide. The query may be for a suitable hiding location, which may be evaluated using various factors, such as the percentage of directions from which the location is visible from a given distance (e.g., 5 meters), the lighting level of the location, the number of other players or virtual characters within a threshold distance (e.g., meters after) of a potential hiding spot, the material that the location is made of, etc.

[0104] Exemplary Computing System FIG. 10 is an exemplary architecture of a computing device according to an embodiment. Although FIG. 10 shows a high-level block diagram illustrating the physical components of a computer that may be used as part or all of one or more entities described herein, according to an embodiment, the computer may have additional, fewer, or variations of the components provided in FIG. 10. Although FIG. 10 shows a computer 1000, this figure is intended as a functional description of various features that may be present in a computer system, rather than as a structural schematic of the implementations described herein. In practice, those skilled in the art will recognize that items shown separately may be combined and some items may be separated.

[0105] 10 shows at least one processor 1002 coupled to a chipset 1004. Also coupled to the chipset 1004 are a memory 1006, a storage device 1008, a keyboard 1010, a graphics adapter 1012, a pointing device 1014, and a network adapter 1016. A display 1018 is coupled to the graphics adapter 1012. In one embodiment, the functionality of the chipset 1004 is provided by a memory controller hub 1020 and an I / O hub 1022. In another embodiment, the memory 1006 is directly coupled to the processor 1002 instead of the chipset 1004. In some embodiments, the computer 1000 includes one or more communication buses for interconnecting these components. The one or more communication buses optionally include circuitry (sometimes referred to as a chipset) that interconnects and controls communication between system components.

[0106] The storage device 1008 is any non-transitory computer-readable storage medium, such as a hard drive, a compact disk read-only memory (CD-ROM), a DVD, or a solid-state memory device, or other optical storage, a magnetic cassette, a magnetic tape, a magnetic disk storage or other magnetic storage device, a magnetic disk storage device, an optical disk storage device, a flash memory device, or other non-volatile solid storage device. Such a storage device 1008 may also be referred to as persistent memory. The pointing device 1014 may be a mouse, a trackball, or other type of pointing device, and is used in combination with the keyboard 1010 to input data into the computer 1000. The graphics adapter 1012 displays images and other information on the display 1018. The network adapter 1016 couples the computer 1000 to a local or wide area network.

[0107] The memory 1006 holds instructions and data used by the processor 1002. The memory 1006 may be a non-persistent memory, examples of which include high-speed random access memory such as DRAM, SRAM, DDR RAM, ROM, EEPROM, flash memory, etc.

[0108] As is known in the art, computer 1000 may have different and / or other components than those shown in FIG. 10. Additionally, computer 1000 may lack certain shown components. In one embodiment, a computer 1000 functioning as a server may lack keyboard 1010, pointing device 1014, graphics adapter 1012, and / or display 1018. Additionally, storage device 1008 may be local and / or remote from computer 1000 (such as embodied in a storage area network (SAN)).

[0109] As known in the art, the computer 1000 is adapted to execute computer program modules for providing the functionality described herein. As used herein, the term "module" refers to computer program logic utilized to provide a specified functionality. Thus, a module may be implemented in hardware, firmware, and / or software. In one embodiment, the program module is stored on the storage device 1008, loaded into the memory 1006, and executed by the processor 1002.

[0110] Additional Considerations Some portions of the above description describe embodiments in terms of algorithmic processes or operations. These algorithmic descriptions and representations are commonly used by those skilled in the data processing arts to effectively convey the substance of their work to others skilled in the art. These operations, while described functionally, computationally, or logically, will be understood to be implemented by a computer program including instructions for execution by a processor or equivalent electrical circuitry, or microcode, or the like. Moreover, without loss of generality, it has also proven convenient at times to refer to such arrangements of functional operations as modules.

[0111] As used herein, a reference to "one embodiment" or "an embodiment" means that a particular element, feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment. The appearances of the phrase "in one embodiment" in various places in the specification do not necessarily all refer to the same embodiment.

[0112] Some embodiments may be described using the terms "coupled" and "connected," along with their derivatives. It should be understood that these terms are not intended as synonyms for each other. For example, some embodiments may be described using the term "connected" to indicate that two or more elements are in direct physical or electrical contact with each other. In another example, some embodiments may be described using the term "coupled" to indicate that two or more elements are in direct physical or electrical contact with each other. However, the term "coupled" may also mean that two or more elements are not in direct contact with each other, but still cooperate or interact with each other. The embodiments are not limited in this context.

[0113] As used herein, the terms "comprises," "comprising," "including," "including," "having," "having," or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, article, or apparatus comprising a list of elements is not necessarily limited to only those elements and may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Furthermore, unless otherwise expressly stated, "or" means an inclusive "or" and not an exclusive "or." For example, a condition A or B is satisfied by any one of the following: A is true (or present) and B is false (or absent), A is false (or absent) and B is true (or present), or both A and B are true (or present).

[0114] Furthermore, the use of "a" or "an" is employed to describe elements and components of an embodiment. This is done merely for convenience and to give a general sense of the disclosure. The specification should be read to include one or at least one, and the singular also includes the plural unless it is clear that it is meant otherwise.

[0115] Upon reading this disclosure, those skilled in the art will recognize still further alternative structural and functional designs for systems and processes for determining or using repeatability of points of interest. Thus, while specific embodiments and applications have been illustrated and described, it is to be understood that the described subject matter is not limited to the precise construction and components disclosed herein, and that various modifications, changes, and variations that will be apparent to those skilled in the art may be made in the arrangement, operation, and details of the disclosed methods and apparatus.

Claims

1. receiving one or more images of a scene from a camera of a mobile device; obtaining a terrain mesh of the scene based on the one or more images, the terrain mesh comprising a plurality of cells; identifying traversable space within the scene from the terrain mesh; determining a location for each of a plurality of polygon tiles within the identified traversable space, the plurality of polygon tiles forming a game board; 11. A computer-implemented method comprising:

2. The identification of the passable space includes, for each cell of the terrain mesh, determining an angle between the cell and a specified plane; comparing the angle between the cell and the designated plane to a threshold angle; assigning the cell to the traversable space in response to the angle between the cell and the designated plane being less than the threshold angle; The computer-implemented method of claim 1 , comprising:

3. The identification of the passable space includes, for each cell of the terrain mesh, determining a distance between the cell and the specified plane; comparing the distance between the cell and the designated plane to a threshold distance; assigning the cell to the traversable space in response to the angle between the cell and the designated plane being less than the threshold angle and the distance between the cell and the designated plane being less than the threshold distance; The computer-implemented method of claim 2 further comprising:

4. Identifying the passable space includes: identifying a traversable space for the virtual character based on the virtual character configuration; The computer-implemented method of claim 1 , further comprising:

5. Determining a location for each of a plurality of polygon tiles within the identified traversable space includes, for each tile of a grid of tiles: determining whether the tile is contained within the traversable space; adding the tile to the game board in response to determining that the tile is contained within the traversable space; and The computer-implemented method of claim 1 , comprising:

6. Determining a location for each of a plurality of polygon tiles within the identified traversable space includes: identifying an initial tile location within the traversable space; determining whether a second tile adjacent to the initial tile overlaps an impassable space around the traversable space; adding the second tile to the game board in response to determining that the second tile does not overlap the impassable space; and The computer-implemented method of claim 1 , comprising:

7. The second tile has the same shape and size as the initial tile. The computer-implemented method of claim 6.

8. For each polygon placed on the game board, determining a property associated with each cell that overlaps the polygon tile; determining a property to assign to the polygon tile based on the determined property associated with each cell that overlaps the polygon tile; The computer-implemented method of claim 1 , further comprising:

9. presenting a game interface to a user, the game interface overlaying a representation of the game board including the plurality of polygon tiles onto a second set of images of the scene captured by the camera of the mobile device; The computer-implemented method of claim 1 , further comprising:

10. Obtaining a terrain mesh of the scene includes: generating the terrain mesh of the scene based on one or more images of the scene received from a camera of a mobile device; The computer-implemented method of claim 1 , comprising:

11. Obtaining a terrain mesh of the scene includes: Retrieving a pre-generated terrain mesh based on the location of the client device; The computer-implemented method of claim 1 , comprising:

12. A non-transitory computer-readable storage medium configured to store instructions that, when executed by a processor, cause the processor to: receiving one or more images of a scene from a camera of a mobile device; obtaining a terrain mesh of the scene based on the one or more images, the terrain mesh comprising a plurality of cells; identifying traversable space within the scene from the terrain mesh; determining a location for each of a plurality of polygon tiles within the identified traversable space, the plurality of polygon tiles forming a game board; A non-transitory computer-readable storage medium that causes

13. The instructions for identifying traversable space cause the processor to, for each cell of the terrain mesh: determining an angle between the cell and a specified plane; comparing the angle between the cell and the designated plane to a threshold angle; assigning the cell to the traversable space in response to the angle between the cell and the designated plane being less than the threshold angle; The non-transitory computer-readable storage medium of claim 12 ,

14. The instructions for identifying traversable space may cause the processor to, for each cell of the terrain mesh: determining a distance between the cell and the specified plane; comparing the distance between the cell and the designated plane to a threshold distance; assigning the cell to the traversable space in response to the angle between the cell and the designated plane being less than the threshold angle and the distance between the cell and the designated plane being less than the threshold distance; The non-transitory computer-readable storage medium of claim 13 , further comprising:

15. For each of a plurality of polygon tiles within the identified traversable space, the instructions for determining a location may include causing the processor to, for each tile of a grid of tiles: determining whether the tile is contained within the traversable space; adding the tile to the game board in response to determining that the tile is contained within the traversable space; and The non-transitory computer-readable storage medium of claim 13 ,

16. For each of a plurality of polygon tiles within the identified traversable space, the instructions for determining a location may include causing the processor to: identifying an initial tile location within the traversable space; determining whether a second tile adjacent to the initial tile overlaps an impassable space around the traversable space; adding the second tile to the game board in response to determining that the second tile does not overlap the impassable space; and The non-transitory computer-readable storage medium of claim 13 ,

17. The instructions cause the processor to, for each polygon placed on the game board: determining a property associated with each cell that overlaps the polygon tile; determining a property to assign to the polygon tile based on the determined property associated with each cell that overlaps the polygon tile; The non-transitory computer-readable storage medium of claim 13 , further comprising:

18. The instructions cause the processor to: presenting a game interface to a user, the game interface overlaying a representation of the game board including the plurality of polygon tiles onto a second set of images of the scene captured by the camera of the mobile device; The non-transitory computer-readable storage medium of claim 13 , further comprising:

19. determining a request to identify an area outside a game board of a parallel reality game to expand the game board; receiving topology information of the area outside the game board from a plurality of sources; determining a confidence level of the topology information received from each source of the plurality of sources; expanding the game board based on a combination of the received topology information, the topology information being combined based on the determined trust level of each source of the plurality of sources; 11. A computer-implemented method comprising:

20. determining a characteristic associated with a location on a game board of the parallel reality game; receiving a query for a location on the game board, the query specifying desired criteria; identifying candidate locations for the game board in response to the query by performing a search of the locations on the game board for locations that satisfy the desired criteria specified by the query; calculating a suitability score for each of the candidate locations; selecting one of the candidate locations having a suitability score equal to or greater than a threshold; 11. A computer-implemented method comprising:

21. Performing a search of the locations on the game board for locations that satisfy the desired criteria specified by the query comprises: performing an initial search of an area within a predefined distance of a client device of a player of the parallel reality game; 21. The computer-implemented method of claim 20.

22. The method of claim 21, further comprising: generating a semantic topology mesh of the area within the predefined distance using one or more sensors of the client device.

22. The computer-implemented method of claim 21.

23. The method according to claim 22, wherein the region is a first region, and the computer-implemented method comprises: in response to determining that no locations in the first region satisfy the desired criteria, performing a search of a second region that is further away from the client device than the predefined distance, the second region being visible to a camera of the client device; further comprising:

22. The computer-implemented method of claim 21.

24. The computer-implemented method, wherein the query is a first query, the location is a first location, the desired criteria is a first desired criteria, receiving a second query specifying second desired criteria; determining, based on image data from a camera of a player's client device of the parallel reality game, that a location visible from the camera does not satisfy the second desired criterion; searching two-dimensional (2D) map data to determine candidate locations that meet the second desired criteria; further comprising:

21. The computer-implemented method of claim 20.

25. The desired criteria include the presence of one or more particular objects, the presence of a particular material, or a minimum size of the desired objects.

21. The computer-implemented method of claim 20.

26. The specific material is one of water, grass, concrete, or sand.

26. The computer-implemented method of claim 25.

27. The received topology information includes height information of points within the region.

20. The computer-implemented method of claim 19.

28. The received topology information includes mesh or three-dimensional (3D) map information about the region.

20. The computer-implemented method of claim 19.

29. A player of said parallel reality game analyzing the topology of a location where said parallel reality game is used; storing respective topology information corresponding to the locations where players of the parallel reality game have used the parallel reality game; 20. The computer-implemented method of claim 19, further comprising:

30. The request is a first request, the region is a first region, and the computer-implemented method comprises: In response to receiving a second request identifying a second region that has not yet been analyzed to determine corresponding topology information, transmitting one or more of 3D map information or satellite image information regarding the second region; further comprising:

20. The computer-implemented method of claim 19.

31. Identifying a first passable area within the game board; identifying a second traversable area within the region for expanding the game board; adding the second traversable area to the first traversable area within the game board; 20. The computer-implemented method of claim 19, further comprising:

32. (i) determining characteristics of the second traversable area based on the received topology information and (ii) a distance between the area outside the game board and a current location of a client device of a player of the parallel reality game; 32. The computer-implemented method of claim 31, further comprising: a third passable area within the region for expanding the game board based on a combination of the received topology information; expanding the game board based on a combination of the received topology information; adding the third traversable area to the first traversable area within the game board; Including, 32. The computer-implemented method of claim 31. periodically receiving updated topology information of the area outside the game board; updating characteristics of the second traversable area based on the updated topology information; 33. The computer-implemented method of claim 32, further comprising:

35. The region is a first region, Detecting that a player of the parallel reality game is located in a second region of the game board associated with topology information having a confidence level below an accuracy threshold; obtaining updated topology information of the second region from a camera assembly or sensor of a client device associated with the player; modifying one or more characteristics of the game board in the second region based on the updated topology information; 20. The computer-implemented method of claim 19, further comprising:

36. When executed, the method includes the steps of: determining a request to identify an area outside a game board of a parallel reality game to expand the game board; receiving topology information of the area outside the game board from a plurality of sources; determining a confidence level of the topology information received from each source of the plurality of sources; expanding the game board based on a combination of the received topology information, the topology information being combined based on the determined trust level of each source of the plurality of sources; A non-transitory computer-readable medium storing instructions for performing operations including:

37. When executed, the method includes the steps of: determining a characteristic associated with a location on a game board of the parallel reality game; receiving a query for a location on the game board, the query specifying desired criteria; identifying candidate locations for the game board in response to the query by performing a search of the locations on the game board for locations that satisfy the desired criteria specified by the query; calculating a suitability score for each of the candidate locations; selecting one of the candidate locations having a suitability score equal to or greater than a threshold; A non-transitory computer-readable medium storing instructions for performing operations including:

Images