Data hierarchy protocol for data transmission path selection

By optimizing the data flow hierarchy protocol and using multiple network paths to select the lowest delay path, the problem of user interaction delay in the AR environment is solved, and a multi-user parallel real gaming experience with low latency is achieved.

CN115136559BActive Publication Date: 2024-11-15NIANTIC INC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202080097227.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-12-20
Filing Date
2020-12-18
Publication Date
2024-11-15
Estimated Expiration
2040-12-18

AI Technical Summary

Technical Problem

Existing connectionless communication protocols, such as UDP, cannot meet the low latency requirements in augmented reality (AR) environments, resulting in significant delays in user interaction in AR environments, affecting the gaming experience, especially in parallel with multiple users participating. In a real game.

Method used

By optimizing the data flow hierarchy protocol, data communication is carried out using multiple network paths (such as game servers, cell towers, LANs), sampling and selecting paths with the lowest latency, and adjusting path selection according to bandwidth availability to achieve low latency Data transmission.

Benefits of technology

Significantly reduces the computer network latency, such as reducing the latency to less than 5 milliseconds, improves the real-time interactive experience in parallel real-life games of multiple users, and reduces the delay between player actions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115136559B_ABST
    Figure CN115136559B_ABST
Patent Text Reader

Abstract

The data stream hierarchy protocol is implemented by one or more devices to optimize how one or more devices process datagrams for network communications. The data stream hierarchy takes into account various available network paths for data streams. The device implementing the data stream hierarchy selects one or more available network paths in the available network paths to provide low latency in data communications with other devices. The device can sample various available network paths to determine path metrics (e.g., latency) and select one or more network paths based on these metrics. Available network paths can include paths through one or more intermediate nodes, such as paths through game servers, paths through cell towers, and paths through the network.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Application No. 62 / 951,926, filed on December 20, 2019, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to computer network protocols, and in particular to protocols for providing low-latency wireless communications between devices that are in physical proximity to each other. Background Art

[0004] A computer network is a collection of interconnected computing devices, such as the Internet, that exchange data. Communication protocols, such as the User Datagram Protocol (UDP), define a system of rules for exchanging data using a computer network. UDP follows a connectionless communication model, with no guaranteed delivery, ordering, or non-repetitiveness of datagrams. A datagram is the basic unit for communication and includes a header and a payload. The header is metadata that specifies aspects of the datagram, such as the source port, destination port, length of the datagram, and a checksum of the datagram. The payload is the data conveyed by the datagram. Computing devices that communicate using UDP transmit datagrams to each other via a computer network.

[0005] Connectionless communication protocols, such as UDP, typically have lower overhead and latency than connection-oriented communication protocols, such as the Transmission Control Protocol (TCP), which establishes a connection between computing devices before transmitting data. However, existing connectionless communication protocols are not suitable for data transmission that requires less latency than accommodated by existing technologies. For example, an augmented reality (AR) environment streamed at 60 frames per second (FPS) may require a latency that is an order of magnitude lower than that provided by current technology. In such an AR environment, frames may be spaced at intervals of approximately sixteen milliseconds, while current network protocols typically provide a latency of approximately one hundred milliseconds (or more).

[0006] Therefore, using existing technology, users may experience some degree of latency when interacting in an AR environment. This latency may cause interactions with old states of AR positioning data. For example, in a parallel reality game, a player may see an AR object at an old position (e.g., where the object was 100 milliseconds ago), while the AR positioning data actually has a new position for the object (e.g., the AR object has been moved by another player). Latency in communications between a client and a server that hosts or coordinates a parallel reality game may cause a frustrating user experience. This problem may be particularly severe when more than one user is involved in a parallel reality game, as latency may cause noticeable delays between the actions of one player appearing in other players' views of the AR environment. In addition, as the number of players increases, the mismatch caused by latency is further exacerbated. Summary of the invention

[0007] Augmented reality (AR) systems supplement views of the real world with computer-generated content. Incorporating AR into parallel reality games may improve the integration between the real and virtual worlds. AR may also increase interactivity between players by providing them with the opportunity to participate in a shared gaming experience in which they interact. For example, in a tank battle game, players may navigate virtual tanks around real-world locations in an attempt to destroy each other's tanks. The movement of the tanks may be limited by real-world geography (e.g., tanks move slower in rivers, faster on roads, cannot move through walls, etc.).

[0008] Existing AR session technology involves the server maintaining a master state and periodically synchronizing the local state of the environment at the client to the master state via a network (e.g., the Internet). However, synchronizing the local state of the device may take a lot of time (e.g., ~100 milliseconds), which is not conducive to the gaming experience. The player is actually interacting with the past game state rather than the current game state. This problem can be particularly serious when more than one user participates in a parallel reality game, because latency can cause noticeable delays between the actions of a player that appear in the view of other players. For example, if one player moves an AR object in the world, other players may not see that it has moved until a hundred milliseconds (or more) later, which is a human-perceptible delay. As a result, another player may try to interact with an object at a previous location and get frustrated when the game corrects the latency (e.g., by refusing to implement the action requested by the player or initially implementing the action and then undoing it the next time the player's client synchronizes with the server).

[0009] This and other problems can be solved by implementing a data stream hierarchy protocol that optimizes how one or more devices process datagrams. The data stream hierarchy includes various network paths for data streams. The device that implements the data stream hierarchy can utilize one or more network paths of the network path to ensure low latency for data communication with other devices. In some embodiments, the device samples various network paths to determine the network path with the minimum latency used in the data communication through the intermediate node. Various network paths can include paths through one or more intermediate nodes, such as paths through game servers, paths through cell towers, and paths through the network. The network path can also include a direct connection between devices (e.g., using Bluetooth). Using the data stream hierarchy protocol, latency can be improved to the order of milliseconds, for example, less than 5 milliseconds. In addition, the data stream hierarchy can optimize bandwidth usage based on bandwidth availability. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A network environment in which a data flow hierarchy protocol may be implemented according to an embodiment is shown.

[0011] Figure 2 is a block diagram illustrating a client according to one embodiment.

[0012] Figure 3 is a block diagram illustrating a game server according to one embodiment.

[0013] Figure 4 is a block diagram illustrating a cell tower according to one embodiment.

[0014] Figure 5 A process for using a data stream hierarchy protocol according to an embodiment is shown.

[0015] Figure 6 A process for transmitting data between clients within a geographic area around a cell tower is shown according to an embodiment.

[0016] Figure 7 is a diagram showing a suitable Figure 1 A high-level block diagram of an example computer for use within the computer network shown.

[0017] The accompanying drawings and the following description describe certain embodiments by way of illustration only. Those skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods may be employed without departing from the principles described. Reference will now be made to several embodiments, examples of which are shown in the accompanying drawings. Where feasible, similar or similar reference numerals are used in the drawings to indicate similar or similar functions. When elements share a common number followed by a different letter, the elements are similar or identical. Individual numbers refer to any one or any combination of these elements. DETAILED DESCRIPTION

[0018] Network environment

[0019] As disclosed herein, the data flow hierarchy protocol can provide reduced computer network latency relative to previous approaches (eg, in one embodiment, latency of ~10 milliseconds). Figure 1 A computer network 100 is illustrated that communicates using a data stream hierarchy protocol according to one embodiment. For clarity, the figure illustrates a simplified example using a block diagram. The computer network 100 includes two clients 110, a game database 115, a game server 120, a cell tower 130, and a local area network (LAN) 140. In some other embodiments, the computer network may include fewer, additional, or other components, such as additional clients 110, additional game servers 120, additional cell towers 130, or other network nodes. The clients 110 communicate with the game servers 120 that host parallel reality games. The clients 110 communicate with each other during the parallel reality games when players interact with other players of the parallel reality games.

[0020] The clients 110 are computing devices such as personal computers, laptops, tablet computers, smartphones, etc. The clients 110 can communicate with each other using a data stream hierarchy protocol, for example, while playing a parallel reality game. In one embodiment, each client 110 includes a local AR module and the game server 120 includes a master AR module. Each local AR module transmits AR data to local AR modules on other clients 110 and / or the master AR module on the game server 120. The clients 110 will communicate with each other using a data stream hierarchy protocol, for example, while playing a parallel reality game. In one embodiment, each client 110 includes a local AR module and the game server 120 includes a master AR module. Each local AR module transmits AR data to local AR modules on other clients 110 and / or the master AR module on the game server 120. Figure 2 Further described in .

[0021] Game server 120 is similarly a computing device that hosts the parallel reality game played by various clients 110. Game server 120 establishes connections between clients 110 to receive and transmit information about the state of the parallel reality game. Game server 120 stores player profiles, game states, maps (real and virtual), and other information related to the parallel reality game in game database 115. Game server 120 will Figure 3 Further described in .

[0022] Cell tower 130 is a network configured for data communication between clients 110. Cell tower 130 can support data transfer between clients 110, for example, via UDP. In one or more embodiments, cell tower 130 has a local AR module that is configured to help host parallel reality games between clients 110 connected to cell tower 130. Cell tower 130 can share game state to game server 120, for example, periodically or in real time. Figure 4 Further described in .

[0023] LAN 140 is a local network to which clients 110 can connect. LAN 140 includes one or more routers to which clients 110 can connect. LAN 140 operates using Wi-Fi and Ethernet. Similar to cell tower 130, LAN 140 can include a local AR module that is configured to help host parallel reality games between clients 110 connected to the LAN. LAN 140 can also share game state with game server 120.

[0024] Clients 110 may communicate with each other using a data stream hierarchy protocol or other protocols in other embodiments. To communicate with each other, two or more clients transmit data via one or more paths. Figure 1 As shown, there are three paths in the computer network 100 between the client 110A and the client 110B. In the first path, the clients 110 can communicate with each other via the game server 120, for example, using TCP. In the second path, the clients 110 can communicate with each other via the cell tower 130, for example, using TCP or UDP. In the third path, the clients 110 can communicate with each other via the LAN 140, for example, using TCP or UDP. In some other embodiments (not shown), there may be other paths for communication between the clients, for example, via Ethernet, Bluetooth, near field communication (NFC), etc. In each of these paths, the client 110 communicates via an intermediate node, for example, a game server 120, a cell tower 130, or a LAN 140. The data stream hierarchy protocol evaluates the available communication paths between the clients 110 and optimizes one or more paths used when transmitting data between the clients 110.

[0025] In one embodiment, client 110 communication can be routed through game server 120 or peer-to-peer (P2P). Communication routed through game server 120 can reach game server 120 from first client 110A via cell tower 130, and then return to second client 110B via cell tower 130. In contrast, P2P communication can be from first client 110A to cell tower 130, and then directly to second client 110B. Please note that in some cases, communication may pass through other intermediate devices, such as signal boosters. As used herein, if communication is routed to target client 110B without passing through game server 120, the communication is considered to be P2P. For example, if target client 110B is connected to the same cell tower 130 as sending client 110A, messages (e.g., datagrams) can be sent P2P, otherwise messages are routed via game server 120. The same cellular tower 130 as sending client 110A, otherwise routed through game server 120. In another embodiment, the clients 110 use P2P communication entirely. In addition, in some embodiments, UDP hole punching or other techniques for P2P communication involving network address translation (NAT) can be used to establish a connection between two or more clients 110.

[0026] In one embodiment, the clients 110 synchronize IP addresses using a coordination service (e.g., hosted at the game server 120 and communicating via TCP). The clients 110 can then communicate using a public-facing IP address or the LAN 140 (e.g., via UDP). For example, a first client 110A can send a request to the coordination service via TCP to join a local AR shared environment. The coordination service can provide the first client 110A with the IP address of a second client 110B connected to the AR environment (e.g., via the same cell tower 130). The coordination service can also provide the first client's IP address to the second client 110B, or the first client 110A can directly use the second client's IP address to provide it (as provided by the coordination service). In some embodiments, the coordination service can prompt the second client 110B to approve the first client 110A before providing the second client's IP address (e.g., by requesting user confirmation or checking a list of clients 110 approved to connect to the second client 110B).

[0027] In one embodiment, client 110 communicates via UDP to a public-facing IP address, and the communication is protected using public key cryptography. In a particular embodiment, elliptic curve cryptography (ECC) is implemented. ECC uses elliptic curves over a finite field that are one-way and deterministic functions to encrypt private keys that are publicly available to other computing systems and networks. One benefit of ECC over other public key cryptography methods is that ECC is a simpler function, allowing for smaller key sizes, thereby reducing bandwidth usage.

[0028] Client

[0029] Figure 2 1 is a block diagram of a client 110 according to an embodiment. The client 110 can be any portable computing device that can be used by a player to interface with a game server 120. For example, the client 110 can be a wireless device, a personal digital assistant (PDA), a portable gaming device, a cellular phone, a smart phone, a tablet computer, a navigation system, a handheld GPS system, a wearable computing device, a display with one or more processors, or other such devices. In another example, the client 110 includes a traditional computer system, such as a desktop or laptop computer. However, the client 110 can be a vehicle with a computing device. In short, the client 110 can be any computer device or system that enables a player to interact with a game server 120. As a computing device, the client 110 can include one or more processors and one or more computer-readable storage media. The computer-readable storage medium can store instructions that cause the processor to perform operations. The client 110 is preferably a portable computing device that can be easily carried by a player or otherwise transported, such as a smart phone or a tablet computer.

[0030] Client 110 communicates with game server 120 to provide sensory data of the physical environment to game server 120. Client 110 includes camera component 210 that captures image data in two dimensions of a scene in the physical environment where client 110 is located. Client 110 also includes a depth estimation model, which is a machine learning model, such as a machine learning model trained by game server 120. Figure 1In the illustrated embodiment, each client 110 includes software components, such as a game module 220 and a positioning module 230. The client 110 may include various other input / output devices for receiving information from a player and / or providing information to a player. Example input / output devices include a display screen, a touch screen, a touch pad, data input keys, a speaker, and a microphone suitable for voice recognition. The client 110 may also include various other sensors for recording data from the client 110, including but not limited to motion sensors, accelerometers, gyroscopes, other inertial measurement units (IMUs), barometers, positioning systems, thermometers, light sensors, etc. The client 110 may also include a network interface for providing communication via a network. The network interface may include any suitable components for interfacing with one or more networks, including, for example, a transmitter, a receiver, a port, a controller, an antenna, or other suitable components.

[0031] The camera assembly 210 captures image data of the scene of the environment in which the client 110 is located. The camera assembly 210 can utilize various different light sensors with different color capture ranges at different capture rates. The camera assembly 210 can include a wide-angle lens or a telephoto lens. The camera assembly 210 can be configured to capture a single image or a video as image data. In addition, the orientation of the camera assembly 210 can be parallel to the ground, and the camera assembly 210 is aimed at the horizon. The camera assembly 210 captures image data and shares the image data with the computing device on the client 110. The image data can be attached with metadata describing other details of the image data, including sensing data (e.g., temperature, brightness of the environment) or capture data (e.g., exposure, warmth, shutter speed, focal length, shooting time, etc.). The camera assembly 210 may include one or more cameras that can capture image data. In one instance, the camera assembly 210 includes one camera and is configured to capture monocular image data. In another instance, the camera assembly 210 includes two cameras and is configured to capture stereoscopic image data. In various other implementations, the camera assembly 210 includes multiple cameras, each of which is configured to capture image data.

[0032] The game module 220 provides an interface for players to participate in the parallel reality game. The game server 120 transmits the game data to the client 110 for use by the game module 220 at the client 110 to provide a local version of the game to players at a location remote from the game server 120. The game server 120 may include a computer for providing Figure 1 A network interface may include any suitable component for interfacing with one or more networks, including, for example, a transmitter, a receiver, a port, a controller, an antenna, or other suitable component.

[0033] The game module 220 executed by the client 110 provides an interface between the player and the parallel reality game. The game module 220 can present a user interface on a display device associated with the client 110, which displays a virtual world associated with the game (e.g., an image of a rendered virtual world) and allows the user to interact in the virtual world to perform various game goals. In some other embodiments, the game module 220 presents image data from the real world (e.g., captured by the camera component 210), which is enhanced by virtual elements from the parallel reality game. In these embodiments, the game module 220 can generate virtual content and / or adjust virtual content based on other information received from other components of the client. For example, the game module 220 can adjust the virtual object to be displayed on the user interface based on the depth map of the scene captured in the image data (e.g., determined by a depth estimation model).

[0034] The game module 220 can also control various other outputs to allow the player to interact with the game without the player looking at the display screen. For example, the game module 220 can control various audio, vibration or other notifications that allow the player to play the game without looking at the display screen. The game module 220 can access the game data received from the game server 120 to provide an accurate representation of the game to the user. The game module 220 can receive and process player input and provide updates to the game server 120 via the network. The game module 220 can also generate and / or adjust game content to be displayed by the client 110. For example, the game module 220 can generate virtual elements based on depth information (e.g., determined by a depth estimation model).

[0035] The positioning module 230 can be any device or circuit system for monitoring the location of the client 110. For example, the positioning module 230 can determine the actual or relative location by using a satellite navigation positioning system (e.g., GPS system, Galileo positioning system, Global Navigation Satellite System (GLONASS), Beidou Satellite Navigation and Positioning System), an inertial navigation system, a dead reckoning system, based on an IP address, by using triangulation and / or proximity to a cell tower or Wi-Fi hotspot and / or other suitable techniques for determining positioning. The positioning module 230 can also include various other sensors that can help accurately locate the location of the client 110. In some embodiments, the client 110 locally stores a portion of a map used in the parallel reality game. The positioning module 230 can locate the client 110 relative to the locally stored map. The positioning module 230 can also periodically synchronize the locally stored (or cached) map with the map data maintained by the game server 120.

[0036] When the player moves around with the client 110 in the real world, the positioning module 230 tracks the player's location and provides the player's location information to the game module 220. The game module 220 updates the player's location in the virtual world associated with the game based on the actual location of the player in the real world. Therefore, the player can interact with the virtual world simply by carrying or transporting the client 110 in the real world. In particular, the position of the player in the virtual world can correspond to the position of the player in the real world. The game module 220 can provide the player's location information to the game server 120 through the network. In response, the game server 120 can formulate various technologies to verify the client 110 position to prevent cheaters from spoofing the client 110 position. It should be understood that the location information associated with the player is only used when the player has been notified that the player's location information is to be accessed and how the location information will be used in the context of the game (such as updating the player's location in the virtual world). In addition, any location information associated with the player will be stored and maintained in a way that protects the player's privacy.

[0037] The data flow module 240 communicates with other computing components. The data flow module 240 can use any type of wired and / or wireless connection, using various communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML, JSON), and / or protection schemes (e.g., VPN, secure HTTP, SSL).

[0038] In one or more embodiments, the data stream module 240 communicates with the game server 120. The data stream module 240 provides actions, updates to the game state, requests for various content items (e.g., for parallel reality games), or other communications to the game server 120. The data stream module 240 establishes a connection with the game server 120, e.g., via TCP. TCP provides a robust way to transmit data. Using TCP, the data stream module 240 can detect lost packets during the transmission of data and request to resend the lost packets. In addition, TCP allows the reconstruction of sequential packets received out of order to generate expected messages.

[0039] The data flow module 240 also manages communications with one or more other clients 110. The data flow module 240 identifies one or more paths for communicating with another client 110. The paths available to a client 110 at a given time may include any combination of the following: a path through the game server 120, a path through the cell tower 130, a path through the LAN 140, another path through another network, and a direct connection (e.g., via Bluetooth). The data flow module 240 implements a data flow hierarchy protocol to select one or more available paths for use in transmitting data between clients 110. In the case of a single available path, the data flow hierarchy protocol instructs data to be transmitted via the uniquely identified path.

[0040] In the case of having multiple available paths, the data flow hierarchy protocol selects one or more paths in the available paths to improve the latency of the communication data. The data flow module 240 calculates one or more metrics (i.e., performance metrics) of the available paths, which indicate the data transmission performance through the available paths. Metrics include latency (ping), jitter, data loss, connection strength (stability of the connection), bandwidth, another data transmission metric, etc. In one embodiment, the data flow hierarchy protocol selects the path (or a certain number of paths) with the lowest latency for transmitting data. In some other embodiments, the data flow hierarchy protocol considers one or more other metrics as an additional or alternative to latency. For example, the data flow hierarchy protocol can combine one or more metrics for each path to determine the overall score (i.e., path score) for each path. The path score can be used to rank the available paths and select a path (e.g., the highest ranked path) or multiple paths (e.g., the highest ranked path of a certain number).

[0041] If multiple paths are selected based on the data stream hierarchy protocol, the selected multiple paths can each be used to transmit a separate copy of the same data to increase the data transmission rate. However, in this case, it is necessary to weigh the use of more paths because sending multiple copies of the same data via different paths will increase bandwidth usage. In some embodiments, the data stream hierarchy protocol periodically samples the available paths to re-evaluate which (which) paths are optimal for use. For example, the data stream hierarchy protocol evaluates the available paths at fixed intervals (e.g., every 30 seconds) to determine the optimal set of one or more paths for use in communication. The data stream hierarchy protocol can sample the available paths by transmitting data to the endpoints of the available paths (such as from client 110A to client 110B). Additionally or alternatively, the data stream hierarchy protocol can sample one or more available paths by transmitting data to an intermediate node connected to many endpoints, such as from client 110A to game server 120 or cell tower 130. In some embodiments, the data stream hierarchy protocol selects all available paths for use in communication. The data stream hierarchy protocol can adjust the path selection according to bandwidth availability. For example, in the case of high bandwidth availability, the data stream hierarchy protocol can instruct to select all available paths. In contrast, when bandwidth availability is limited, a data flow hierarchy protocol may select an optimal path (eg, a lowest latency path).

[0042] In some embodiments, the data flow module 240 encrypts data transmitted between clients, such as during a shared AR experience in a parallel reality game. The data flow module 240 can implement various cryptographic methods. In a specific implementation, ECC is implemented to protect UDP transmissions between clients 110, for example, via cell tower 130 or via LAN 140.

[0043] Game Server

[0044] Figure 3 1 is a block diagram illustrating a game server according to an embodiment. Game server 120 may be any computing device and may include one or more processors and one or more computer-readable storage media. The computer-readable storage medium may store instructions that cause the processor to perform operations. Game server 120 may include or may communicate with game database 115. Game database 115 stores game data used in parallel reality games to be supplied or provided to (multiple) clients 110.

[0045] The game data stored in the game database 115 may include: (1) data associated with the virtual world in the parallel reality game (e.g., image data used to render the virtual world on a display device, geographic coordinates of locations in the virtual world, etc.); (2) data associated with players of the parallel reality game (e.g., player profiles, including but not limited to player information, player experience level, player currency, current player position in the virtual world / real world, player energy level, player preferences, team information, faction information, etc.); (3) data associated with game objectives (e.g., data associated with the current game objective, the status of the game objective, past game objectives, future game objectives, expected game objectives, etc.); (4) data associated with virtual elements in the virtual world (e.g., the positioning of virtual elements); , the type of virtual element, the game objective associated with the virtual element; the corresponding real-world location information for the virtual element; the behavior of the virtual element, the relevance of the virtual element, etc.); (5) data associated with real-world objects, landmarks, and locations linked to virtual world elements (e.g., the location of real-world objects / landmarks, descriptions of real-world objects / landmarks, the relevance of virtual elements linked to real-world objects, etc.); (6) game status (e.g., the number of current players, the current state of game objectives, player leaderboards, etc.); (7) data associated with player actions / inputs (e.g., current player location, past player locations, player movements, player inputs, player queries, player communications, etc.); and (8) any other data used, related to, or obtained during the implementation of the parallel reality game. The game data stored in the game database 115 can be populated offline or in real time by a system administrator and / or by data received from the client 110.

[0046] The game server 120 may be configured to receive requests for game data from the clients 110 (e.g., via remote procedure calls (RPCs)) and to respond to those requests. For example, the game server 120 may encode the game data in one or more data files and provide the data files to the clients 110. In addition, the game server 120 may be configured to receive game data (e.g., player positions, player actions, player input, etc.) from the clients 110. For example, the clients 110 may be configured to periodically send player input and other updates to the game server 120, which uses the player input and other updates to update the game data in the game database 115 to reflect any and all changed conditions for the game.

[0047] In the illustrated embodiment, server 120 includes universal game module 310, data collection module 320 and event module 330. As mentioned above, game server 120 interacts with game database 115, and game database 115 can be a part of game server 120 or be accessed remotely (for example, game database 115 can be a distributed database accessed through a network). In some other embodiments, game server 120 comprises different and / or additional elements. In addition, function can be distributed between elements in a different manner than described. For example, game database 115 can be integrated into game server 120.

[0048] The universal game module 310 hosts parallel reality games for all players, and serves as an authoritative source for the current state of parallel reality games for all players. As a host, the universal game module 310 generates game content to present to the player, for example, via their respective clients 110. The universal game module 310 can access the game database 115 to obtain and / or store game data when hosting parallel reality games. The universal game module 310 also receives game data (such as depth information, player input, player positioning, player action, landmark information, etc.) from the client 110, and merges the received game data into the overall parallel reality game for all players of the parallel reality game. The universal game module 310 can also manage the transmission of game data to the client 110 through the network. The universal game module 310 can also manage the security aspects of the client 110, including but not limited to protecting the connection between the client 110 and the game server 120, establishing a connection between each client 110, and verifying the position of each client 110.

[0049] The game server 120 may also include a data collection module 320. In an embodiment including a data collection module 320, the data collection module 320 may be separate from or part of the general game module 310. The data collection module 320 may manage various game functions within the parallel reality game that include links to data collection activities in the real world. For example, the data collection module 320 may modify game data stored in the game database 115 to include game features linked to data collection activities in the parallel reality game. The data collection module 320 may also analyze data collected by players based on data collection activities and provide the data for access by various platforms.

[0050] The event module 330 manages player access to events in the parallel reality game. Although the term "event" is used for convenience, it should be understood that the term does not necessarily refer to a specific event at a specific location or time. Rather, it can refer to any provision for access-controlled game content, where one or more access criteria are used to determine whether a player can access the content. Such content can be part of a larger parallel reality game that includes game content with less or no access controls, or can be a stand-alone, access-controlled parallel reality game.

[0051] Community Tower

[0052] Figure 4 is a block diagram illustrating a cell tower according to an embodiment. In the illustrated embodiment, the cell tower 130 includes a routing module 410, a data ingestion module 420, an AR environment module 430, a map processing module 440, a permission check module 450, and a local data repository 460. The cell tower 130 also includes hardware and firmware or software (not shown) for establishing a connection with the game server 120 and the client 110 to exchange data. For example, the cell tower 130 can be connected to the game server 120 via an optical fiber or other wired Internet connection and to the client 110 using a wireless connection (e.g., 4G or 5G). In some other embodiments, the cell tower 130 may include different or additional components. In addition, the functions may be distributed between the elements in a manner different from that described. In one or more embodiments, the LAN 140 may include modules similar to those described for the cell tower 130, so that the LAN can also help host parallel reality games.

[0053] The routing module 410 receives data packets and sends those packets to one or more receiving devices. In one embodiment, the routing module 410 receives datagrams from the client 110 and sends the received datagrams to the appropriate destination. The routing module 410 may also receive data packets from the server that are addressed to a specific client 110 or to all clients connected to the cell tower 130. The routing module 410 forwards the data packets to the client 110 to which they are addressed. In some embodiments, the routing module 410 receives information (e.g., information within the data packet) indicating a path selected by the sending client 110 for routing the data packets, such as directly to another client 110 or through a game server 120. In this case, the routing module 410 routes the data packets using the selected path.

[0054] The data ingestion module 420 receives data from one or more sources, which the cell tower 130 uses to provide a shared AR experience to players via connected clients 110. In one embodiment, the data ingestion module 420 receives real-time or substantially real-time information about real-world conditions (e.g., from a third-party service). For example, the data ingestion module 420 may periodically (e.g., hourly) receive weather data from a weather service that indicates weather conditions in a geographic area around the cell tower. As another example, the data ingestion module 420 may retrieve opening hours for a park, museum, or other public space. As yet another example, the data ingestion module 420 may receive traffic data indicating how many vehicles are traveling on roads in a geographic area around the cell tower 130. Such information about real-world conditions may be used to improve synergy between the virtual world and the real world.

[0055] The AR environment module 430 manages an AR environment in which players in a geographic area around the cell tower 130 can participate in a shared AR experience. In one embodiment, the client 110 is connected to the cell tower 130 while executing a parallel reality game, and the AR environment module 430 connects the client to the AR environment for the game. All players of the game connected to the cell tower 130 can share a single AR environment, or the players can be divided into multiple AR environments. For example, there can be a maximum number of players (e.g., ten, twenty, one hundred, etc.) in a particular AR environment. In the presence of multiple AR environments, the newly connected client 110 can be randomly placed in the session, or the client can provide a user interface (UI) to enable the player to choose which session to join. Therefore, players can choose to participate in the AR environment with friends. In some embodiments, players can establish access to a protected private AR environment (e.g., a password or code is required to join).

[0056] In various embodiments, in order to make AR objects (e.g., creatures, vehicles, etc.) appear to interact with real-world features (e.g., jump over obstacles rather than pass through them), the AR environment module 430 provides map data representing the client's vicinity (e.g., stored in the local data store 460) to the connected client 110. The AR environment module 430 can receive location data (e.g., GPS location) for the client 110 and provide map data for a geographic area surrounding the client (e.g., within a threshold distance of the client's current location).

[0057] The received map data may include one or more representations of the real world. For example, the map data may include a point cloud model, a plane matching model, a line matching model, a geographic information system (GIS) model, a building recognition model, a landscape recognition model, etc. The map data may also include more than one representation of a given type at different levels of detail. For example, the map data may include two or more point cloud models, each including a different number of points.

[0058] Client 110 can compare the map data with data collected by one or more sensors to refine the client's position. For example, by mapping an image captured by a camera on client 110 to a point cloud model, the client's position and orientation can be accurately determined (e.g., within one centimeter and 0.1 degrees). Client 110 provides the determined position and orientation back to AR environment module 430 along with any actions taken by the player (e.g., shooting, selecting a virtual item to interact with, dropping a virtual item, etc.). Thus, AR environment module 430 can update the state of the game for all players participating in the AR environment.

[0059] The map processing module 440 updates the map data based on current conditions (e.g., data from the data ingestion module 420). Because the real world is not static, the map data in the local data repository 460 may not represent current real-world conditions. For example, the same park trail in Vermont may look very different in different seasons. In the summer, the trail may be clear and the surrounding trees may be covered with leaves. In contrast, in the winter, the trail may be blocked by snow and the trees may be bare. The map processing module 440 can transform the map data to approximate this change.

[0060] In one embodiment, the map processing module 440 retrieves current condition data to identify a transformation and applies the transformation to the map data. The transformations for different conditions can be defined by heuristic rules, in the form of a trained machine learning model, or using a combination of both approaches. For example, the map processing module 440 can receive current weather condition data, select a transformation for the current weather conditions, and apply the transformation to the map data. Alternatively, the map processing module 440 can pre-compute the transformed maps and store them (e.g., in a local data repository 460). In this case, when the client 110 connects to a cell tower, the map processing module determines the current conditions, selects an appropriate pre-computed version of the map data, and provides the version to the client.

[0061] The permission check module 450 maintains synchronization between the game states of different clients 110. In one embodiment, the permission check module 450 confirms that the game actions received from the client 110 are consistent with the game state maintained by the AR environment module 430. For example, if two players both attempt to pick up the same in-game item, the permission check module 450 determines which player receives the item (e.g., based on a timestamp associated with the request). As described above, the use of a P2P protocol and local processing at the cell tower can significantly reduce the latency of a player's actions seen on other players' clients 110. Therefore, the likelihood (and number) of such instances of conflicts occurring and being resolved by the permission check module 450 is reduced. As a result, the AR experience can be improved.

[0062] The permission check module 450 may also maintain synchronization between its copy of the state of the AR environment (the intermediate node state) and the master state maintained by the game server 120. In one embodiment, the permission check module 450 periodically (e.g., every 1 to 10 seconds) receives global updates about the state of the AR environment from the game server 120. The permission check module 450 compares these updates to the intermediate node state and resolves any differences. For example, if a player's request to pick up an item is initially approved by the permission check module 450, but a game update from the game server 120 indicates that the item was picked up by another player (or otherwise became unavailable) before the player attempted to pick it up, the permission check module 450 may send an update to the player's client 110 indicating that the item should be removed from the player's inventory.

[0063] This process may provide value to clients 110 located close to a boundary between coverage provided by two or more different cell towers 130. In this case, players whose client devices 110 are connected to different cell towers 130 may all be able to interact with the same virtual element. Thus, each individual cell tower 130 may initially approve conflicting interactions with the element from different clients 110. In this case, the server 120 may detect the conflict after synchronizing with the different cell towers 130 and send an update to resolve the conflict (e.g., instructing one of the cell towers to revoke its initial approval of the action and update its local state accordingly).

[0064] The local data repository 460 is one or more non-transitory computer-readable media configured to store data used by the cell tower. In one embodiment, the stored data may include map data, current condition data, a list of currently (or recently) connected clients 110, a local copy of the game state for the geographic area, etc. Although the local data repository 460 is shown as a single entity, the data may be split across multiple storage media. In addition, some of the data may be stored elsewhere in the communication network and accessed remotely. For example, the cell tower 130 may access the current condition data remotely (e.g., from a third-party server) as needed.

[0065] Example Method

[0066] Figure 5 A process 500 for using a data flow hierarchy protocol is shown according to an embodiment. Figure 5 The steps are shown from the perspective of the client 110A executing the method 500. However, some or all of the steps may be performed by other entities or components. In addition, some embodiments may perform the steps in parallel, in a different order, or in different steps.

[0067] The client 110A identifies 510 one or more available network paths for data transmission. The client 110A identifies such network paths by determining which of a set of predetermined paths to the target client 110B is currently available to the client 110A. If the client 110A can establish a connection with the game server 120 and the target client 110B is also connected to the game server 120, the client 110A identifies a network path to the other client 110 via the game server 120. If the client 110A is connected to a cell tower 130, the client 110 may determine whether the other client 110B is also connected to the cell tower 130. If so, the client 110A identifies another network path through the cell tower 130. In addition, if the client 110A and the other client 110B are connected to the same LAN 140, the client 110A may identify a network path through the LAN 140. The client 110A may identify other networks or paths connecting the two clients 110 as other available network paths (e.g., a direct connection between the clients 110, such as a Bluetooth connection). One or more of the network paths may implement UDP for data transmission. For the UDP path, the client 110 may utilize public key cryptography for encryption of data. In particular, the client 110 may implement ECC.

[0068] The client 110A detects 520 one or more performance metrics for each of the available network paths. The client 110A detects the performance metrics by sending a test data packet via the available network path. The data packet may request another test data packet to be sent back from another client 110B. Based on the successful transmission of these test packets, the client 110A may calculate (or detect) metrics for the network path. Metrics include latency (ping), jitter, data loss, connection strength, bandwidth availability, etc.

[0069] The client 110A selects 530 one or more available network paths for use in data transmission based on the detected metrics. The client 110A may consider one or more specific metrics when selecting a network path. In one embodiment, the client 110A considers only latency (ping). In another embodiment, the client 110A considers latency and connection strength. As previously described, the client 110A may select one or more "best" paths (as indicated by one or more metrics) or also select all available network paths for use in data transmission.

[0070] Client 110A transmits 540 data via the selected one or more network paths. In the context of a parallel reality game, client 110A transmits game data to other clients 110 and game server 120. Client-to-client communication is particularly efficient when maintaining a shared AR experience in a parallel reality game, where multiple players on their clients 110 may be interacting with each other and with the game server 120. Some of these interactions may be hosted locally by the client 110 or other network (e.g., cell tower 130 or LAN 140) so that the game state does not need to be synchronized by the game server 120. Locally hosted AR experiences provide a more streamlined AR experience with low latency.

[0071] Client 110A may iterate process 500 to periodically sample available paths and iteratively optimize which paths to use in maintaining data transmissions between devices.

[0072] Figure 6 A method 600 for transmitting data between clients 110 in a geographic area around a cell tower is shown according to an embodiment. The method 600 is described from the perspective of a cell tower 130 that performs the method 600 to provide P2P communication between client devices 110 in the geographic area. Figure 6 However, some or all of these steps may be performed by other entities or components. In addition, some embodiments may perform these steps in parallel, perform these steps in different orders, or perform different steps.

[0073] exist Figure 6In the illustrated embodiment, a cell tower 130 provides 610 P2P communications to clients 110 within a geographic area around the cell tower via a network path. For example, the cell tower 130 may provide P2P communications to clients 110 to provide a parallel reality game (e.g., hosted by a game server 120). The cell tower 130 receives 620 data from a first client 110A via the network path. In particular, the network path is selected by the first client 110A from available network paths based on one or more metrics for each of the available network paths. For example, the first client device may select a network path that provides P2P communications using a data stream hierarchy protocol, as described above. The data received from the first client 110A may be associated with virtual content of the AR environment, such as AR data corresponding to a geo-location within the geographic area.

[0074] Using the network path, the cell tower 130 transmits 630 the data to a second client 110B within the geographic area. For example, if the data describes an update to the virtual content of an AR environment associated with a parallel reality game, the cell tower 130 may provide the update to some or all of the clients 110 in the client segment 110 of the parallel reality game connected to the cell tower 130. As another example, the data may be specifically addressed to the second client 110B, such as a message or interaction with the virtual content associated with a player using the second client.

[0075] Sample Computer

[0076] Figure 7 is a diagram showing a suitable Figure 1 A high-level block diagram of an example computer 700 for use within a computer network is shown. The example computer 700 includes at least one processor 702 coupled to a chipset 704. The chipset 704 includes a memory controller hub 720 and an input / output (I / O) controller hub 722. Memory 706 and a graphics adapter 712 are coupled to the memory controller hub 720, and a display 718 is coupled to the graphics adapter 712. Storage devices 708, keyboard 710, pointing device 714, and network adapter 716 are coupled to the I / O controller hub 722. Other embodiments of the computer 700 have different architectures.

[0077] exist Figure 7In the illustrated embodiment, storage device 708 is a non-transitory computer-readable storage medium, such as a hard drive, a compact disk read-only memory (CD-ROM), a DVD, or a solid-state storage device. Memory 706 stores instructions and data used by processor 702. Pointing device 714 is a mouse, trackball, touch screen, or other type of pointing device and is used in conjunction with keyboard 710 (which may be an on-screen keyboard) to enter data into computer system 700. Graphics adapter 712 displays images and other information on display 718. Network adapter 716 couples computer system 700 to one or more computer networks.

[0078] Depend on Figure 1 The type of computer used by the entity may vary depending on the embodiment and the processing power required by the entity. For example, the game server 120 may include a distributed database system that includes multiple blade servers that work together to provide the described functionality. In addition, the computer may lack some of the above components, such as keyboard 710, graphics adapter 712, and display 718.

[0079] Those skilled in the art may make various uses and modifications and departures to the devices and techniques disclosed herein without departing from the described concepts. For example, the components or features shown or described in this disclosure are not limited to the positions, settings or contexts shown or described. Examples of devices according to this disclosure may include all, fewer or different components compared to those described with reference to one or more of the aforementioned figures. Therefore, this disclosure is not limited to the specific implementations described herein, but is to be given the widest possible scope consistent with the attached claims and their equivalents.

[0080] Additional considerations

[0081] The description of the above embodiments is given for the purpose of illustration; it is not intended to be exhaustive or to limit the patent rights to the precise form disclosed. Those skilled in the relevant art will appreciate that many modifications and variations are possible in light of the above disclosure.

[0082] Certain parts of this specification describe embodiments in terms of algorithms and symbolic representations of operations on information. These algorithmic descriptions and representations are often used by those skilled in the art of data processing to effectively convey the essence of their work to other technicians in the field. Although these operations are described functionally, computationally, or logically, they are understood to be implemented by computer programs or equivalent circuits, microcodes, etc. In addition, without loss of generality, it sometimes proves convenient to refer to the arrangement of these operations as modules. The described operations and their associated modules may be embodied in software, firmware, hardware, or any combination thereof.

[0083] Any steps, operations or processes described herein may be performed or implemented using one or more hardware or software modules, alone or in combination with other devices. In one embodiment, the software module is implemented using a computer program product that includes a computer-readable medium containing computer program code that can be executed by a computer processor to perform any or all of the steps, operations or processes described.

[0084] Embodiments may also relate to apparatus for performing the operations herein. The apparatus may be specially constructed for the desired purpose, and / or it may include a general-purpose computing device selectively activated or reconfigured by a computer program stored in a computer. Such a computer program may be stored in a non-transitory, tangible computer-readable storage medium, or any type of medium suitable for storing electronic instructions, which may be coupled to a computer system bus. In addition, any computing system mentioned in this specification may include a single processor or may be an architecture that employs multiple processor designs to increase computing power.

[0085] Embodiments may also relate to products produced by the computing processes described herein. Such products may include information produced by the computing processes, wherein the information is stored on a non-transitory, tangible computer-readable storage medium and may include any embodiment of a computer program product or other data combination described herein.

[0086] As used herein, any reference to "one embodiment" or "an embodiment" means that a particular element, feature, structure, or characteristic described in conjunction with the embodiment is included in at least one embodiment. The phrase "in one embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment. Similarly, the use of "a" or "an" before an element or component is merely for convenience. The description should be understood to mean that there are one or more elements or components unless it is clear that it has another meaning.

[0087] Where a value is described as "approximately" or "substantially" (or derivatives thereof), such value should be interpreted as being accurate to + / - 10%, unless another meaning is apparent from the context. For example, "approximately ten" should be understood to mean "within the range of from nine to eleven."

[0088] As used herein, the terms "comprise," "comprising," "include," "including," "have," "having," or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. In addition, unless expressly stated to the contrary, "or" refers to an inclusive or and not an exclusive or. For example, a condition A or B is satisfied by any of the following: A is true (or present) and B is false (or absent), A is false (or absent) and B is true (or present), and both A and B are true (or present).

[0089] After reading this disclosure, those skilled in the art will appreciate additional alternative structural and functional designs that may be used to employ the described techniques and methods. Therefore, although specific embodiments and applications have been illustrated and described, it should be understood that the described subject matter is not limited to the precise configurations and components disclosed. The scope of protection shall be limited only by the appended claims.

[0090] Finally, the language used in the specification is selected primarily for readability and instructional purposes, and it may not be selected to delineate or limit the patent rights. Therefore, it is intended that the scope of the patent rights be limited not by this detailed description, but by any claims issued on an application based thereon. Therefore, the disclosure of the embodiments is intended to illustrate, but not to limit, the scope of the patent rights set forth in the appended claims.

Claims

1. A non-transitory computer-readable medium storing instructions that, when executed by a computing device, cause the computing device to perform operations comprising: identifying an available network path for data transmission, the available network path passing through a plurality of intermediate nodes, the plurality of intermediate nodes including a cell tower and a server corresponding to an interactive application; detecting one or more performance metrics for each of the available network paths through the plurality of intermediate nodes; selecting, based on the detected one or more performance metrics, a first available network path from the available network paths for use in data transmission; transmitting data via the first available network path, the data describing an interaction with the interactive application, wherein the interaction is directed to an action on geo-located data of the interactive application; receiving, from the cell tower, information describing a local state of the geolocated data after transmitting the data describing the action, the information indicating whether the local state was successfully updated by the cell tower to reflect the action, wherein the cell tower generates the information describing the local state by performing steps comprising: maintaining the local state of the geo-located data for the interactive application, the geo-located data associated with a geo-location within a geographic area surrounding the cell tower; comparing the action to the local state of the geolocated data; In response to determining, based on the comparison, that the action does not conflict with the local state, updating the local state to reflect the action; and transmitting information describing the local state of the geolocated data; transmitting to the cell tower data describing additional actions associated with the geolocated data; receiving, from the cell tower, information describing a conflict between the additional action and the local state of the geolocated data, wherein the cell tower generates the information describing the conflict by executing steps comprising: receiving, from a server, information describing a master status of geolocated data maintained by the server; synchronizing the local state of the geolocated data with the master state; comparing the additional action to the local state of the geolocated data; In response to determining, based on the comparing, that the additional action conflicts with the local state of the geolocated data, rejecting the additional action; and Information describing the local status of conflicts of the geolocated data is generated.

2. The non-transitory computer-readable medium of claim 1 , wherein selecting the first available network path comprises: determining a score for each of the available network paths using the detected one or more performance metrics; ranking the available network paths according to the determined scores; as well as The first available network path is selected using the ranking.

3. The non-transitory computer-readable medium of claim 1, wherein the one or more performance metrics include latency, and wherein selecting the first available network path further comprises: Determining that a delay for the first available network path is less than a delay for one or more other available network paths; as well as The first available network path is selected based on the determination.

4. The non-transitory computer readable medium of claim 1, wherein the one or more performance metrics include bandwidth, and wherein the operations further comprise: In response to determining that a bandwidth of a set of one or more of the available network paths is above a bandwidth threshold, selecting each of the one or more available network paths; as well as The data is transmitted via each of the one or more available network paths.

5. The non-transitory computer readable medium of claim 1 , wherein the operations further comprise: re-detecting the one or more performance metrics for each of the available network paths; reselecting another available network path from the available network paths for use in data transmission based on the redetected one or more performance metrics; as well as The data is transmitted via the reselected another available network path. 6 . The non-transitory computer-readable medium of claim 1 , wherein the available network path comprises a path through a local area network (LAN).

7. The non-transitory computer-readable medium of claim 1, wherein the available network path is used to transmit the data between a client and another client, and wherein transmitting the data comprises transmitting the data by the client to the another client via the first available network path.

8. The non-transitory computer readable medium of claim 7, wherein the operations further comprise: requesting, by the client from a network coordination service, one or more network addresses associated with a shared network environment, the shared network environment including network addresses for client devices within a geographic area; In response to the request, receiving a network address of the other client from the network coordination service; and The data is transmitted via the first available network path using the network address of the other client.

9. The non-transitory computer-readable medium of claim 7, wherein the client and the other client are associated with a public network address, and wherein transmitting the data comprises cryptographically protecting the data via public key encryption.

10. The non-transitory computer-readable medium of claim 7, wherein the interactive application corresponds to the geo-located data, the geo-located data being associated with a geo-location within the geographic area.

11. The non-transitory computer readable medium of claim 1 , wherein transmitting the data further comprises: selecting an additional available network path from the available network paths based on the detected one or more performance metrics; as well as Additional copies of the data are transmitted via the additional available network paths.

12. A computer-implemented method comprising: providing peer-to-peer network communications by a cell tower to clients within a geographic area surrounding the cell tower via a network path among a plurality of available network paths, the plurality of available network paths passing through a plurality of intermediate nodes, the plurality of intermediate nodes including the cell tower and a server corresponding to an interactive application; receiving, from a first client within the geographic region, data for transmission via the selected network path, the data describing an interaction with the interactive application by the first client, wherein the first client selected the network path based on one or more performance metrics for each of the plurality of available network paths, wherein the interaction with the interactive application is directed to an action on the geographically located data of the interactive application; transmitting the data by the cell tower to a second client within the geographic area according to the selected network path; maintaining a local state of the geo-located data for the interactive application at the cell tower, the geo-located data being associated with a geo-location within the geographic area surrounding the cell tower; comparing the action to the local state of the geolocated data; In response to determining, based on the comparison, that the action does not conflict with the local state, updating the local state to reflect the action; transmitting information describing an update of the local state of the geolocated data to the second client according to the selected network path; receiving, by the cell tower, from the server associated with the interactive application, information describing a master status of geo-located data maintained by the server; synchronizing the local state of the geolocated data with the master state; receiving, from the first client, data describing an additional action associated with the geolocated data; comparing the additional action to the local state of the geolocated data; in response to determining, based on the comparing, that the additional action conflicts with the local state of the geolocated data, rejecting the additional action; as well as Information describing the local status of conflicts of the geolocated data is transmitted to the first client.

13. The computer-implemented method of claim 12, wherein the master state reflects further actions taken by a third client device within an overlap area between the geographic area and an additional geographic area around an additional cell tower, and wherein the synchronizing further comprises: determining, by said cell tower, that said further action conflicts with said action; In response to determining that the further action conflicts with the action, updating the local state to reflect the further action instead of the action; and Information describing the updated local state of the geolocated data is transmitted to the first client.

14. The computer-implemented method of claim 12, wherein the first client selects the network path by: determining a score for each of the plurality of available network paths using the detected one or more performance metrics; ranking the plurality of available network paths according to the determined scores; as well as The ranking is used to select the available network path.

15. The computer-implemented method of claim 12, wherein the one or more performance metrics include one or more of: latency, jitter, data loss, connection stability, or bandwidth.

16. A system comprising: server; Cell tower; as well as A first client device connected to a second client device via a plurality of network paths, the plurality of network paths comprising a first network path through the server, a second network path through the cell tower, and a third network path through a local area network, the first client device being configured to: detecting one or more performance metrics for each of the plurality of network paths; selecting an available network path from the plurality of network paths for use in data transmission based on the detected one or more performance metrics; transmitting data to the second client device via the available network path, the data describing an interaction with an interactive application, wherein the interaction is directed to an action on geo-located data of the interactive application; Wherein, the cell tower is configured as: providing, by a cell tower, peer-to-peer network communications to clients within a geographic area surrounding the cell tower via a network path among the plurality of network paths; receiving, from the first client within the geographic area, data for transmission via the available network paths, the data describing the interaction with the interactive application by the first client, wherein the first client selects the available network paths based on one or more performance metrics for each of the plurality of network paths; transmitting the data by the cell tower to the second client within the geographic area according to the available network path; maintaining a local state of the geo-located data for the interactive application at the cell tower, the geo-located data being associated with a geo-location within the geographic area surrounding the cell tower; receiving, from the first client, information describing the action associated with the geolocated data; comparing the action to the local state of the geolocated data; In response to determining, based on the comparison, that the action does not conflict with the local state, updating the local state to reflect the action; transmitting information describing an update of the local state of the geolocated data to the second client according to the available network path; receiving, from the server, information describing a master status of geolocated data maintained by the server; synchronizing the local state of the geolocated data with the master state; receiving, from the first client, data describing an additional action associated with the geolocated data; comparing the additional action to the local state of the geolocated data; In response to determining, based on the comparing, that the additional action conflicts with the local state of the geolocated data, rejecting the additional action; and Information describing the local status of conflicts of the geolocated data is transmitted to the first client.