Augmented reality mobile edge computing

Abstract

The processing of shared actions in an augmented reality experience is distributed between an edge node (e.g., a base station) and a server within a communication network. Therefore, the computation of the current state can be naturally fragmented based on real-world location, where state updates are generally provided by the edge node and the server provides conflict resolution based on a master state (e.g., where actions connected to different edge nodes may interfere with each other). In this way, latency is reduced because peer-to-peer (P2P) protocols are used to transmit these actions between user terminals connected to the same edge node without routing game actions via a game server.

Description

Technical Field

[0001] The present invention relates to augmented reality (AR), and more particularly, to a layered network architecture for providing a low-latency shared AR experience. Prior Art

[0002] In a parallel reality game system, players interact via computing devices in a shared virtual world that parallels at least a portion of the real world. A player's position in the virtual world is based on their position in the real world. However, when actions are primarily based on the virtual world, the virtual world can become an obstacle for players engaging with each other. Players interact with the virtual world in a way that makes other players appear distant or unreal. Therefore, even if multiple individuals play in a geographical area in the real world, the gaming experience can feel individual rather than social.

[0003] A parallel reality game system may use one or more protocols to update the game state of the virtual world, allowing a player to interact with the virtual world to reflect changes (e.g., the consequences of actions taken by other players in the game). Communication protocols, such as the User Datagram Protocol (UDP), define a system of rules for exchanging data using computer networks. UDP relies on a connectionless communication model without guaranteed delivery, ordering, or non-duplication of datagrams. Computing devices communicating using UDP transmit datagrams to each other over a computer network. Datagrams are the basic communication unit, each consisting of a header and a payload.

[0004] Connectionless communication protocols, such as UDP, generally have lower overhead and latency than connection-oriented communication protocols, such as the Transmission Control Protocol (TCP), which establish connections between computing devices before transmitting data. However, existing connectionless communication protocols are not well-suited for data transmission requiring less latency than accommodated by existing technologies. For example, streaming a parallel live gaming session at 60 frames per second (FPS) may require latency an order of magnitude lower than that provided by current technologies. In such a gaming session, frames are spaced at approximately 16 millisecond intervals, while current communication protocols typically provide latency of approximately 100 milliseconds (or more).

[0005] Therefore, the latency of these existing connectionless communication protocols becomes a barrier between players and the virtual world in parallel reality game systems. Using these existing communication protocols, a player cannot interact with the current game state, but only with the most recent game state. For example, in a parallel reality game, a player may see a virtual object at an old position (e.g., where the object was 100 milliseconds ago), while the virtual position data actually contains a new position for the virtual object (e.g., the virtual object has been moved by another player). This communication latency between the client and a server hosting or coordinating the parallel reality game can result in a frustrating user experience. This problem is particularly acute when more than one user is participating in the game, as the latency can cause a noticeable delay between a player's actions appearing in the other players' views of the virtual world. Summary of the Invention

[0006] Augmented reality (AR) systems use computer-generated content to supplement the view of the real world. Incorporating AR into a parallel reality game can improve the integration between the real and virtual worlds. AR can also increase player interactivity by giving players the opportunity to participate in a shared gaming experience where players interact with each other. For example, in a tank battle game, players can navigate virtual tanks around a real-world location in an attempt to destroy each other's tanks. In another example, in a tag battle game, players can attempt to tag each other using energy balls to score points.

[0007] Conventional AR session technology involves a game server maintaining a master game state and periodically synchronizing the local game state of each player's device with the master game state via a network (e.g., the internet). However, synchronizing a player's local game state can take a significant amount of time (e.g., approximately 100 milliseconds), which can be detrimental to the gaming experience. Players are effectively interacting with a past game state rather than the current one. This problem is particularly acute when multiple users are participating in an AR game, as latency introduces a noticeable delay between a player's actions appearing in the other players' views. For example, if one player moves an AR object in the virtual world, other players may not see the movement until 100 milliseconds (or more) later—a human-perceptible delay. Consequently, the other player may attempt to interact with the object in its previous location and become frustrated when the game corrects for the delay (e.g., by refusing to perform the player's requested action).

[0008] This and other issues can be addressed by performing some game state processing at an edge node in the network, such as a base station. This allows for natural sharding of game state computation based on real-world location, with the master game state maintained by the server to provide conflict resolution (for example, where the actions of players connected to nearby base stations might interfere with each other). Latency can also be reduced using peer-to-peer (P2P) protocols, which exchange updates between clients connected to the same edge node without routing them through a game server. For example, these approaches can be used to reduce latency to approximately 10 milliseconds or less. Furthermore, this can increase bandwidth availability, enabling more players to share a common AR experience.

[0009] In one embodiment, a method for providing a shared AR experience by an edge node includes receiving a connection request from a client at the edge node and identifying a shared AR experience for the client based on the connection request. The method also includes providing map data and a local game state for the shared AR experience to the client. The edge node receives an action request from the client indicating a desired interaction with a virtual item in the shared AR experience and determines an outcome of the action request based on a local game state maintained by the edge node. The method further includes providing the outcome to a plurality of clients connected to the edge node and verifying the outcome using a master game state maintained by a server. The outcome may be verified after initially providing the outcome to the plurality of clients. Simple diagram description

[0010] FIG. 1 illustrates an embodiment of a network computing environment in which AR content is provided to a client.

[0011] 2 is a block diagram of a datagram suitable for use within the network computing environment of FIG. 1 according to one embodiment.

[0012] FIG3 is a flow chart illustrating a process for routing a datagram by an edge node in a network computing environment according to one embodiment.

[0013] FIG. 4 is a block diagram of the base station in FIG. 1 according to an embodiment.

[0014] FIG5 is a flowchart illustrating a method for providing a shared AR experience according to an embodiment.

[0015] FIG6 is a block diagram illustrating an example of a computer suitable for use in the network computing environment of FIG1 according to one embodiment. Implementation Method

[0016] Cross-reference to related applications This application claims the benefit of U.S. Provisional Application No. 62 / 810,115, filed February 25, 2019, which is incorporated by reference.

[0017] The figures and the following description depict specific embodiments by way of illustration only. Those skilled in the art will readily appreciate from the following description that alternative embodiments of the structures and methods may be employed without departing from the principles described. Reference will now be made to several embodiments, examples of which are illustrated in the accompanying drawings. Similar or identical element numbers are used in the drawings to indicate similar or identical functions, as appropriate. Elements are similar or identical when they share a common number followed by a different letter. Numbers refer only to any one or any combination of such elements. Although the embodiments described below relate to a parallel reality game, those skilled in the art will appreciate that the present invention can be applied to other types of shared AR experiences. Instance computing environment

[0018] FIG1 illustrates an embodiment of a network computing environment that provides AR content to a client 130. The AR content may be part of a parallel reality game in which players interact in a shared virtual world that is parallel to at least a portion of the real world. In the illustrated embodiment, the network computing environment includes a server 110, a base station 120, and two clients 130. Although only two clients 130 are shown for convenience, in reality, many more (e.g., dozens or hundreds) of clients 130 may be connected to the base station 120. In other embodiments, the network computing environment may include different or additional components. For example, the network computing environment may be a local area network (LAN) that uses a WiFi router as an edge node instead of a base station 120. Furthermore, functionality may be distributed among the components in a manner different from that described.

[0019] Server 110 provides services to one or more computing devices of client 130 in a communication network. Server 110 communicates with client 130 via edge nodes in the communication network, such as base station 120. In one embodiment, server 110 maintains a master game state for a parallel reality game, i.e., the final ground-truth state of the parallel reality game. In this context, the master game state corresponds to the ground-truth state because it governs when conflicts arise between the master game state and a local game state. This is particularly useful when two local game states (e.g., two local game states maintained by adjacent base stations 120 in a network) correspond to overlapping geographic areas. Therefore, devices connected to two base stations 120 can interact with the same game content (in the master game state) while simultaneously connected to different AR sessions.

[0020] Base station 120 is an edge node of the communication network through which client 130 connects. As described above, a computer network may include other edge nodes in addition to or in place of base station 120 that facilitate similar communications. In one embodiment, a base station includes one or more computing devices configured to store AR data and provide AR services to connected client 130. Because client 130 is located relatively close to the base station and directly connected to it, the latency in providing data and services to the client can be significantly shorter than that of server 110. Various embodiments of base station 120 will be described in more detail below with reference to FIG4 .

[0021] Client 130 is a computing device, such as a personal computer, laptop, tablet, smartphone, or the like. In embodiments involving a parallel reality game, client 130 is typically a smartphone or tablet with a connected or built-in camera that is easily portable for the player. A client (e.g., client 130A) can communicate by sending P2P messages to base station 120, which forwards the P2P messages to other clients (e.g., client 130B) connected to the same base station. Clients 130 can also communicate with each other by sending messages to server 110 (via base station 120), which in turn forwards the messages to the recipients. For example, client 130A can send a message in this manner to another client 130B connected to a different base station 120 or server 110.

[0022] In one embodiment, client communications may be routed through server 110 or peer-to-peer (P2P). Communications routed through server 110 may travel from a first client 130A to server 110 via base station 120 and then return to a second client 130B via base station 120. In contrast, P2P communications may travel from first client 130A to base station 120 and then directly to second client 130B. It should be noted that in some cases, communications may pass through other intermediary devices (or nodes), such as broadcasters. As used herein, a communication is considered P2P if it is routed from a sending client 130A to a destination client 130B without passing through server 110. This can reduce latency by eliminating the need to send the communication to server 110 before sending it to destination client 130B. For example, a message (eg, a datagram) may be sent P2P when the target client 130B is connected to the same base station 120 as the sending client 130A, but otherwise routed through the server 110. In another embodiment, the clients 130 communicate entirely using P2P communication.

[0023] In one embodiment, clients 130 synchronize their IP addresses using a coordination service (e.g., hosted on a server and communicating via TCP). Thus, clients 130 can communicate using public IP addresses or a local area network (LAN) (e.g., via UDP). For example, a first client 130A can send a request to the coordination service via TCP to join a local shared AR experience. The coordination service can provide first client 130A with the IP address of a second client 130B connected to the AR session providing the AR experience (e.g., via the same base station 120). The coordination service can also provide the first client's IP address to the second client 130B, or the first client 130A can directly provide its IP address using the second client's IP address (e.g., as provided by the coordination service). In some embodiments, the coordination service may prompt the second client 130B to approve the first client 130A before providing the second client's IP address (eg, by requesting user confirmation or checking a list of approved clients 130 to which the second client 130B is to connect).

[0024] Among other advantages, structuring the network computing environment in the manner shown in FIG1 enables efficient distribution of computation. Information can be exchanged between clients 130 peer-to-peer with low latency, enabling players to interact in a shared AR experience in a manner similar to how they interact in the real world. Similarly, base station 120 can determine what map data to provide to client 130 and synchronize game states between connected clients 130. Base station 120 can synchronize the same state with less latency than would be incurred by using server 110 to synchronize game states globally. Server 110 can then be used to handle longer-term transactions and resolve conflicts. For example, server 110 can manage connections between geographic areas handled by different base stations 120, double-check (and optionally correct) decisions made by the base stations, perform additional security checks, analyze data received from clients 130 to detect cheating, maintain global data (e.g., team score totals for areas larger than the coverage area of ​​a single base station), and so on. Example P2P Agreement

[0025] FIG2 illustrates an example of a datagram 200 suitable for use in the network computing environment of FIG1 . As previously described, a datagram 200 is a basic communication unit. In the illustrated embodiment, datagram 200 includes a payload 202 and a header 204. Header 204 includes a P2P flag 206, also known as an indicator. Header 204 is metadata that specifies the state of datagram 200, such as a source port, a destination port, a length of datagram 200, and a checksum of datagram 200. Payload 202 is the data conveyed by datagram 200. In other embodiments, datagram 200 may include different or additional elements.

[0026] Payload 202 contains the contents of datagram 200 intended for one or more recipients 130. In one embodiment, header 204 may resemble a UDP header, except for P2P flag 206. Header 204 may also contain additional metadata. P2P flag 206 is used to determine whether datagram 200 is sent to server 110 or P2P to another client 130. In other embodiments, P2P flag 206 is replaced by one or more other indicators within the header that provide similar functionality.

[0027] FIG3 illustrates a flow chart of a process for utilizing a P2P communication protocol at an edge node (e.g., a base station 120) according to one embodiment. In FIG3 , base station 120 receives 305 a datagram 200 from a client 130A. Base station 120 analyzes 310 datagram 200 to determine whether it should be P2P or sent via server 110. In one embodiment, a P2P flag 206 indicates that datagram 200 is a P2P message, and base station 120 sends 315 datagram 200 to one or more clients 130 connected to client 130A. For example, if header 204 of datagram 200 indicates that the destination port is that of client 130B, base station 120 sends 315 datagram 200 to client 130B. Alternatively, the base station may maintain a list of connected clients 130 participating in the local AR session and send 315 the datagram 200 to all clients 130 participating in the local AR session (or a subset of clients, such as clients corresponding to a player's teammates). In contrast, if the P2P flag 206 indicates that the datagram 200 is intended for the server 110, the base station 120 sends 320 the datagram 200 to the server 110.

[0028] In another embodiment, the sending client 130A may set the P2P flag 206 to indicate that the datagram 200 should be sent P2P, if possible. The base station 120 receives 305 and analyzes 310 the datagram 200 and, assuming that the P2P flag 206 indicates that the datagram 200 should be sent P2P, determines whether a target client 130B is currently connected to the base station 120 (e.g., by comparing an identifier of the target client 130B with a list of currently connected clients 130). If the target client 130B is connected to the base station 120, the base station 120 sends 315 the datagram 200 directly to the target client 130B rather than via the server 110. In contrast, if the target client 130B is not connected to the base station 120, the base station 120 sends 320 the datagram 200 to the server 110 for forwarding to the target client 130B (e.g., via a second base station 120 to which it is currently connected). For example, the server 110 may maintain a database or other list of which base stations 120 are currently or recently connected to which client devices 130. In some embodiments, the base station 120 may send the datagram 200 to both the target client 130B and the server 110.

[0029] In other embodiments, the P2P flag 206 may be an identifier of an entity (such as an AR session, a user, a device, a gaming account, or the like). The base station 120 maintains a list of P2P flags 206 for datagrams 200 that should be sent P2P (or P2P, if applicable). The base station 120 receives 305 and analyzes 310 the datagram 200 to determine whether the datagram 200 should be sent via the server 110 or P2P. If the P2P flag 206 includes an identifier of an entity on the list, the datagram 200 is a P2P message, and the base station 120 sends 315 the datagram 200 to one or more clients 130B associated with the identified entity. For example, if the header 204 of the datagram 200 includes an identifier of the target client 130B, the base station 120 sends 315 the datagram 200 to the target client 130B. For another example, if the P2P flag 206 identifies an AR session, the datagram 200 is sent to all clients 130 connected to the session. If it is a gaming account, the datagram is sent to one or more clients associated with the gaming account, and so on. In contrast, if the P2P flag 206 identifies an entity not on the list, the base station 120 sends 320 the P2P flag 206 to the server 110 for forwarding to the client 130 associated with the entity. Alternatively, the list may indicate to the P2P flag 206 that information is not to be sent P2P, in which case the default behavior when the P2P flag 206 is not on the list is to P2P send the corresponding datagram to the target client 130B. Instance edge node

[0030] Figure 4 illustrates an embodiment of an edge node in a communications network. In the illustrated embodiment, the edge node is a base station 120, which includes a routing module 410, a data ingestion module 420, an AR session module 430, a map processing module 440, a permission check module 450, and a local data storage 460. Base station 120 also includes hardware and firmware or software (not shown) for establishing connections to server 110 and client 130 to exchange data. For example, base station 120 may connect to server 110 via a fiber optic or other wired Internet connection and to client 130 using a wireless connection (e.g., 4G or 5G). In other embodiments, base station 120 may include different or additional components. Furthermore, functionality may be distributed among components in a manner different from that described.

[0031] Routing module 410 receives data packets and sends them to one or more receiving devices. In one embodiment, routing module 410 receives datagram 200 from client 130 and uses the method previously described with reference to FIG3 to determine where to send the received datagram. Routing module 410 may also receive data packets from a server addressed to a specific client 130 or to all clients connected to base station 120. Routing module 410 forwards the data packet to the client 130 to which the data packet is addressed.

[0032] Data ingestion module 420 receives data from one or more sources that base station 120 uses to provide a shared AR experience to players via connected clients 130. In one embodiment, data ingestion module 420 receives real-time or substantially real-time information about real-world conditions (e.g., from a third-party service). For example, data ingestion module 420 may periodically (e.g., hourly) receive weather data from a weather service indicating weather conditions in the geographic area surrounding base station 120. As another example, data ingestion module 420 may retrieve the opening hours of a park, museum, or other public space. As yet another example, data ingestion module 420 may receive traffic data indicating how many vehicles are traveling on roads in the geographic area surrounding base station 120. This information about real-world conditions can be used to improve the synergy between the virtual and real worlds.

[0033] The AR session module 430 manages AR sessions in which players in the geographic area surrounding the base station 120 can participate in a shared AR experience. In one embodiment, a client 130 connects to the base station 120 while playing an AR game, and the AR session module 430 connects the client 130 to an AR session for game play. All game players connected to the base station 120 can share a single AR session, or players can be divided among multiple AR sessions. For example, a maximum number of players can exist in a particular AR session (e.g., 10, 20, 100, etc.). When multiple AR sessions exist, newly connected clients 130 can be randomly placed into a session, or a client 130 can provide a user interface (UI) that allows players to select which session to join. Thus, a player can choose to join an AR session with friends. In some embodiments, players can create private AR sessions that are access-protected (e.g., requiring a password or code to join).

[0034] In various embodiments, to enable AR objects (e.g., creatures, vehicles, etc.) to appear to interact with real-world features (e.g., jumping over obstacles rather than passing through them), the AR session module 430 provides map data representing the real world near the connected client 130 (e.g., stored in the local data storage 460). The AR session module 430 may receive location data (e.g., a GPS location) of a client 130 and provide map data of a geographic area surrounding the client (e.g., within a threshold distance of the client's current location).

[0035] Map data can include one or more different types of representations of the real world. For example, map data can include a point cloud model, a plane matching model, a line matching model, a geographic information system (GIS) model, a building recognition model, a landscape recognition model, and so on. Map data can also include more than one representation of a given type at different levels of detail. For example, map data can include two or more point cloud models, each containing a different number of points.

[0036] The client 130 can compare the map data with data collected by one or more sensors on the client 130 to refine the client's location. For example, the client's position and orientation can be accurately determined (e.g., to within 1 centimeter and 0.1 degree) by mapping an image captured by a camera on the client 130 onto a point cloud model. The client 130 provides the determined position and orientation, as well as any actions taken by the player (e.g., shooting, selecting a virtual item to interact with, dropping a virtual item, etc.), back to the AR session module 430. As a result, the AR session module 430 can update the game state of all players participating in the AR session to accurately reflect the players' positions within the AR session.

[0037] Map processing module 440 updates map data based on current conditions (e.g., data from data ingestion module 420). Because the real world is not static, the map data in local data storage 460 cannot represent current real-world conditions. For example, the same park trail in Vermont may look very different in different seasons. In summer, the trail is clear and surrounded by lush trees. In contrast, in winter, the trail is blocked by snow and the trees are bare. Map processing module 440 can transform the map data to account for these changes.

[0038] In one embodiment, the map processing module 440 retrieves current condition data to identify a transformation and applies the transformation to the map data. Transformations for different conditions can be defined by heuristic rules, in the form of a trained machine learning model, or using a combination of both approaches. For example, the map processing module 440 may receive current weather condition data, select a transformation for the current weather condition, and apply the transformation to the map data. Alternatively, the map processing module 440 may precompute the transformed map and store it (e.g., in the local data storage 460). In this case, when a client 130 connects to the base station 120, the map processing module 440 determines the current conditions, selects an appropriate precomputed form of map data, and provides it to the client 130.

[0039] The permission check module 450 maintains synchronization between the game states of different clients 130. In one embodiment, the permission check module 450 confirms that game actions received from a client 130 are consistent with the game state maintained by the AR session module 430. For example, if two players attempt to pick up the same in-game item, the permission check module 450 determines which player receives the item (e.g., based on a timestamp associated with the request). As described, using a P2P protocol and local processing at the base station 120 can significantly reduce the latency of a player's actions being seen by other players' clients 130 during an AR session by directly providing the results of an action requested by one player (e.g., grabbing a virtual object) to the other player's client 130. Consequently, the likelihood (and number) of instances where conflicts between actions arise and need to be resolved by the permission check module 450 can be reduced, improving the AR experience.

[0040] The permission check module 450 can also maintain synchronization between its local copy of the game state and a master game state maintained by the server 110. In one embodiment, the permission check module 450 periodically (e.g., every 1 to 10 seconds) receives global updates about the game state from the server 110. The permission check module 450 compares these updates with the local copy of the game state and resolves any inconsistencies. For example, if a player's request to pick up an item is initially approved by the permission check module 450, but a game update from the server 110 indicates that the item was picked up by another player before the player attempted to pick it up (or is otherwise unavailable), the permission check module 450 can send an update to the player's client 130 indicating that the item should be removed from the player's inventory.

[0041] This process can provide value to a client 130 located near a boundary between coverage areas provided by different base stations 120. In this case, two players connected to different base stations 120 can interact with the same virtual element. Therefore, each individual base station 120 may initially approve conflicting interactions with the element, but the server 110 will detect the conflict and send an update to resolve it (e.g., instructing a base station 120 to revoke its initial action approval and update its local game state accordingly).

[0042] Local data storage 460 is one or more non-transitory computer-readable media configured to store data used by base station 120. In one embodiment, the stored data may include map data, current status data, a list of currently (or recently) connected clients 130, a list of P2P flags, a local copy of the game state for a geographic area, and so on. Although local data storage 460 is shown as a single entity, the data may be distributed across multiple storage media. Furthermore, some data may be stored elsewhere in the communication network and accessed remotely. For example, base station 120 may optionally access current status data remotely (e.g., from a third-party server). Example Method

[0043] FIG5 illustrates a method 500 for providing a shared AR experience according to one embodiment. The steps of FIG5 are illustrated from the perspective of an edge node (e.g., base station 120) performing method 500. However, some or all of the steps may be performed by other entities or components. Furthermore, some embodiments may perform steps in parallel, in a different order, or at different times.

[0044] In the embodiment shown in FIG5 , method 500 begins when base station 120 receives 510 a connection request from a client 130 . The connection request may include one or more of an identifier for the client 130, an identifier for a game, a player identifier (e.g., a username), the current location of the client 130 (e.g., a GPS location), and so on. The connection request may be divided into one or more parts. For example, the client 130 may first establish a communication channel with base station 120 , and then an AR game running on the client 130 may send a request to join a shared AR session providing a shared AR experience.

[0045] The base station 120 identifies 520 a shared AR session for the client 130. The specific shared AR session may be selected based on the specific AR game running on the client 130, information about the player (e.g., the base station may facilitate connections between the player and other players with similar in-game profiles, etc.), the number of players already connected to the ongoing AR session, etc. In one embodiment, the client 130 identifies which shared AR session to join (e.g., based on player input), which is communicated to the base station 120. Note that in some cases, only a single shared AR session may be available. In such cases, the base station 120 may automatically connect the client 130 to that shared AR session.

[0046] Base station 120 provides 530 map data and a local game state for a shared AR session to client 130. In one embodiment, base station 120 uses location data (e.g., GPS data) received from client 130 to identify a subset of available map data to provide 530 to the client. For example, base station 120 may provide map data describing real-world locations within a threshold distance (e.g., within 100 meters) of client 130. As previously described, the map data may include one or more models representing the real-world geographic area surrounding client 130. Client 130 uses the map data to locate client 130 within the shared AR session and present an AR experience including AR content (e.g., virtual items, creatures, landmarks, etc.) based on the local game state.

[0047] Base station 120 receives 540 an action request from client 130. The action request may represent any desired interaction between the player and content within the AR experience (such as a virtual item, virtual character, virtual creature, or other player). For example, the player may desire to pick up a virtual item, drop a virtual item, use a virtual item, interact with a virtual item, talk to a virtual character, attack a virtual creature, attack another player, and so on. Those skilled in the art will recognize the various possible actions that can be performed based on the specific nature of the AR game.

[0048] Based on the game state, base station 120 determines 550 an outcome of the action request. The outcome can be based on a set of rules of varying complexity, depending on the specific action. For example, if the action is picking up a virtual item, the action is likely to succeed unless the base station determines that another player has already picked up, moved, or destroyed the item. As mentioned previously, this is unlikely to occur due to the reduced latency achieved by using the revealing technology. In contrast, if the action is attacking a virtual creature, the outcome can be based on a set of calculations and virtual tile characters. Those skilled in the art will recognize various possible rule sets and methods that can be used to determine the outcome of an action.

[0049] Base station 120 provides 560 the results of the action to connected clients 130. In one embodiment, base station 120 provides the results to all clients 130 connected to the AR session. In another embodiment, base station 120 only provides the results to clients 130 within a threshold real-world distance (e.g., 100 meters) of the player who performed the action. Because the latency introduced by the exposure technology is relatively low (e.g., 10 milliseconds or less), players can experience the results of their actions virtually instantly. As a result, players have the impression that they are directly interacting with each other in a shared AR experience.

[0050] Base station 120 verifies the results of actions 570 using a master game state maintained by server 110. In some embodiments, if an action is validated against the master game state, base station 120 sends an indication to server 110 to update the master game state to reflect any actions taken by players in its local game state. In one embodiment, base station 120 periodically (e.g., every 1 to 10 seconds) verifies that its local game state is consistent with the master game state. As previously described, this is particularly useful for handling situations at the edges of the area covered by base station 120, where actions by players connected to different base stations 120 could interfere with each other. If a conflict is detected between the local game state and the master game state, base station 120 may prompt the connected client 130 to update its game state to synchronize it with the master game state. Furthermore, in some embodiments, base station 120 may send an action and its result to the server for verification. If the server indicates that the result conflicts with the master game state, base station 120 updates the local game state to negate the result. Instance Computing System

[0051] FIG6 illustrates a high-level block diagram of an exemplary computer 600 suitable for use within the network computing environment 100 shown in FIG1 , according to one embodiment. Exemplary computer 600 includes at least one processor 602 coupled to a chipset 604. For convenience, processor 602 is referred to as a single entity, but it should be understood that corresponding functions can be distributed among multiple processors in various ways, including using multi-core processors, assigning specific operations to dedicated processors (e.g., graphics processing units), and dividing operations across a distributed computing environment. Any reference to a processor 602 should be interpreted as encompassing this architecture.

[0052] Chipset 604 includes a memory controller hub 620 and an input / output (I / O) controller hub 622. A memory 606 and a graphics adapter 612 are coupled to memory controller hub 620, and a display 618 is coupled to graphics adapter 612. A storage device 608, keyboard 610, pointing device 614, and network adapter 616 are coupled to I / O controller hub 622. Other embodiments of computer 600 have different architectures.

[0053] In the embodiment shown in FIG6 , storage device 608 is a non-transitory computer-readable storage medium, such as a hard drive, compact disc-read only memory (CD-ROM), DVD, or solid-state memory device. Memory 606 stores instructions and data used by processor 602. Pointing device 614 is a mouse, trackball, touchscreen, or other type of pointing device and is used in conjunction with keyboard 610 (which may be an on-screen keyboard) to input data into computer system 600. Graphics adapter 612 displays images and other information on display 618. Network adapter 616 couples computer system 600 to one or more computer networks.

[0054] The type of computer used by the entity of FIG. 1 may vary depending on the embodiment and the processing power required by the entity. For example, a server 110 may comprise a distributed database system comprising multiple servers working together to provide the described functionality. Furthermore, the computer may lack some of the aforementioned components, such as keyboard 610, graphics adapter 612, and display 618. Additional considerations

[0055] Portions of the above description describe embodiments in terms of algorithmic procedures or operations. Such algorithmic descriptions and representations are often used by those skilled in the data processing arts to effectively convey the essence of their operation to others skilled in the art. Although these operations are described functionally, computationally, or logically, they should be understood to be implemented by computer programs comprising instructions, microcode, or the like executed by a processor or equivalent circuitry. Furthermore, it has sometimes proven convenient to refer to these configurations of functional operations as modules, without loss of generality.

[0056] As used herein, any reference to "one embodiment" means that a particular element, feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearances of the phrase "in one embodiment" in various places in this specification do not necessarily all refer to the same embodiment.

[0057] The expressions "coupled" and "connected," along with their derivatives, may be used to describe some embodiments. It should be understood that these terms are not intended to be synonymous with each other. For example, the term "connected" may be used to describe some embodiments to indicate that two or more elements are in direct physical or electrical contact with each other. In another example, the term "coupled" may be used to describe some embodiments to indicate that two or more elements are in direct physical or electrical contact with each other. However, the term "coupled" may also mean that two or more elements are not in direct contact with each other, but yet still cooperate or interact with each other. The embodiments are not limited to the present invention.

[0058] As used herein, the terms "include," "comprising," "including," "having," or any other variations thereof are intended to cover a non-exclusive inclusion. For example, a process, method, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to the process, method, article, or apparatus. Furthermore, unless expressly stated to the contrary, "or" refers to an inclusive or and not an exclusive or. For example, a condition A or B satisfies any of the following: A is true (or exists) and B is false (or does not exist); A is false (or does not exist) and B is true (or exists); and both A and B are true (or exist).

[0059] In addition, the use of "a" or "an" is used to describe elements and components of the embodiments. This is merely for convenience and to give a general sense of the present invention. This description should be read to include one or at least one, and unless it is obvious that it is intended otherwise, the singular also includes the plural.

[0060] Finally, although specific embodiments and applications have been shown and described, it should be understood that these embodiments are illustrative only. Those skilled in the art will readily appreciate various modifications, variations, and alterations to the disclosed embodiments. Therefore, the disclosed embodiments should not be construed as limiting the scope of protection. Rather, the scope of protection shall be limited solely to the following claims.

[0061] 100: Network Computing Environment 110: Server 120: Base Station 130: User terminal 130A: User end 130B: User side 200: Information Report 202: Payload 204: Header 206: Peer-to-peer (P2P) flag 305: Receive 310: Analysis 315:Send 320:Send 410: Routing module 420:Data Ingestion Module 430: Augmented Reality (AR) Conversation Module 440: Map processing module 450: Permission check module 460: Local data storage 500:Method 510: Receive 520: Identification 530: Provide 540: Receive 550:Judgment 560: Provide 570: Verification 600: Computer Systems / Computers 602: Processor 604: Chipset 606: Memory 608: Storage device 610: Keyboard 612: Graphics Adapter 614:Indicator device 616: Network Adapter 618: Display 620:Memory Controller Center 622: Input / Output (I / O) Controller Center

Claims

1. A method for providing a shared augmented reality (AR) experience by an edge node of a communication network, the method comprising: At this edge node, a connection request is received from a user terminal. Based on the connection request, identify a shared AR session of the user terminal; provide the user terminal with map data and local state of an AR experience provided by the shared AR session, maintained by the edge node; receive an action request from the user terminal, the action request indicating an interaction with a virtual object in the AR experience; determine a result of the action request based on the local state of the AR experience; provide the result point-to-point to one or more of a plurality of user terminals connected to the edge node based on the satisfaction of a first predetermined condition associated with the action request; and provide the result to a server based on the satisfaction of a second predetermined condition associated with the action request.

2. The method of request item 1, wherein each of the first predetermined condition and the second predetermined condition is associated with a flag in a data packet corresponding to the action request.

3. The method of request item 2, wherein the edge node maintains a list of flags, and wherein the method further includes: The result is provided point-to-point to one or more of the plurality of user terminals based on whether the flag in the data packet corresponding to the action request is included in the flag list maintained by the edge node.

4. The method of request item 1, wherein the first predetermined condition and the second predetermined condition are associated with a location of the user terminal.

5. The method of claim 1, further comprising: After the result is provided peer-to-peer, it is verified using one of the master states of the AR experience maintained by the server.

6. The method of request 5, wherein the master state is computed using a plurality of different local states, each local state being maintained by a different edge node at a different real-world location.

7. As in request item 5, verifying the result using the master state of the AR experience includes: In response to receiving an indication from the server that the result conflicts with the master state, update the local state to revoke the result; and send an update to each of the plurality of user terminals to indicate that the result has been revoked.

8. The method of request item 5, wherein the verification includes checking that the result does not conflict with the master state.

9. The method of request item 1, wherein the edge node is connected to a plurality of user terminals and one of the base stations of the server.

10. The method of request 1, wherein the map data includes one or more of the following: a point cloud model, a plane matching model, a line matching model, a geographic information system (GIS) model, a building identification model, and a landscape identification model.

11. A method for providing a shared augmented reality (AR) experience by an edge node of a communication network, the method comprising: At this edge node, a connection request is received from a user terminal. Based on the connection request, identify a shared AR session of the user terminal; provide the user terminal with map data and a local state of an AR experience provided by the shared AR session, maintained by the edge node; receive an action request from the user terminal, the action request indicating interaction with one of the virtual objects in the AR experience; receive an additional action request from an additional user terminal of the shared AR session, the additional action request indicating interaction with one of the virtual objects in the AR experience; and determine the result of the action request from the user terminal and the result of the additional action request from the additional user terminal based on a comparison of a timestamp associated with the action request and a timestamp associated with the additional action request.

12. The method of claim 11, further comprising: The result of the action request and the result of the additional action request are provided point-to-point to multiple user terminals connected to the edge node.

13. The method of claim 12, further comprising: After providing these results, the results are verified using a master state of the AR experience maintained by a server.

14. The method of request item 13, wherein using the master state of the AR experience to verify the results includes: In response to receiving an indication from the server that the results conflict with the master state, update the local state to revert the results; And send an update to each of the plurality of user terminals to indicate that the results have been revoked.

15. A non-transitory computer-readable storage medium comprising instructions executable by a processor, the instructions including: Instructions for receiving a connection request from a user terminal at an edge node of a communication network; Instructions for identifying a shared AR session of the user terminal based on the connection request; instructions for providing the user terminal with map data maintained by the edge node and a local state of an AR experience provided by the shared AR session; instructions for receiving an action request from the user terminal, the action request indicating an interaction with a virtual object in the AR experience; instructions for determining a result of the action request based on the local state of the AR experience; instructions for providing the result point-to-point to one or more of a plurality of user terminals connected to the edge node based on the satisfaction of a first predetermined condition associated with the action request; and instructions for providing the result to a server based on the satisfaction of a second predetermined condition associated with the action request.

16. The non-transitory computer-readable storage medium of claim 15, wherein each of the first predetermined condition and the second predetermined condition is associated with a flag in a data packet corresponding to the action request.

17. The non-transitory computer-readable storage medium of claim 16, wherein the edge node maintains a flag list, and wherein such instructions further include: Instructions for providing the result point-to-point to one or more of the plurality of user terminals based on whether the flag in the data packet corresponding to the action request is included in the flag list maintained by the edge node.

18. A non-transitory computer-readable storage medium comprising instructions executable by a processor, the instructions including: Instructions for receiving a connection request from a user terminal at an edge node of a communication network; Instructions for identifying a shared AR session of the user terminal based on the connection request; instructions for providing map data and terminal status of an AR experience provided by the shared AR session, maintained by the edge node, to the user terminal; instructions for receiving an action request from the user terminal, the action request indicating interaction with one of the virtual objects in the AR experience; instructions for receiving an additional action request from an additional user terminal of the shared AR session, the additional action request indicating interaction with one of the virtual objects in the AR experience; and instructions for determining a result of the action request from the user terminal and from the additional user terminal based on a comparison of a timestamp associated with the action request and a timestamp associated with the additional action request.

Images