Repeatability predictions of interest points

Abstract

The present disclosure describes approaches for evaluating interest points for localization uses based on a repeatability of the detection of the interest point in images capturing a scene that includes the interest point. The repeatability of interest points is determined by using a trained repeatability model. The repeatability model is trained by analyzing a time series of images of a scene and determining repeatability functions for each interest point in the scene. The repeatability function is determined by identifying which images in the time series of images allowed for the detection of the interest point by an interest point detection model.

Description

[Technical Field]

[0001] The subject matter described is generally related to camera positioning, and more specifically, to determining the repeatability of a point of interest in order to evaluate the usability of a point of interest in camera positioning applications. [Previous Technology]

[0002] Problem

[0003] Camera positioning generally refers to the process of determining the position and orientation (pose) of a camera within an environment using images captured by the camera. Camera positioning has a wide and ever-expanding range of applications. In augmented reality (AR) applications, a virtual environment coexists with a real-world environment. If the pose of a camera capturing images of the real-world environment (e.g., a video feed) is accurately determined, virtual elements can be precisely overlaid on the depiction of the real-world environment. For example, a virtual hat can be placed on top of a real statue, a virtual character can be partially depicted behind a physical object, and so on.

[0004] Camera localization can be performed by identifying points of interest (POIs) within an image captured by a camera and mapping these POIs onto a 3D map. However, due to the constantly changing conditions in the real world, POIs may occasionally disappear from a scene. For example, a POI may disappear from time to time due to a moving vehicle passing in front of it. In another instance, lighting conditions may cause a model used to detect POIs to miss the identification of a POI within an image of a scene. Therefore, it is necessary to be able to determine how stable a POI will be in order to determine whether the POI is a suitable candidate for localization purposes. [Summary of the Invention]

[0005] This invention describes a method for evaluating interest points (OPs) for localization purposes based on detecting the repetition of OOPs in images of a scene containing OOPs. The repetition of OOPs is determined using a trained repetition model. The repetition model is trained by analyzing a time-series of images of a scene and determining the repetition function of each OOP in the scene. The repetition function is determined by identifying which images in the time-series are allowed to be detected by an OOP detection model.

[0006] The repetitive model can be used to generate a summary map by filtering out points of interest that are impossible to detect during a current time period. In addition, the repetitive model can be used to identify points of interest within an input image to query a 3D map for receiving information to execute a localization algorithm for determining the pose of a camera capturing the input image.

Implementation Method

[0017] Cross-reference to related applications

[0018] This application asserts the rights of U.S. Provisional Application No. 63 / 182,648, filed April 30, 2021, the entire contents of which are incorporated herein by reference.

[0019] The figures and the following description are for illustrative purposes only and represent certain embodiments. Those skilled in the art will recognize from the following description that alternative embodiments of the structure and method may be employed without departing from the principles described. Where feasible, similar or identical element symbols are used in the figures to indicate similar or identical functionality. Where elements share a common numeral followed by a different letter, this indicates that the elements are similar or identical. Unless the context otherwise indicates, references to numbers alone generally refer to any one or any combination of such elements.

[0020] Embodiments are described in the context of a parallel reality game that includes augmented reality content in a virtual world geography parallel to at least a portion of a real-world geography, such that a player's movement and actions in the real world affect actions in the virtual world. Furthermore, the inherent flexibility of computer-based systems allows for a variety of possible configurations, combinations, and divisions of system components and their tasks and functionalities. Example: Location-based parallel reality game

[0021] Figure 1 is a concept diagram of a virtual world 110 parallel to the real world 100. Virtual world 110 can serve as a game board for players in a parallel reality game. As illustrated, virtual world 110 includes a geography parallel to the geography of the real world 100. Specifically, a coordinate range defining a geographical area or space in the real world 100 is mapped to a corresponding coordinate range defining a virtual space in the virtual world 110. The coordinate range in the real world 100 can be associated with towns, neighborhoods, cities, campuses, venues, countries, continents, the globe, or other geographical regions. Each geographical coordinate within the geographical coordinate range is mapped to a corresponding coordinate in a virtual space within the virtual world 110.

[0022] A player's location in virtual world 110 corresponds to a player's location in real world 100. For example, player A, located at location 112 in real world 100, has a corresponding location 122 in virtual world 110. Similarly, player B, located at location 114 in real world 100, has a corresponding location 124 in virtual world 110. When a player moves within a geographic coordinate range in real world 100, the player also moves within a coordinate range defining the virtual space in virtual world 110. Specifically, a positioning system (e.g., a GPS system, a positioning system, or both) associated with a mobile computing device carried by the player can be used to track the player's location as the player moves within the geographic coordinate range in real world 100. Data associated with the player's location in real world 100 is used to update the player's location within the corresponding coordinate range defining the virtual space in virtual world 110. In this way, players can simply move within the corresponding geographical coordinates in the real world 100 and then travel along a continuous trajectory within the coordinates of the virtual space defined in the virtual world 110, without having to log in or periodically update location information at specific discrete locations in the real world 100.

[0023] Location-based games may include game objectives that require players to move to or interact with various virtual elements or objects scattered throughout a virtual world 110. A player can move to these virtual locations by moving to the corresponding location of the virtual element or object in the real world 100. For example, a positioning system can track the player's location so that when the player travels through the real world 100, the player also travels through the parallel virtual world 110. The player can then interact with various virtual elements or objects at specific locations to achieve or perform one or more game objectives.

[0024] A game objective allows a player to interact with virtual elements 130 located at various virtual locations within a virtual world 110. These virtual elements 130 may be linked to landmarks, geographical locations, or objects 140 in the real world 100. Real-world landmarks or objects 140 may be works of art, monuments, buildings, businesses, libraries, museums, or other suitable real-world landmarks or objects. Interactions include capturing, claiming ownership of, using a virtual item, and spending virtual currency. To capture such virtual elements 130, a player travels to a landmark or geographical location 140 linked to the virtual element 130 in the real world and performs any necessary interaction with the virtual element 130 in the virtual world 110 (as defined by the game rules). For example, player A may need to travel to one of the landmarks 140 in the real world 100 to interact with or capture one of the virtual elements 130 linked to that particular landmark 140. Interaction with virtual element 130 may require real-world actions, such as taking photos or verifying, obtaining or retrieving other information about landmarks or objects 140 associated with virtual element 130.

[0025] Game objectives may require players to use one or more virtual items collected by the player in a location-based game. For example, a player may travel through virtual world 110 in search of virtual items 132 (e.g., weapons, creatures, power-ups, or other items) that can be used to complete game objectives. These virtual items 132 can be discovered or collected by traveling to different locations in real world 100 or by performing various actions in virtual world 110 or real world 100 (such as interacting with virtual elements 130, fighting non-player characters or other players, or completing quests). In the example shown in Figure 1, a player uses virtual items 132 to acquire one or more virtual elements 130. Specifically, a player may deploy virtual items 132 in virtual world 110 near or within virtual elements 130. Deploying one or more virtual items 132 in this manner may result in the acquisition of virtual elements 130 by the player or by the player's team / faction.

[0026] In one particular implementation, a player may need to collect virtual energy as part of a parallel reality game. Virtual energy 150 may be distributed across different locations within the virtual world 110. A player can collect virtual energy 150 by moving to a location in the real world 100 corresponding to the location of the virtual energy in the virtual world 110 (or within a certain distance of that location). Virtual energy 150 can be used to power virtual objects or to perform various game objectives. A player who loses all of their virtual energy 150 may be disconnected from the game or prevented from playing for a certain amount of time or until they have collected additional virtual energy 150.

[0027] According to the present invention, a parallel reality game can be a large-scale, multiplayer, location-based game in which each participant in the game shares the same virtual world. Players can be divided into separate teams or factions and can cooperate to achieve one or more game objectives, such as acquiring or obtaining ownership of a virtual element. In this way, a parallel reality game can essentially be a social game that encourages cooperation among players within the game. Players from opposing teams can compete against each other (or sometimes cooperate to achieve a common goal) during a parallel reality game. A player can use virtual objects to attack or hinder the progress of players from opposing teams. In some cases, players are encouraged to gather at real-world locations to conduct cooperative or interactive events in the parallel reality game. In these cases, the game server attempts to ensure that players actually exist and are not faking their locations.

[0028] Figure 2 depicts one embodiment of a game interface 200, which may be presented as part of the interface between a player and a virtual world 110 (e.g., on a player's smartphone). The game interface 200 includes a display window 210 that can be used to display various other states of the virtual world 110 and the game, such as the player's location 122 and the locations of virtual elements 130, virtual items 132, and virtual energy 150 within the virtual world 110. The user interface 200 may also display other information, such as game data information, game communications, player information, client location verification instructions, and other information associated with the game. For example, the user interface may display player information 215, such as player name, experience level, and other information. The user interface 200 may include a menu 220 for accessing various game settings and other information associated with the game. The user interface 200 may also include a communication interface 230, which enables communication between the game system and the player, as well as between one or more players in a parallel reality game.

[0029] According to this invention, a player can interact with a parallel reality game by carrying a client device 310 in the real world. For example, a player can play the game by accessing an application associated with the parallel reality game on a smartphone and moving around in the real world with the smartphone. In this respect, the player does not need to continuously watch a visual representation of the virtual world on a display screen to play a location-based game. Therefore, the user interface 200 may include non-visual elements that allow a user to interact with the game. For example, the game interface may provide an audio notification to the player when the player approaches a virtual element or object in the game or when an important event occurs in the parallel reality game. In some embodiments, a player can use an audio control 240 to control such audio notifications. Different types of audio notifications may be provided to the user depending on the type of virtual element or event. The frequency or volume of the audio notification may be increased or decreased depending on the proximity of the player to a virtual element or object. It can provide users with other non-visual notifications and signals, such as a vibration notification or other suitable notifications or signals.

[0030] In some embodiments, the virtual world is shared by multiple players simultaneously. That is, for all users interacting in the same virtual world, virtual objects placed in the virtual world will appear in the same (or substantially the same) location. Furthermore, a user's interaction with virtual objects affects the gameplay of other users interacting in the same virtual world. For example, if a first player moves a virtual object from a first position to a second position, other players interacting in the same virtual world will experience the object moving from the first position to the second position.

[0031] In a parallel world setting where the location in the virtual world corresponds to the location in the real world, the location of a virtual object in the virtual world corresponds to its location in the real world. When interacting with the virtual world using an augmented reality application, it is beneficial to understand the position of the camera used to generate augmented reality content. Furthermore, since multiple users may be able to see and interact with the same virtual object through their respective augmented reality interfaces, it is also beneficial to understand the actual posture of the camera used to generate virtual reality content. That is, by understanding the position and posture of the camera used to generate augmented reality content, the positioning and orientation of the virtual content presented to the player within the augmented reality interface can be improved, resulting in a more accurate experience when multiple players interact with the same object in the virtual world simultaneously.

[0032] Parallel reality games may have various features to enhance and encourage gameplay within the game. For example, players may accumulate virtual currency or another virtual reward (e.g., virtual tokens, virtual points, virtual resources, etc.) that can be used throughout the game (e.g., to purchase in-game items, exchange for other items, craft items, etc.). As players complete one or more game objectives and gain experience within the game, they can progress through various levels. Players may also be able to acquire enhancements such as "powers" or virtual items that can be used to complete game objectives within the game.

[0033] Those skilled in the art will understand from the provided disclosure that numerous game interface configurations and basic functionalities are possible. Unless expressly stated otherwise, the present invention is not intended to be limited to any particular configuration. Example: Location-based Parallel Reality Game System

[0034] Embodiments are described in the context of a parallel reality game that includes augmented reality content in a virtual world geography parallel to at least a portion of a real-world geography, such that a player's movement and actions in the real world affect actions in the virtual world and vice versa. Those skilled in the art will understand using the disclosure provided herein that the described subject matter can be applied to other situations where it is desirable to determine the repetition of a point of interest within an image. Furthermore, the inherent flexibility of computer systems allows for a variety of possible configurations, combinations, and divisions of the system's components and their tasks and functionalities. For example, systems and methods according to the present invention can be implemented using a single computing device or across multiple computing devices (e.g., connected in a computer network).

[0035] Figure 3 illustrates one embodiment of a network-connected computing environment 300. The network-connected computing environment 300 uses a client-server architecture, wherein a game server 320 communicates with a client device 310 via a network 370 to provide a parallel reality game to a player at the client device 310. The network-connected computing environment 300 may also include other external systems, such as sponsor / advertiser systems or commercial systems. Although only one client device 310 is shown in Figure 3, any number of client devices 310 or other external systems can be connected to the game server 320 via the network 370. Furthermore, the network-connected computing environment 300 may contain different or additional components, and its functionality may be distributed between the client device 310 and the server 320 in a manner different from that described below.

[0036] The network-connected computing environment 300 provides interaction between players in a virtual world with a geographic location parallel to the real world. Specifically, a geographic region in the real world can be directly linked to or mapped to a corresponding region in the virtual world. A player can move around in the virtual world by moving to various geographic locations in the real world. For example, a player's location in the real world can be tracked and used to update the player's location in the virtual world. Typically, a player's location in the real world is determined by finding the location of the player through a client device 310 that is interacting with the virtual world and assuming that the player is in the same (or approximately the same) location. For example, in various embodiments, if the player's location in the real world is within a certain distance (e.g., 10 meters, 20 meters, etc.) of the real-world location corresponding to the virtual location of a virtual element in the virtual world, the player can interact with the virtual element. For convenience, various embodiments are described with reference to "player's location," but those skilled in the art will understand that such references may refer to the location of the player's client device 310.

[0037] The network-connected computing environment 300 uses a client-server architecture, wherein a game server 320 communicates with a client device 310 via a network 370 to provide a parallel reality game to the player at the client device 310. The network-connected computing environment 300 may also include other external systems, such as sponsor / advertiser systems or commercial systems. Although only one client device 310 is shown in Figure 3, any number of client devices 310 or other external systems can be connected to the game server 320 via the network 370. Furthermore, the network-connected computing environment 300 may contain different or additional components and its functionality may be distributed between the client device 310 and the server 320 in a manner different from one described below.

[0038] A client device 310 may be any portable computing device that can be used by a player to interface with the game server 320. For example, a client device 310 is preferably a portable wireless device that can be carried by a player, such as a smartphone, portable gaming device, augmented reality (AR) headset, cellular phone, tablet computer, personal digital assistant (PDA), navigation system, handheld GPS system, or other such devices. In some use cases, the client device 310 may be a less mobile device, such as a desktop or laptop computer. In addition, the client device 310 may be a vehicle with a built-in computing device.

[0039] The client device 310 communicates with the game server 320 to provide the game server 320 with sensing data of a physical environment. The client device 310 includes a camera assembly 312 that captures two-dimensional image data of a scene in the physical environment in which the client device 310 is located. In the embodiment shown in FIG3, each client device 310 includes software components, such as a game module 314 and a positioning module 316. The client device 310 also includes a positioning module 318. The client device 310 may include various other input / output devices for receiving information from a player and / or providing information to a player. Indicative input / output devices include a display screen, a touch screen, a touchpad, data input buttons, a speaker, and a microphone suitable for voice recognition. User terminal device 310 may also include various other sensors for recording data from user terminal device 310, including (but not limited to) motion sensors, accelerometers, gyroscopes, other inertial measurement units (IMUs), barometers, positioning systems, thermometers, light sensors, etc. User terminal device 310 may further include a network interface for providing communication via network 370. A network interface may include any suitable components for interfacing with one or more networks, including (for example) transmitters, receivers, ports, controllers, antennas, or other suitable components.

[0040] The camera assembly 312 includes one or more cameras capable of capturing image data. The cameras capture image data describing a scene of the environment surrounding the user device 310, which is in a specific posture (the camera's position and orientation within the environment). The camera assembly 312 may use various light sensors with different color capture ranges and different capture rates. Similarly, the camera assembly 312 may include cameras with a series of different lenses (such as a wide-angle lens or a telephoto lens). The camera assembly 312 can be configured to capture a single image or multiple images as a frame of a video.

[0041] Furthermore, the camera assembly 312 may be oriented parallel to the ground, with the camera assembly 312 aimed at the horizon. The camera assembly 312 captures image data and shares the image data with the computing device on the user terminal device 310. The image data may be supplemented with additional details describing the image data, including sensing data (e.g., temperature, ambient brightness) or captured data (e.g., exposure, warmth, shutter speed, focal length, capture time, etc.). The camera assembly 312 may include one or more cameras capable of capturing image data. In one example, the camera assembly 312 includes one camera configured to capture monocular image data. In another example, the camera assembly 312 includes two cameras configured to capture stereoscopic image data. In various other embodiments, the camera assembly 312 includes a plurality of cameras each configured to capture image data.

[0042] The user terminal device 310 may also include additional sensors for collecting data about the environment surrounding the user terminal device, such as motion sensors, accelerometers, gyroscopes, barometers, thermometers, light sensors, microphones, etc. The image data captured by the camera assembly 312 may be supplemented with additional information describing the image data (such as additional sensing data (e.g., temperature, ambient brightness, air pressure, position, posture, etc.)) or captured data (e.g., exposure length, shutter speed, focal length, capture time, etc.)).

[0043] Game module 314 provides a player with an interface for participating in a parallel reality game. Game server 320 transmits game data via network 370 to user device 310 for use by game module 314 at user device 310 to provide a local version of the game to a player located away from game server 320. Game server 320 may include a network interface for providing communication via network 370. A network interface may include any suitable components for interfacing with one or more networks, including (for example) transmitters, receivers, ports, controllers, antennas, or other suitable components.

[0044] Game module 314 provides a player with an interface for participating in a parallel reality game. Game server 320 transmits game data via network 370 to client device 310 for use by game module 314 to provide a local version of the game to a player located remotely from the game server. In one embodiment, game module 314 presents a user interface on a display of client device 310 that depicts a virtual world (e.g., presents images of the virtual world) and allows a user to interact with the virtual world to perform various game objectives. In some embodiments, game module 314 presents images of the real world enhanced with virtual elements from the parallel reality game (e.g., captured by camera assembly 312). In these embodiments, game module 314 may generate or adjust virtual content based on other information received from other components of client device 310. For example, game module 314 may adjust a virtual object to be displayed on the user interface based on a depth map of a scene captured from image data.

[0045] Game module 314 can also control various other outputs to allow a player to interact with the game without the player looking at a display screen. For example, game module 314 can control various audio, vibration, or other notifications that allow the player to play the game without looking at a display screen. Game module 314 can access game data received from game server 320 to provide the user with an accurate representation of the game. Game module 314 can receive and process player input and provide updates to game server 320 via network 370. Game module 314 can also generate and / or adjust game content to be displayed by user device 310. For example, game module 314 can generate a virtual element based on depth information.

[0046] The positioning module 316 can be any device or circuit system used to determine the location of the user terminal device 310. For example, the positioning module 316 can determine the actual or relative location by using a satellite navigation positioning system (e.g., a GPS system, a Galileo positioning system, a Global Navigation Satellite System (GLONASS), a BeiDou Navigation Satellite System), an inertial navigation system, a dead reckoning system, IP address analysis, triangulation and / or proximity cellular towers or Wi-Fi hotspots or other suitable technologies.

[0047] When a player moves around in the real world with the client device 310, the positioning module 316 tracks the player's location and provides the player's location information to the game module 314. The game module 314 updates the player's location in the virtual world associated with the game based on the player's actual location in the real world. Therefore, a player can easily interact with the virtual world by carrying or transporting the client device 310 in the real world. Specifically, the player's location in the virtual world can correspond to the player's location in the real world. The game module 314 can provide the player's location information to the game server 320 via the network 370. In response, the game server 320 can implement various technologies to verify the location of the client device 310 to prevent cheaters from spoofing their location. It should be understood that the location information associated with the player is only used after a player has been notified of the access to the player's location information and how the location information will be used in the context of the game (e.g., to update the player's location in the virtual world). In addition, any location information associated with a player will be stored and maintained in a manner that protects player privacy.

[0048] The positioning module 318 receives the position determined by the positioning module 316 for the user device 310 and refines the position by determining the pose of one or more cameras in the camera assembly 312. In one embodiment, the positioning module 318 uses the position generated by the positioning module 316 to select a 3D map of the environment surrounding the user device 310. The positioning module 318 may obtain the 3D map from local storage or from the game server 320. The 3D map may be a point cloud, a grid, or any other suitable 3D representation of the environment surrounding the user device 310.

[0049] In one embodiment, the positioning module 318 applies a trained model to determine the pose of the images captured by the camera assembly 312 relative to a 3D map. Therefore, the positioning model can accurately determine the position and orientation of the user device 310 (e.g., within a few centimeters and degrees). The position of the user device 310 can then be tracked over time using dead reckoning based on sensor readings, periodic repositioning, or a combination of both. Having an accurate pose of the user device 310 allows the game module 314 to present virtual content superimposed on real-world images (e.g., by displaying virtual elements on a display and real-time feeds from the camera assembly 312) or the real world itself (e.g., by displaying virtual elements on a transparent display of an AR headset) in a manner that gives the impression that virtual objects are interacting with the real world. For example, a virtual character might hide behind a real tree, a virtual hat might be placed on a real statue, or if a real person approaches a virtual creature too quickly, the virtual creature might run away and hide.

[0050] In some embodiments, the model used by the positioning module 318 is trained to determine the relative pose of the camera from one or more images captured by a camera relative to one or more existing images of the physical environment surrounding the user device 310. In one embodiment, the positioning module 318 may convert the relative pose into an absolute pose by referring to the known absolute pose of one or more existing images. For example, the game database 315 may store a set of reference images of the physical environment captured by cameras with different poses. The absolute pose of each reference image may be stored in association with the reference image (e.g., as meta-data). Therefore, once the pose of the user device 310's camera has been determined relative to one or more reference images, the absolute pose of the reference images and the relative pose of the camera can be used to determine the absolute pose of the user device's camera.

[0051] The game server 320 includes one or more computing devices that provide game functionality to the client device 310. The game server 320 may include a game database 330 or be able to communicate with a game database 330. The game database 330 stores game data used in parallel reality games for service or provision to the client device 310 via the network 370.

[0052] The game data stored in the game database 330 may include: (1) data associated with the virtual world in the parallel reality game (e.g., image data used to present the virtual world on a display device, geographical coordinates of the location in the virtual world, etc.); (2) data associated with the player in the parallel reality game (e.g., player profile, including (but not limited to) player information, player experience level, player currency, current player location in the virtual world / real world, player energy level, player preferences, team information, faction information, etc.); (3) data associated with the game objective (e.g., data associated with the current game objective, the status of the game objective, past game objectives, future game objectives, desired game objectives, etc.); (4) data associated with virtual elements in the virtual world (e.g., (5) Information related to the location of real-world objects, landmarks, and game objectives associated with virtual elements (e.g., the location of real-world objects / landmarks, descriptions of real-world objects / landmarks, and the correlation of virtual elements associated with real-world objects); (6) Game status (e.g., the current number of players, the current status of game objectives, player leaderboards, etc.); (7) Information related to player actions / inputs (e.g., current player location, past player location, player movement, player input, player queries, player communication, etc.); or (8) Any other information used, related to, or obtained during the implementation of parallel reality games. Game data stored in the game database 330 may be filled offline or in real-time by the system administrator or by data received from users of the system (e.g., players) (such as from a client device 310 via network 370).

[0053] Game server 320 can be configured to receive requests for game data from a client device 310 (e.g., via Remote Procedure Call (RPC)) and respond to such requests via network 370. For example, game server 320 can encode game data into one or more data files and provide such data files to client device 310. Additionally, game server 320 can be configured to receive game data (e.g., player position, player actions, player input, etc.) from client device 310 via network 370. For example, client device 310 can be configured to periodically send player input and other updates to game server 320, which uses these to update game data in game database 315 to reflect any and all changes in the game.

[0054] In the embodiment shown in FIG3, the game server 320 includes a general game module 322, a commercial game module 323, a data collection module 324, an event module 326, a repetitive training system 380, a repetitive model 385, a map drawing system 327, and a 3D map 328. As mentioned above, the game server 320 interacts with a game database 315 that can be partially or remotely accessed by the game server 320 (e.g., the game database 315 may be a distributed database accessed via a network 370). In other embodiments, the game server 320 includes different and / or additional components. Furthermore, functionality may be distributed among the components in a manner different from that described. For example, the game database 315 may be integrated into the game server 320.

[0055] The Universal Game Module 322 hosts one instance of a parallel reality game for a group of players (e.g., all players in a parallel reality game) and acts as the authoritative source of the current state of the parallel reality game for that group of players. As a host, the Universal Game Module 322 generates game content for (e.g., via its respective client devices 310) to present to the players. The Universal Game Module 322 can access the game database 330 to retrieve or store game data while hosting the parallel reality game. The Universal Game Module 322 can also receive game data (e.g., depth information, player input, player location, player actions, landmark information, etc.) from the client devices 310 and incorporate the received game data into the overall parallel reality game for the entire group of players in the parallel reality game. The Universal Game Module 322 can also manage the delivery of game data to the client devices 310 via the network 370. In some embodiments, the general game module 322 also manages the security of the interaction between the client device 310 and the parallel reality game, such as securing the connection between the client device and the game server 320, establishing connections between various client devices, or verifying the location of various client devices 310 to prevent players from cheating by faking their location.

[0056] The commercial game module 323 may be separate from or part of the general game module 322. The commercial game module 323 can manage various game features within the parallel reality game that are linked to a real-world business activity. For example, the commercial game module 323 may receive requests via network 370 from external systems such as sponsors / advertisers, businesses, or other entities to include game features linked to a real-world business activity. The commercial game module 323 can then configure the inclusion of these game features in the parallel reality game upon confirmation that the linked business activity has occurred. For instance, if a business pays an agreed amount to the provider of the parallel reality game, a virtual object identifying the business may appear in the parallel reality game at a virtual location corresponding to a real-world location of the business (e.g., a shop or restaurant).

[0057] The data collection module 324 can be separate from or part of the general game module 322. The data collection module 324 can manage various game features within the parallel reality game that are linked to a data collection activity in the real world. For example, the data collection module 324 can modify game data stored in the game database 330 to include game features linked to the data collection activity in the parallel reality game. The data collection module 324 can also analyze data collected by players according to data collection activities and provide the data for access on various platforms.

[0058] Event Module 326 manages player access to events in a parallel reality game. Although the term "event" is used for convenience, it should be understood that this term does not necessarily refer to a specific event at a specific location or time. Rather, it can refer to any layout of access-controlled game content, in which one or more access criteria are used to determine whether a player can access that content. This content can be a portion of a larger parallel reality game containing game content with less or no access control, or it can be a standalone access-controlled parallel reality game.

[0059] The repetitive training system 380 trains the repetitive model 385. The repetitive training system 380 receives image data for training the repetitive model 385. Typically, the repetitive training system 380 inputs a time-series image into the repetitive model 385 to generate or predict a repetitive function for a point of interest identified in the image. As used herein, a point of interest is a 3D point on an object and surface within a geographic area, having properties that allow the point to be robustly detected in its depiction (e.g., image or video). Points of interest can be depicted in an image (or video) and are equally likely to be identified within the image by recognizing the portion of the image depicting the point of interest. The repetitive training system 380 may use a supervised training algorithm to train the repetitive model 385. The repetitive training system 380 may define a total loss threshold for the repetitive model, which can be used to determine whether the repetitive model is sufficiently accurate in estimating a repetitive function.

[0060] Once the repeatability model 385 is trained, it receives data about a point of interest (e.g., image data depicting the point of interest) and outputs a prediction of the repeatability of the point of interest. As used herein, repeatability refers to the probability of detecting a point of interest in an image capturing the same scene but under different conditions (e.g., different times / dates, lighting conditions, angles, weather, etc.). In some embodiments, the prediction of repeatability is presented as a repeatability function indicating repeatability values ​​for time intervals within a defined time period.

[0061] Mapping system 327 generates a 3D map of one of the geographic regions based on a set of images. The 3D map may be a point cloud, a polygonal grid, or any other suitable representation of the 3D geometry of the geographic region. The 3D map may contain semantic tags that provide additional contextual information (such as identifying objects like tables, chairs, clocks, lampposts, trees, etc.), materials (concrete, water, bricks, grass, etc.), or game attributes (e.g., traversable by a character, suitable for certain in-game actions, etc.). In one embodiment, mapping system 327 stores the 3D map along with any semantic / contextual information in 3D map storage 328. The 3D map may be stored in 3D map storage 328 along with location information (e.g., GPS coordinates of the center of the 3D map, a ring grid defining the extent of the 3D map, or the like). Thus, game server 320 may provide 3D maps to client devices 310, which provide location data indicating their location within or near the geographic region covered by the 3D map.

[0062] The map drawing system 390 additionally identifies points of interest within the image (e.g., identifies a region of the image depicting the points of interest) and stores information about the points of interest in the 3D map 328. In some embodiments, the map drawing system 390 generates a 3D map by calculating the position of the points of interest based on multiple images depicting the points of interest from various angles.

[0063] The map rendering system 390 additionally generates a summary map 3D containing a subset of the data contained in the full 3D map. The summary map can be generated to allow transmission and storage of the map via a user device (e.g., a mobile device) with limited bandwidth or storage capacity. The summary map contains information relevant to the user device during a specific period of time. In some embodiments, as the user device moves or over time, the map rendering system 390 updates the summary map to remove points of interest that are no longer relevant (e.g., because the repetition value of a point of interest during a current period is below a threshold) and to add new points of interest that have become relevant.

[0064] Network 370 can be any type of communication network, such as a local area network (e.g., intranet), a wide area network (e.g., internet), or a combination thereof. The network may also include a direct connection between a client device 310 and a game server 320. Generally, communication between the game server 320 and a client device 310 can be conducted via a network interface using any type of wired or wireless connection, using various communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encoding or format (e.g., HTML, XML, JSON), or protection scheme (e.g., VPN, Secure HTTP, SSL).

[0065] This invention refers to servers, databases, software applications, and other computer-based systems, as well as the actions taken and the information sent to and from such systems. Those skilled in the art will recognize that the inherent flexibility of computer-based systems allows for a wide variety of possible configurations, combinations, and divisions of tasks and functionalities between and within components. For example, a program disclosed as being implemented by a single server can be implemented using a single server or multiple servers working in combination. Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0066] In situations where the disclosed systems and methods access and analyze personal information about a user, or utilize personal information (such as location information), the user may be provided with the opportunity to control whether a program or feature collects information and to control whether or how content is received from the system or other applications. This information or data will not be collected or used until the user has been provided with meaningful notification of what information will be collected and how it will be used. Information will not be collected or used unless the user provides consent, which may be withdrawn or modified by the user at any time. Therefore, the user can control how information about themselves is collected and how it is used by applications or systems. Furthermore, certain information or data may be processed in one or more ways before it is stored or used, such that personally identifiable information is removed. For example, a user's identity may be processed so that personally identifiable information cannot be determined for the user. Example Method

[0067] Figure 4A is a flowchart illustrating one iteration of a method 400 for training a repetitive model 385 using reference images according to one or more embodiments. During training, method 400 typically repeats the process many times using different input images until one or more training objectives are met. Method 400 generates a trained repetitive model 385 that can determine the repetitiveness of a point of interest depicted within an input image. The steps of method 400 are illustrated in Figure 4A from the perspective of the repetitive training system 380. However, some or all of the steps may be performed by other entities and / or components. Additionally, some embodiments may perform the steps in parallel, in a different order, or in different ways.

[0068] In the illustrated embodiment, method 400 begins with the repetitive training system 380 receiving 410 a series of training images (e.g., a series of RGB images captured by a camera of a user terminal device 310). In some embodiments, the series of training images is captured over a set time period (e.g., a day, a week, a month, or a year). Furthermore, each image in the series of training images is associated with a timestamp indicating when the image was captured. In some embodiments, the series of training images is extracted from one or more videos recorded over the set time period.

[0069] Figure 4B illustrates training images of a time series of a scene according to one or more embodiments. Specifically, Figure 4B illustrates six images of the same scene acquired at six different times (T0 to T5). In the example of Figure 4B, the first image 470A is associated with a first timestamp T0, the second image 470B is associated with a second timestamp T1, the third image 470C is associated with a third timestamp T2, the fourth image 470D is associated with a fourth timestamp T3, the fifth image 470E is associated with a fifth timestamp T4, and the sixth image 470F is associated with a sixth timestamp T5.

[0070] The repetitive training system 380 identifies 420 a set of interest points depicted within the received series of training images. In some embodiments, the repetitive training system 380 uses a trained model to detect interest points within an input image (e.g., identify portions of the input image depicting the interest points). In the example of FIG4B, the repetitive training system 380 identifies interest point 480A corresponding to a window of a building, interest point 480B corresponding to a pickup truck parked in front of a building, and interest point 480C corresponding to a car on a road.

[0071] For each identified point of interest, the repetitive training system 380 identifies 430 images within the series of training images in which the point of interest was detected. That is, for each image in the series of training images, the repetitive training system 380 determines whether the trained model used to detect the point of interest has detected a specific point of interest. Based on this identification, the repetitive training system 380 generates a repetitive function for 440 points of interest. In some embodiments, the repetitive function is further generated based on timestamps associated with each image in the series of training images. For example, the repetitive function indicates whether a point of interest was detected in one of the images in the training image set during various time intervals (e.g., every ten-minute period, every hour, morning / afternoon / evening / night, daily, monthly, etc.) of a set time period associated with the series of training images. Alternatively, if each time interval is associated with multiple images (e.g., if multiple images are associated with a timestamp within the same time interval), the repetitive function may indicate a percentage of images associated with a specific time interval in which the point of interest was detected.

[0072] For example, in the embodiment of FIG4B, the repetitive training system 380 generates a repetitive function for a first interest point 480A, a repetitive function for a second interest point 480B, and a repetitive function for a third interest point 480C. The repetitive function for the first interest point 480A can specify that the first interest point 480A is detected in all six images 470A to 470F (across timestamps T0 to T5). The repetitive function for the second interest point 480B can specify that the second interest point is detected in the first five images 470A to 470E (across timestamps T0 to T4). Finally, the repetitive function for the third interest point 480C can specify that the third interest point is detected only in the first image 470A (at timestamp T0).

[0073] Using the generated repetition function, the repetition training system 380 trains a repetition model 385 to estimate a repetition function of a point of interest detected in an input image. In some embodiments, the repetition model 385 is trained to determine the probability of detecting a time series of the point of interest in an image of a scene where the point of interest is located. That is, for each time interval within a set time period, the repetition model 385 is trained to predict the probability of detecting the point of interest in an image of a scene containing the point of interest and capturing the point of interest by a camera within a specific time interval.

[0074] Figure 5 is a flowchart illustrating a method 500 for constructing a 3D map 328 using a repeatable model 385 according to one or more embodiments. Method 500 generates a database of points of interest, which can be queried to collect information for determining the pose of a camera capturing an input image of a scene containing a set of input points of interest. The steps of method 500, as shown in Figure 5, are performed from the perspective of a map drawing system 390. However, some or all of the steps may be performed by other entities and / or components. Additionally, some embodiments may perform the steps in parallel, in a different order, or in different steps.

[0075] Mapping system 390 receives 510 an image of a scene. Mapping system 390 identifies 520 points of interest depicted within the received image. In some embodiments, mapping system 390 uses a trained model to detect points of interest within an input image (e.g., identify portions of the input image depicting points of interest). For example, mapping system 390 uses the same trained model that was repeatedly trained by system 380 to identify points of interest depicted within an input image.

[0076] For each identified point of interest, the map rendering system 390 uses a repetition model 385 trained by the repetition training system 380 to predict the repetition of 530 points of interest. For example, the repetition model 385 is used to determine a repetition function (or repetition score) for the points of interest. Based on the determined repetition function, the map rendering system 390 selects a subset of 540 points of interest. For example, the map rendering system 390 selects points of interest with a repetition function indicating an average repetition above a certain threshold. Alternatively, the map rendering system 390 sorts the points of interest based on their respective repetition functions and selects the highest-ranking point of interest based on the sorting. Next, the map rendering system 390 stores 550 pieces of information about each point of interest in the selected subset of points of interest to construct a 3D map 328. For example, the map rendering system 390 stores images, coordinates, and other relevant information about each point of interest in the selected subset of points of interest.

[0077] Figure 6 is a flowchart illustrating one of the methods 600 for using a repeatability model 385 to determine the pose of a camera according to one or more embodiments. Method 600 generates an estimated pose of an input image. The steps of method 600 are illustrated from the perspective of a user terminal device 310 performing the steps of Figure 6. However, some or all of the steps may be performed by other entities and / or components. In addition, some embodiments may perform the steps in parallel, in a different order, or in different ways.

[0078] A positioning module of a user terminal device 310 receives an image of a scene from 610. The image of the scene can be captured by a camera that is a component of the user terminal device 310 or external to the user terminal device 310. In the context of a parallel reality game, the scene may have a real-world location that can be mapped to a virtual location in the virtual world. The image of the scene may also have intrinsic parameters corresponding to the geometric properties of the camera that captures the image. The image may be a single image captured by the camera. Alternatively, the image may be a frame from a video captured by the camera.

[0079] The positioning module 318 identifies 620 a set of interest points depicted in the received image. In some embodiments, the positioning module 318 uses a trained model to detect interest points within an input image (e.g., identify portions of the input image depicting interest points). For example, the positioning module 318 uses the same trained model used by the repetitive training system 380 to identify interest points depicted within an input image.

[0080] For each identified point of interest (POI), the localization module 318 applies a repetition model 385 trained by the repetition training system 380 to predict the repetition of 630 POIs. For example, the repetition model 385 is used to determine a repetition function (or repetition score) for each identified POI. Based on the determined repetition function, the localization module 318 selects a subset of 640 POIs. For example, the localization module 318 selects POIs with a repetition function indicating an average repetition above a certain threshold. Alternatively, the localization module 318 sorts the POIs based on their respective repetition functions and selects the highest-ranking POI. Next, the localization module 318 searches for 650 POIs within the 3D map 328 and extracts 660 information about one or more POIs from the 3D map 328. Based on the extracted information about one or more POIs, the localization module 318 determines 670 the pose of a camera capturing the received image.

[0081] Figure 7 is a flowchart illustrating a method 700 for constructing a summary map of a specific location using a repeating model 385 according to one or more embodiments. Method 600 generates a summary map, which can be transmitted to a user device 310 to allow the user device to determine its pose in 3D space. The steps of method 700 are drawn from the perspective of map drawing system 390. However, some or all of the steps may be performed by other entities and / or components. In addition, some embodiments may perform the steps in parallel, in a different order, or in different ways.

[0082] The map rendering system 390 determines the current time of a user terminal device 310. The map rendering system 390 may receive the current time from the user terminal device, or it may determine the current local time of the user terminal device based on the internal time of a game server 320 and the location of the user terminal device 310. In other embodiments, the map rendering system 390 uses a universal time (e.g., UTC) to store time values. Therefore, the current time of the user terminal device 310 is the same as the local time of the map rendering system 390. The map rendering system 390 identifies a set of points of interest within a defined geographic area 720. For example, the map rendering system 390 identifies a set of points of interest (e.g., the nearest point of interest) near the user terminal device 310.

[0083] The map rendering system 390 applies a repeatability model 385 to each of the identified points of interest in the group of points of interest to predict the repeatability of the points of interest during the current time period 730. For example, the repeatability model 385 is used to determine the repeatability value of each point of interest during the current time period. Based on the determined repeatability of each point of interest, the map rendering system 390 selects a subset of 740 points of interest to include in the summary map. For example, the map rendering system 390 selects points of interest with a repeatability value at the current time that is higher than a certain threshold value. Alternatively, the map rendering system 390 sorts the points of interest based on their respective repeatability values ​​and selects the highest-ranking point of interest based on the sorting. Then, the map rendering system 390 generates a summary map 750 based on the selected subset of points of interest. The summary map can be used to determine the behavior of the user device (e.g., using method 600 of FIG. 6). Instance computing system

[0084] FIG8 is an exemplary architecture of a computing device according to one embodiment. Although FIG8 depicts a high-level block diagram illustrating one or more physical components of a computer that serves as part or all of one or more entities described herein according to an embodiment, a computer may have additional, fewer, or different components as provided in FIG8. Although FIG8 depicts a computer 800, the figure is intended as a functional description of various features that may exist in a computer system and not as a structural schematic diagram of one of the embodiments described herein. In practice, and as will be recognized by those skilled in the art, items shown separately may be combined and some items may be separated.

[0085] Figure 8 illustrates at least one processor 802 coupled to a chipset 804. A memory 806, a storage device 808, a keyboard 810, a graphics adapter 812, a pointing device 814, and a network adapter 816 are also coupled to the chipset 804. A display 818 is coupled to the graphics adapter 812. In one embodiment, the functionality of the chipset 804 is provided by a memory controller hub 820 and an I / O hub 822. In another embodiment, the memory 806 is directly coupled to the processor 802 instead of the chipset 804. In some embodiments, the computer 800 includes one or more communication buses for interconnecting these components. The one or more communication buses may include, as appropriate, a circuitry (sometimes referred to as a chipset) that interconnects system components and controls communication between system components.

[0086] Storage device 808 is any non-transitory computer-readable storage medium, such as a hard disk drive, optical disc read-only memory (CD-ROM), DVD, or a solid-state memory device or other optical storage device, magnetic tape cassette, magnetic tape, magnetic disk storage device or other magnetic storage device, optical disk storage device, flash memory device or other non-volatile solid-state storage device. This storage device 808 may also be referred to as permanent memory. Pointer 814 may be a mouse, trackball or other type of pointer device, and is used in conjunction with keyboard 810 to input data into computer 800. Graphics adapter 812 displays images and other information on monitor 818. Network adapter 816 couples computer 800 to a local area network or wide area network.

[0087] Memory 806 stores instructions and data used by processor 802. Memory 806 may be non-permanent memory, examples of which include high-speed random access memory, such as DRAM, SRAM, DDR RAM, ROM, EEPROM, and flash memory.

[0088] As is known in the art, a computer 800 may have different and / or other components besides those shown in FIG8. Additionally, the computer 800 may lack certain illustrated components. In one embodiment, a computer 800 acting as a server may lack a keyboard 810, a pointing device 814, a graphics adapter 812, and / or a display 818. Furthermore, a storage device 808 may be located at the computer 800 itself and / or remotely from the computer 800 (e.g., embodied within a storage area network (SAN)).

[0089] As known in the art, computer 800 is adapted to execute a computer program module for providing the functionality described herein. As used herein, the term "module" refers to computer program logic for providing specified functionality. Therefore, a module can be implemented in hardware, firmware, and / or software. In one embodiment, the program module is stored on storage device 808, loaded into memory 806, and executed by processor 802. Additional considerations

[0090] Some parts of the above description describe embodiments in terms of algorithmic procedures or operations. These algorithmic descriptions and representations are generally used by those skilled in data processing techniques to effectively convey their working principles to others skilled in the art. When described functionally, operationally, or logically, these operations are understood to be implemented by a computer program comprising instructions, microcode, or the like executed by a processor or equivalent circuitry. Furthermore, without loss of generality, it has sometimes proven convenient to refer to such configurations of functional operations as modules.

[0091] As used herein, any reference to "an embodiment" or "an embodiment" means that a particular element, feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment. The phrase "in an embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment in all cases.

[0092] The terms "coupled" and "connected," along with their derivatives, may be used to describe some embodiments. It should be understood that these terms are not intended to be synonyms. For example, the term "connected" may be used to describe some embodiments to indicate that two or more elements are in direct physical or electrical contact with each other. In another instance, the term "coupled" may be used to describe some embodiments to indicate that two or more elements are in direct physical or electrical contact with each other. However, the term "coupled" may also mean that two or more elements are not in direct contact with each other, but still cooperate or interact with each other. Embodiments are not limited to this context.

[0093] As used herein, the terms "comprises," "includes," "has," or any other variation thereof are intended to cover a non-exclusive inclusion. For example, a procedure, method, article, or apparatus that includes a list of components is not necessarily limited to those components alone, but may include other components not expressly listed or inherent to the procedure, method, article, or apparatus. Furthermore, unless expressly stated to the contrary, "or" means inclusive or non-exclusive or. For example, a condition A or B is satisfied by either: A is true (or exists) and B is false (or does not exist); A is false (or does not exist) and B is true (or exists); and both A and B are true (or exist).

[0094] Additionally, the use of "a" or "an" is used to describe the elements and components of the embodiments. This is done for convenience only and to give the general meaning of the invention. This description should be interpreted as including one or at least one, and the singular includes the plural, unless it is obvious otherwise.

[0095] Upon reading this invention, those skilled in the art will understand additional alternative structures and functional designs for a system and procedure used to determine or use repetitive points of interest. Therefore, although specific embodiments and applications have been illustrated and described, it should be understood that the described subject matter is not limited to the precise construction and components disclosed herein and that various modifications, alterations, and variations will be apparent to those skilled in the art in the configuration, operation, and details of the disclosed methods and apparatus. The scope of protection should be limited only to the following claims. [Simplified Explanation of the Diagram]

[0007] Figure 1 depicts a representation of a virtual world having a geography parallel to the real world according to one embodiment.

[0008] Figure 2 depicts an exemplary game interface of a parallel reality game according to one embodiment.

[0009] Figure 3 illustrates a network connection computing environment according to one or more embodiments.

[0010] Figure 4A is a flowchart illustrating one of the general procedures for training a repetitive model according to one or more embodiments.

[0011] Figure 4B illustrates a time sequence of images of a scene according to one or more embodiments.

[0012] Figure 5 is a flowchart illustrating one example use of a repeating function for constructing a 3D map according to one or more embodiments.

[0013] Figure 6 is a flowchart illustrating one example use of a repeatability function for determining the posture of a camera according to one or more embodiments.

[0014] Figure 7 is a flowchart illustrating one example use of a repeating function for constructing a summary map according to one or more embodiments.

[0015] Figure 8 illustrates an example computer system suitable for training or applying a repetitive model according to one or more embodiments.

[0016] The figures and the following description are merely illustrative of certain embodiments. Those skilled in the art will readily recognize from the following description alternative embodiments of the structure and method that can be employed without departing from the principles described. Examples of these embodiments will now be illustrated in the accompanying drawings with reference to several embodiments.

Claims

1. A computer-implemented method, comprising: The system receives a location identification of a user terminal device; identifies a plurality of points of interest located near the location of the user terminal device; determines a repetition score for each of the identified points of interest; selects a subset of the points of interest based on the determined repetition scores of each of the identified points of interest; generates a summary map of the vicinity of the location of the user terminal device based on the selected subset of points of interest; and sends the summary map to the user terminal device.

2. The method of request item 1, further comprising: Information about each point of interest in the selected subset is stored in a 3D map.

3. The method of claim 1, wherein determining the repetition score of one of the identified points of interest includes: A repetitive model is applied based on the identified points of interest, and the repetitive model is trained based on a time series of images of a scene spanning a set time period.

4. The method of claim 3, further comprising: Determine a current time, wherein the repetition score of the identified point of interest is further based on the determined current time.

5. The method of claim 4, wherein the current time is determined based on at least one of a time received from one of the user devices or the identified location of the user device and the internal time of one of the servers.

6. The method of claim 3, wherein the repetition model is trained by: receiving the time series images of the scene across the set time period; using an interest point detection model to identify a set of training interest points in the received time series images; for each training interest point in the set of training interest points, determining a repetition function by identifying the images in the time series images in which the training interest point is detected by the interest point detection model; and training the repetition model using information associated with one or more training interest points from the set of training interest points and the corresponding repetition function of the one or more training interest points.

7. The method of claim 1, wherein the repetition score indicates the likelihood that a trained point of interest detection model will detect the point of interest based on images captured by the user device.

8. A non-transitory computer-readable storage medium configured to store instructions that, when executed by a processor, cause the processor to: receive an image of a scene; identify a set of points of interest from the received image of the scene; determine a repetition score for each of the points of interest in the set; select a subset of points of interest based on the determined repetition score of each of the identified points of interest; search in a map for one or more points of interest from the selected subset of points of interest; receive information about one or more points of interest from the selected subset of points of interest; and determine a pose of a camera capturing the image of the scene based on the received information about the one or more points of interest.

9. The non-transitory computer-readable storage medium as described in claim 8, wherein the set of interest points is identified based on a trained interest point detection model.

10. The non-transitory computer-readable storage medium of claim 8, wherein the repetition score of the identified interest point is determined by applying a repetition model based on an identified interest point.

11. The non-transitory computer-readable storage medium of claim 10, wherein the repetitive model is trained based on a time series of images of a scene spanning a set time period.

12. The non-transitory computer-readable storage medium of claim 10, wherein searching for one or more points of interest from the selected subset of points of interest in a map includes searching for the one or more points of interest in a three-dimensional (3D) map.

13. As in claim 8, a non-transitory computer-readable storage medium, wherein determining the posture of a camera capturing the image of the scene includes: The relative pose of the camera is determined based on the received information about one or more points of interest; and the absolute pose of the camera is determined based on a set of reference images and the determined relative pose of the camera.

14. A non-transitory computer-readable storage medium as claimed in claim 8, wherein the scenario corresponds to a real-world location mapped to a virtual location in a virtual world.

15. A computer-implemented method, comprising: Receive a time-series image of a scene across a set time period; An interest point detection model is used to identify a set of interest points in the received time series images; for each interest point in the set of interest points, a repetition function is determined by identifying the images in the time series images in which the interest point is detected by the interest point detection model; and a repetition model is trained using information associated with one or more interest points from the set of interest points and the corresponding repetition function of the one or more interest points.

16. The method of claim 15, wherein each image in the time series images is associated with a timestamp, and wherein the repeatability function is further based on the timestamp associated with each image in which the interest point is detected by the interest point detection model.

17. The method of claim 15, wherein the repeatability function indicates for each time period of a set of time periods whether the point of interest is detected in an image corresponding to that time period by the point of interest detection model.

18. The method of claim 15, wherein the repeatability function indicates for each time period of a set of time periods a percentage of the image of that time period in which the point of interest was detected by the point of interest detection model.

19. The method of claim 15, wherein the repeatability model is trained to predict the probability of detecting a target interest point from an image captured within a given time interval by means of the interest point detection model.

20. The method of claim 15, wherein the repeatability model is trained to predict the probability of detecting a target point of interest from an image of the scene in which the point of interest is located by means of the point of interest detection model.

Images