Determining a location using multi-source geolocation data

Abstract

Systems, methods, and computer-readable media for determining a location of a device using multi-source geolocation data, where the method includes accessing new location data from a location source of a plurality of location sources, where the new location data includes a new fix and an accuracy of the new fix, and determining a current fix and an accuracy of the current fix based on the new fix, the accuracy of the new fix, a previous current fix, and an accuracy of the previous current fix. The method also includes determining a change in location based on a difference between the current fix and the previous current fix. Some systems, methods, and computer-readable media target scheduling location requests for generating location data, where the scheduling and making of actual requests is based on a plurality of conditions.

Description

[0001] Priority requirements

[0002] This application claims the benefit of priority to U.S. Application Serial No. 17 / 444,111, filed July 30, 2021, and U.S. Application Serial No. 63 / 200,789, filed March 29, 2021, each of which is incorporated herein by reference in its entirety. Technical Field

[0003] Examples of this disclosure generally involve using multi-source geolocation data to determine the geolocation of a mobile device. More specifically, but not as a limitation, examples of this disclosure involve fusing multi-source geolocation data together to determine geolocation. Background Technology

[0004] Many applications frequently use geolocation to enhance the user experience on mobile devices. However, determining a mobile device's geolocation consumes power, and mobile devices are typically limited by their batteries. Additionally, applications may request frequent updates to the mobile device's geolocation. Attached Figure Description

[0005] In accompanying drawings that are not necessarily drawn to scale, the same reference numerals may describe similar parts in different views. For ease of identification of any particular element or action in the discussion, one or more of the most significant digits in the reference numerals refer to the figure number in which that element is first introduced. Some examples are shown in the accompanying drawings by way of example rather than limitation, in which:

[0006] Figure 1 It is a diagrammatic representation of a networked environment in which the content of this disclosure can be deployed, based on some examples.

[0007] Figure 2 It is a graphical representation of a messaging system with both client-side and server-side functionalities, based on some examples.

[0008] Figure 3 It is a graphical representation of the data structures maintained in the database based on some examples.

[0009] Figure 4 It is a graphical representation based on some example messages.

[0010] Figure 5 It is a flowchart of an access restriction process based on some examples.

[0011] Figure 6 The location data source for mobile devices is shown based on some examples.

[0012] Figure 7 A system 700 for determining location using multi-source geographic location data is shown, based on some examples.

[0013] Figure 8 The operation of methods for fusing locations is shown based on some examples.

[0014] Figure 9 An example of fused location data is shown based on some examples.

[0015] Figure 10 The following examples illustrate methods for scheduling requests.

[0016] Figure 11 Examples of wearable electronic devices in the form of glasses with enabled electronic functions are shown, based on some examples.

[0017] Figure 12 The diagram illustrates location tracking based on some examples.

[0018] Figure 13 The following are examples of methods for scheduling requests for location data.

[0019] Figure 14 It is a graphical representation of a machine in the form of a computer system, based on some examples, within which a set of instructions can be executed to cause the machine to perform any or more of the methods discussed herein.

[0020] Figure 15 It is a block diagram showing a software architecture in which examples can be implemented.

[0021] Figure 16 It is a graphical representation of the processing environment based on some examples. Detailed Implementation

[0022] The following description includes illustrative examples of systems, methods, techniques, instruction sequences, and computer program products that embody the contents of this disclosure. In the following description, numerous specific details are set forth for illustrative purposes to provide an understanding of various examples of the subject matter of the invention. However, it will be apparent to those skilled in the art that examples of the subject matter of the invention can be practiced without these specific details. Generally, well-known examples of instructions, protocols, structures, and techniques are not necessarily shown in detail.

[0023] Users of mobile devices enjoy services provided by applications that can determine the current location of the mobile device. For example, a location-aware application on an augmented reality (AR) headset can provide information about landmarks (such as the Eiffel Tower or the Empire State Building) that the AR headset user is near. If the current location of the mobile device is known, many other applications on the mobile device can be enhanced. The examples disclosed herein pertain to providing the current location to applications on a mobile device while reducing the power used and fusing location data to provide the current location. Some examples disclosed herein pertain to providing location data to applications while performing one or more of the following operations: reducing the latency of providing location data, increasing the accuracy of location data, and reducing the power used to determine the location data.

[0024] The technical challenge is how to provide current location information to applications on mobile devices while reducing the power required to provide that current location. In some examples, this is addressed by merging two or more locations together to determine the mobile device's current location. Each location includes data indicating its accuracy. Typically, mobile devices require less power to acquire or determine less accurate location data. In some examples, a current location with associated accuracy is merged with a new location with associated accuracy to determine an updated current location with updated accuracy. The current location is merged with the new location based on geographic location data, associated optional data, and associated accuracy. According to some examples, merging location data provides a more accurate estimate of the mobile device's location.

[0025] In some examples, this technical problem is addressed by scheduling requests for updated location data from a location source. Scheduling waits to send a request to the location source based on whether certain conditions are met. For example, a request for location data from a satellite receiver may not be made until the mobile device's display is powered off and / or the mobile device is outdoors. In this way, power demand does not surge, potentially consuming more battery power than is needed to retrieve the location from the satellite. Furthermore, scheduling decisions are based on the required accuracy of the location indicated by the application.

[0026] Networked computing environment

[0027] Figure 1 This is a block diagram illustrating an example messaging system 100 for exchanging data (e.g., messages and associated content) over a network. The messaging system 100 includes multiple instances of client devices 102, each instance hosting multiple applications including messaging clients 104. Each messaging client 104 is communicatively coupled to other instances of messaging client 104 and a messaging server system 108 via a network 106 (e.g., the Internet).

[0028] Message transceiver client 104 can communicate and exchange data with another message transceiver client 104 and message transceiver server system 108 via network 106. The data exchanged between message transceiver clients 104 and between message transceiver client 104 and message transceiver server system 108 includes functions (e.g., commands to invoke functions) and payload data (e.g., text, audio, video, or other multimedia data).

[0029] The message transceiver server system 108 provides server-side functionality to specific message transceiver clients 104 via network 106. Although some functions of the message transceiver system 100 are described herein as being performed by either the message transceiver client 104 or the message transceiver server system 108, the location of certain functions within the message transceiver client 104 or the message transceiver server system 108 can be a design choice. For example, it may be technically preferred that certain technologies and functions are initially deployed within the message transceiver server system 108, but that technology and functions are later migrated to the message transceiver client 104 of the client device 102 with sufficient processing power.

[0030] The messaging server system 108 supports various services and operations provided to the messaging client 104. Such operations include sending data to and receiving data from the messaging client 104, and processing data generated by the messaging client 104. As an example, this data may include message content, client device information, geolocation information, media enhancements and overlays, message content persistence conditions, social network information, and live event information. Data exchange within the messaging system 100 is activated and controlled through functions available via the user interface (UI) of the messaging client 104.

[0031] Specifically, turning to message transceiver server system 108, application programming interface (API) server 110 is coupled to application server 112 and provides a programming interface to application server 112. Application server 112 is communicatively coupled to database server 118, which provides access to database 120, which stores data associated with messages processed by application server 112. Similarly, web server 124 is coupled to application server 112 and provides a web-based interface to application server 112. For this purpose, web server 124 handles incoming network requests via Hypertext Transfer Protocol (HTTP) and several other related protocols.

[0032] Application Programming Interface (API) server 110 receives and sends message data (e.g., commands and message payloads) between client device 102 and application server 112. Specifically, API server 110 provides a set of interfaces (e.g., routines and protocols) that message sending and receiving client 104 can invoke or query to activate the functionality of application server 112. Application Programming Interface (API) server 110 exposes various functionalities supported by application server 112, including: account registration; login functionality; sending messages from one messaging client 104 to another messaging client 104 via application server 112; sending media files (e.g., images or videos) from messaging client 104 to messaging server 114 and for possible access by another messaging client 104; setting up media data collections (e.g., stories); retrieving the friend list of the user of client device 102; retrieving such collections; retrieving messages and content; adding and deleting entities (e.g., friends) in an entity graph (e.g., a social graph); locating friends within the social graph; and opening application events (e.g., related to messaging client 104).

[0033] Application server 112 hosts multiple server applications and subsystems, including, for example, messaging server 114, image processing server 116, and social networking server 122. Messaging server 114 implements multiple messaging technologies and functions, particularly those related to the aggregation and other processing of content (e.g., text and multimedia content) included in messages received from multiple instances of messaging client 104. As will be described in further detail, text and media content from multiple sources can be aggregated into content collections (e.g., referred to as stories or galleries). These collections are then made available to messaging client 104. Given the hardware requirements for such processing, additional processor- and memory-intensive processing of data can also be performed on the server side by messaging server 114.

[0034] Application server 112 also includes image processing server 116, which is dedicated to performing various image processing operations, typically relative to the images or videos within the payload of messages sent from or received at message transceiver server 114.

[0035] Social network server 122 supports various social network functions and services, and makes these functions and services available to message sending and receiving server 114. To this end, social network server 122 maintains and accesses entity graph 306 (such as...) within database 120. Figure 3(As shown). Examples of functions and services supported by the social network server 122 include identifying other users of the messaging system 100 who have a relationship with or “follow” a particular user, as well as identifying the interests and other entities of a particular user.

[0036] System Architecture

[0037] Figure 2 This is a block diagram illustrating further details of a messaging system 100 according to some examples. Specifically, the messaging system 100 is shown as including a messaging client 104 and an application server 112. The messaging system 100 includes multiple subsystems supported on the client side by the messaging client 104 and on the server side by the application server 112. These subsystems include, for example, a short-timer system 202, a collection management system 204, a modification system 206, a map system 208, a game system 210, and a geolocation system 214.

[0038] The short-lived timer system 202 is responsible for forcing temporary or time-limited access to content by message sending client 104 and message sending server 114. The short-lived timer system 202 includes multiple timers that selectively enable access (e.g., for rendering and displaying) of messages and associated content via message sending client 104 based on duration and display parameters associated with a message or message set (e.g., a story). Further details regarding the operation of the short-lived timer system 202 are provided below.

[0039] The collection management system 204 is responsible for managing groups or collections of media (e.g., collections of text, image, video, and audio data). Collections of content (e.g., messages, including images, videos, text, and audio) can be organized into "event libraries" or "event stories." Such collections can be made available for a specified period of time (e.g., the duration of an event related to the content). For example, content related to a concert can be made available as a "story" for the duration of a concert. The collection management system 204 can also be responsible for publishing icons that notify the user interface of the messaging client 104 of the existence of a specific collection.

[0040] Furthermore, the collection management system 204 includes a curation interface 212 that enables collection managers to manage and curate specific collections of content. For example, curation interface 212 allows event organizers to curate collections of content related to a specific event (e.g., removing inappropriate content or redundant messages). Additionally, the collection management system 204 employs machine vision (or image recognition technology) and content rules to automatically curate content collections. In some examples, users may be paid compensation for including user-generated content in the collection. In such cases, the collection management system 204 operates to automatically pay such users for using their content.

[0041] Enhancement system 206 provides various functionalities that enable users to enhance (e.g., annotate or otherwise modify or edit) media content associated with messages. For example, enhancement system 206 provides functionality related to generating and publishing media overlays for messages processed by messaging system 100. Enhancement system 206 can operablely provide media overlays or enhancements (e.g., image filters) to messaging client 104 based on the geographic location of client device 102. In another example, enhancement system 206 can operablely provide media overlays to messaging client 104 based on other information (e.g., the social network information of the user of client device 102). Media overlays can include audio and visual content as well as visual effects. Examples of audio and visual content include pictures, text, logos, animations, and sound effects. Examples of visual effects include color overlays. Audio and visual content or visual effects can be applied to media content items (e.g., photos) at client device 102. For example, a media overlay can include text or images that can be overlaid on top of a photo taken by client device 102. In another example, media overlays include location identifier overlays (e.g., Venice Beach), names of live events, or business names (e.g., Beach Cafe). In yet another example, enhancement system 206 uses the geographic location of client device 102 to identify media overlays that include the name of a business located at the geographic location of client device 102. Media overlays may include other tags associated with the business. Media overlays may be stored in database 120 and accessed through database server 118.

[0042] In some examples, Enhancement System 206 provides a user-based publishing platform that allows users to select geographic locations on a map and upload content associated with those locations. Users can also specify when particular media overlays should be provided to other users. Enhancement System 206 generates media overlays that include the uploaded content and associate it with the selected geographic locations.

[0043] In other examples, Enhancement System 206 provides a merchant-based publishing platform that enables merchants to select specific media coverage associated with geographic locations via a bidding process. For example, Enhancement System 206 associates the media coverage of the highest-bidding merchant with a corresponding geographic location for a predefined amount of time.

[0044] Map system 208 provides various geolocation functions and supports the presentation of map-based media content and messages by messaging client 104. For example, map system 208 enables the display (e.g., stored in profile data 308) of user icons or avatars on a map to indicate the current or past locations of the user's "friends" in the context of the map, as well as media content (e.g., a collection of messages including photos and videos) generated by such friends. For example, on the map interface of messaging client 104, a message posted by a user from a specific geolocation to messaging system 100 can be displayed to the specific user's "friends" in the context of that specific location on the map. The user can also share his or her location and status information with other users of messaging system 100 (e.g., using appropriate status avatars) via messaging client 104, where the location and status information is similarly displayed to the selected user in the context of the map interface of messaging client 104.

[0045] The game system 210 provides various game functions within the context of the messaging client 104. The messaging client 104 provides a game interface with a list of available games that can be started by a user within the context of the messaging client 104 and played with other users of the messaging system 100. The messaging system 100 also enables specific users to invite other users to play specific games by sending invitations from the messaging client 104 to such other users. The messaging client 104 also supports voice and text messaging (e.g., chat) within the game context, provides leaderboards for the game, and supports in-game rewards (e.g., coins and items).

[0046] The geolocation system 214 provides various functions to identify mobile devices 602 (see...). Figure 6 The current position is 708 (see Figure 7In some examples, the geolocation system 214 interfaces with an external device to determine the current location 708 of the mobile device 602. In some examples, the geolocation system 214 responds to a request for geolocation information from the mobile device 602. In some examples, the geolocation system 214 provides information to assist the mobile device 602 in determining its geolocation, such as almanac data from a GNSS system or information relating to other wireless devices with which the mobile device 602 can interact to determine the geolocation of the mobile device 602.

[0047] Data Architecture

[0048] Figure 3 This is a schematic diagram illustrating a data structure 300 that can be stored in a database 120 of a message transceiver server system 108, according to certain examples. Although the contents of the database 120 are shown as including multiple tables, it should be understood that the data can be stored in other types of data structures (e.g., as an object-oriented database).

[0049] Database 120 includes message data stored in message table 302. For any given message, this message data includes at least message sender data, message receiver (or recipient) data, and a payload. See below for reference. Figure 4 Further details describe information that can be included in the message and is contained within the message data stored in message table 302.

[0050] Entity table 304 stores entity data and (for example, links to entity diagram 306 and profile data 308). Entities for which records are maintained in entity table 304 can include individuals, company entities, organizations, objects, locations, events, etc. Regardless of the entity type, any entity whose data is stored in message transceiver server system 108 can be an identifiable entity. Each entity is provided with a unique identifier and an entity type identifier (not shown).

[0051] Entity graph 306 stores information about the relationships and associations between entities. As an example only, such relationships can be social relationships based on interests or activities, or professional relationships (e.g., working in a common company or organization).

[0052] Profile data 308 stores various types of profile data about a specific entity. Profile data 308 can be selectively used and presented to other users of messaging system 100 based on privacy settings specified by the specific entity. In the case that the entity is an individual, profile data 308 includes, for example, a username, phone number, address, settings (e.g., notification and privacy settings), and an avatar representation (or a set of such avatar representations) selected by the user. A specific user can then selectively include one or more of these avatar representations in the content of messages transmitted via messaging system 100, as well as in a map interface displayed to other users by messaging client 104. The set of avatar representations may include “status avatars,” which present a graphical representation of a user’s chosen state or activity at a specific time.

[0053] In the case that the entity is a group, in addition to the group name, members and various settings of the associated group (e.g., notifications), the group profile data 308 may similarly include one or more avatars associated with the group.

[0054] Database 120 also stores enhancement data, such as overlays or filters, in enhancement table 310. The enhancement data is associated with and applied to videos (for which data is stored in video table 314) and images (for which data is stored in image table 316).

[0055] In one example, a filter is an overlay displayed on top of an image or video during presentation to the receiving user. Filters can be of various types, including filters selected by the user from a set of filters presented to the sending user by the messaging client 104 when the sending user is composing a message. Other types of filters include geolocation filters (also known as geographic filters), which can be presented to the sending user based on geographic location. For example, a geolocation filter specific to a nearby or particular location can be presented by the messaging client 104 within the user interface based on geographic location information determined by the Global Positioning System (GPS) unit of the client device 102.

[0056] Another type of filter is a data filter, which can be selectively presented to the sending user by the messaging client 104 based on other inputs or information collected by the client device 102 during the message creation process. Examples of data filters include the current temperature at a specific location, the current speed of the sending user, the battery life of the client device 102, or the current time.

[0057] Other augmented data that can be stored in image table 316 includes augmented reality content items (e.g., corresponding to an application lens or augmented reality experience). Augmented reality content items can be real-time special effects and sounds that can be added to images or videos.

[0058] As described above, augmented data includes augmented reality content items, overlays, image transformations, AR images, and similar items involving modifications that can be applied to image data (e.g., video or images). This includes real-time modifications, which modify images as they are captured using the device sensors (e.g., one or more cameras) of client device 102 and then display the modified images on the screen of client device 102. This also includes modifications to stored content, such as modifications to video clips in a library that can be modified. For example, in client device 102, which has access to multiple augmented reality content items, a user can use a single video clip with multiple augmented reality content items to see how different augmented reality content items will modify the stored clip. For example, by selecting different augmented reality content items for the content, multiple augmented reality content items with different pseudo-random motion models can be applied to the same content. Similarly, real-time video capture can be used with the illustrated modifications to show how the video image currently captured by the sensors of client device 102 will modify the captured data. Such data can be displayed on the screen without being stored in memory, or the content captured by the device's sensors can be recorded and stored in memory with or without modification (or both). In some systems, the preview feature can simultaneously show how different augmented reality content items will look in different windows on the display. For example, this allows viewing multiple windows with different pseudo-random animations on the display at the same time.

[0059] Therefore, using augmented reality content items and various systems, or other such transformation systems that modify content using that data, can involve detecting objects (e.g., faces, hands, bodies, cats, dogs, surfaces, objects, etc.) in video frames, tracking these objects as they leave, enter, and move around within the field of view, and modifying or transforming them while tracking them. Different methods can be used to implement such transformations in various examples. Some examples may involve generating a 3D mesh model of one or more objects, and then using transformations and animated textures of the model within the video to achieve the transformation. In other examples, tracking points on objects can be used to place images or textures (which can be two-dimensional or three-dimensional) at the tracked locations. In further examples, neural network analysis of video frames can be used to place images, models, or textures within content (e.g., images or video frames). Therefore, augmented reality content items refer both to the images, models, and textures used to create transformations within content, and to the additional modeling and analysis information required to achieve such transformations through object detection, tracking, and placement.

[0060] Real-time video processing can be performed using any type of video data (e.g., video streams, video files, etc.) stored in the memory of any type of computerized system. For example, a user can load a video file and store it in the device's memory, or a video stream can be generated using the device's sensors. Furthermore, computer-animated models can be used to process any object, such as a human face and parts of the human body, animals, or inanimate objects (e.g., chairs, cars, or other objects).

[0061] In some examples, when a specific modification is selected along with the content to be transformed, the element to be transformed is identified by the computing device and then detected and tracked if the element to be transformed exists in a frame of the video. The elements of the object are modified according to the modification request, thereby transforming the frames of the video stream. For different types of transformations, the transformation of the frames of the video stream can be performed using different methods. For example, for a transformation of frames that primarily refers to changing the form of elements of an object, feature points of each element of the object (e.g., using an Active Shape Model (ASM) or other known methods) are calculated. Then, a feature point-based mesh is generated for each element in at least one of the elements of the object. This mesh is used for subsequent stages of tracking the elements of the object in the video stream. During tracking, the mesh mentioned for each element is aligned with the position of each element. Then, additional points are generated on the mesh. A first set of first points is generated for each element based on the modification request, and a second set of points is generated for each element based on the first set of points and the modification request. The frames of the video stream can then be transformed by modifying the elements of the object based on the first and second sets of points and the mesh. In this method, the background of the object being modified can also be changed or distorted by tracking and modifying the background.

[0062] In some examples, transforming certain regions of an object using its elements can be performed by calculating feature points for each element of the object and generating a mesh based on those calculated feature points. Points are generated on the mesh, and then various regions are generated based on these points. The elements of the object are then tracked by aligning the regions of each element with the positions of at least one of the elements, and the properties of the regions can be modified based on modification requests, thereby transforming frames of the video stream. Depending on the specific modification request, the properties of the mentioned regions can be transformed in different ways. Such modifications can involve: changing the color of the region; removing at least some portions of the region from the frames of the video stream; including one or more new objects in the region based on the modification request; and modifying or distorting the elements of the region or object. In various examples, any combination of such modifications or other similar modifications can be used. For certain models to be animated, some feature points can be selected as control points to determine the entire state space for options used in model animation.

[0063] In some examples of computer animation models that use face detection to transform image data, a specific face detection algorithm (e.g., Viola-Jones) is used to detect faces in the image. The Active Shape Model (ASM) algorithm is then applied to the facial regions of the image to detect facial feature reference points.

[0064] In other examples, other methods and algorithms suitable for face detection can be used. For instance, in some examples, landmarks are used to locate features that represent distinguishable points present in most of the images considered. For example, for a face landmark, the location of the left pupil could be used. If the initial landmark is unrecognizable (e.g., if the person is wearing an eye patch), secondary landmarks can be used. Such a landmark recognition process can be used for any such object. In some examples, a set of landmarks forms a shape. The shape can be represented as a vector using the coordinates of points within it. One shape is aligned with another shape through a similarity transformation (allowing translation, scaling, and rotation) that minimizes the average Euclidean distance between the points of the shapes. The average shape is the average of the aligned training shapes.

[0065] In some examples, a landmark search begins with an average shape aligned with the position and size of a face determined by a global face detector. This search then repeats the following steps: proposing provisional shapes by adjusting the positions of shape points using template matching of the image texture around each point, and then conforming the provisional shapes to a global shape model until convergence occurs. In some systems, individual template matching is unreliable, and the shape model pools the results of weak template matching to form a stronger overall classifier. The entire search is repeated at each level of the image pyramid, from coarse to fine resolution.

[0066] The transformation system can capture image or video streams on a client device (e.g., client device 102) and perform complex image manipulations locally on client device 102 while maintaining an appropriate user experience, computation time, and power consumption. Complex image manipulations can include size and shape changes, emotion shifts (e.g., changing a face from frowning to smiling), state shifts (e.g., aging an object, reducing its apparent age, or changing its gender), style shifts, application of graphical elements, and any other suitable image or video manipulations implemented by a convolutional neural network that has been configured to execute efficiently on client device 102.

[0067] In some examples, a computer animation model for transforming image data can be used by a system in which a user can use a client device 102 with a neural network to capture an image or video stream of the user (e.g., a selfie), the neural network operating as part of a messaging client application operating on the client device 102. A transformation system operating within the messaging client 104 determines the presence of a face within the image or video stream and provides a modification icon associated with the computer animation model to transform the image data, or the computer animation model can be presented as associated with the interface described herein. The modification icon includes changes that can be the basis for modifying the user's face within the image or video stream as part of a modification operation. Once a modification icon is selected, the transformation system initiates a process of transforming the user's image to reflect the selected modification icon (e.g., generating a smiley face on the user). Once the image or video stream is captured and the specified modification is selected, the modified image or video stream can be presented in a graphical user interface displayed on the client device 102. The transformation system can implement a complex convolutional neural network on a portion of the image or video stream to generate and apply the selected modification. In other words, once an edit icon is selected, the user can capture an image or video stream and view the changes in real-time or near real-time. Furthermore, while a video stream is being captured, the edits can be persistent, and the selected edit icon continues to be toggled. Machine-trained neural networks can be used to achieve such edits.

[0068] A graphical user interface (GUI) presenting modifications performed by a transformation system can provide users with additional interactive options. Such options can be based on an interface used to initiate the selection and content capture of a specific computer animation model (e.g., initiated from a content creator GUI). In various examples, modifications can be persistent after an initial selection of the modification icon. Users can turn modifications on or off by tapping or otherwise selecting a face modified by the transformation system and save it for later viewing or browsing to other areas of the imaging application. In cases where multiple faces are modified by the transformation system, users can globally turn modifications on or off by tapping or selecting a single face modified and displayed within the GUI. In some examples, individual faces within a set of multiple faces can be modified separately, or such modifications can be toggled individually by tapping or selecting individual faces or a series of individual faces displayed within the GUI.

[0069] Story table 312 stores data about collections of messages and associated image, video, or audio data, compiled into collections (e.g., stories or galleries). The creation of a specific collection can be initiated by a specific user (e.g., each user for whom a record is maintained in entity table 304). A user can create a "personal story" in the form of a collection of content that has already been created and sent / broadcast by that user. For this purpose, the user interface of messaging client 104 may include user-selectable icons that allow the sending user to add specific content to his or her personal story.

[0070] The collection can also constitute a "live story" as a collection of content from multiple users, created manually, automatically, or using a combination of manual and automatic techniques. For example, a "live story" can constitute a curated stream of user-submitted content from various locations and events. Users whose client devices have location services enabled and are at a common location event at a specific time can be presented with options, for example, via the user interface of messaging client 104, to contribute content to a specific live story. The messaging client 104 can identify live stories to users based on their location. The end result is a "live story" told from a community perspective.

[0071] Another type of content collection is called a "location story," which allows users whose client devices 102 are located in a specific geographic location (e.g., on a college or university campus) to contribute to a specific collection. In some examples, contributing to a location story may require secondary authentication to verify that the end user belongs to a specific organization or other entity (e.g., a student on a university campus).

[0072] As mentioned above, video table 314 stores video data, which, in one example, is associated with messages for which records are maintained within message table 302. Similarly, image table 316 stores image data associated with messages whose message data is stored in entity table 304. Entity table 304 can associate various enhancements from enhancement table 310 with various images and videos stored in image table 316 and video table 314. Database 120 can also store information used to assist mobile device 602 in determining its location. For example, database 120 can store information related to access points (APs) that mobile device 602 can use to determine its geographic location. Database 120 can also store additional information, such as data that can be used to interpret signals from satellites to determine location.

[0073] Data communication architecture

[0074] Figure 4This is a schematic diagram illustrating the structure of message 400 according to some examples. Message 400 is generated by message transceiver client 104 for transmission to another message transceiver client 104 or message transceiver server 114. The content of a particular message 400 is used to populate message table 302 within database 120 accessible by message transceiver server 114. Similarly, the content of message 400 is stored in memory as "in transit" or "in flight" data of client device 102 or application server 112. Message 400 is shown to include the following example components:

[0075] Message Identifier 402 (MSG_ID 402): A unique identifier that identifies message 400. Message Text Payload 404 (MSG_TEXT 404): The text to be generated by the user via the user interface of client device 102 and included in message 400.

[0076] Message Image Payload 406 (MSG_IMAGE 406): Image data captured by the camera component of the client device 102 or retrieved from the memory component of the client device 102 and included in the message 400. The image data for the sent or received message 400 can be stored in the image table 316.

[0077] Message video payload 408: Video data captured by the camera device component or retrieved from the memory component of the client device 102 and included in the message 400. The video data for the sent or received message 400 may be stored in the video table 314.

[0078] Message audio payload 410: Audio data captured by the microphone or retrieved from the memory component of the client device 102 and included in message 400.

[0079] Message enhancement data 412: Enhancement data (e.g., filters, stickers, or other annotations or enhancements) representing enhancements to be applied to the message image payload 406, message video payload 408, or message audio payload 410 of message 400. Enhancement data for the sent or received message 400 can be stored in enhancement table 310.

[0080] Message Duration Parameter 414 (MSG_DUR 414): Parameter value that indicates the amount of time, in seconds, for the content of the message (e.g., message image payload 406, message video payload 408, message audio payload 410) to be presented to the user or made accessible to the user via the message transceiver client 104.

[0081] Message geolocation parameter 416: Geographic location data (e.g., latitude and longitude coordinates) associated with the message's content payload. Multiple message geolocation parameter 416 values ​​may be included in the payload, each of which is associated with a content item included in the content (e.g., a specific image within the message image payload 406, or a specific video within the message video payload 408).

[0082] Message Story Identifier 418: An identifier value that identifies one or more content sets (e.g., "Stories" identified in Story Table 312), wherein a specific content item in the message image payload 406 of message 400 is associated with one or more content sets. For example, the identifier value can be used to associate multiple images within the message image payload 406 with multiple content sets, respectively.

[0083] Message Label 420: Each message 400 can be labeled with multiple labels, each of which indicates the subject of the content included in the message payload. For example, in the case where a specific image depicting an animal (e.g., a lion) is included in the message image payload 406, a label value indicating the relevant animal can be included in the message label 420. The label values ​​can be generated manually based on user input or automatically using, for example, image recognition.

[0084] Message sender identifier 422: An identifier (e.g., message sending system identifier, email address, or device identifier) ​​indicating the user of the client device 102 on which message 400 is generated and from which message 400 is sent.

[0085] Message receiver identifier 424: An identifier (e.g., message sending and receiving system identifier, email address, or device identifier) ​​indicating the user of the client device 102 to which message 400 is addressed.

[0086] The content (e.g., values) of each component of message 400 can be pointers to locations in tables where content data values ​​are stored. For example, image values ​​in message image payload 406 can be pointers to locations (or addresses of locations) within image table 316. Similarly, values ​​in message video payload 408 can point to data stored in video table 314, values ​​in message enhancement 412 can point to data stored in enhancement table 310, values ​​in message story identifier 418 can point to data stored in story table 312, and values ​​in message sender identifier 422 and message receiver identifier 424 can point to user records stored in entity table 304.

[0087] Although the described flowcharts may show operations as a sequential process, many operations within an operation can be executed in parallel or simultaneously. Furthermore, the order of operations can be rearranged. A process terminates when its operations are complete. A process can correspond to a method, program, algorithm, etc. The operations of a method can be executed in whole or in part, in combination with some or all of the operations of other methods, and can be executed by any number of different systems, such as the system described herein, or any part thereof, such as a processor included in any system.

[0088] Time-based access restriction architecture

[0089] Figure 5 This is a schematic diagram illustrating an access restriction process 500, according to which access to content (e.g., a short message 502 and a multimedia payload of associated data) or a collection of content (e.g., a short message group 504) can be time-restricted (e.g., so that it is short).

[0090] A brief message 502 is shown to be associated with a message duration parameter 506, the value of which determines the amount of time the message sending and receiving client 104 will display the brief message 502 to the receiving user. In one example, depending on the amount of time specified by the sending user using the message duration parameter 506, the receiving user can view the brief message 502 for up to 10 seconds.

[0091] Message duration parameter 506 and message receiver identifier 424 are shown as inputs to message timer 512, which is responsible for determining the amount of time for which brief message 502 is shown to a specific receiving user identified by message receiver identifier 424. Specifically, brief message 502 is shown to the relevant receiving user only within the time period determined by the value of message duration parameter 506. Message timer 512 is shown as providing an output to a more generalized brief timer system 202, which is responsible for the overall timing of displaying content (e.g., brief message 502) to the receiving user.

[0092] Figure 5The transient message 502 shown is included within a transient message group 504 (e.g., a collection of messages in a personal story or event story). The transient message group 504 has an associated group duration parameter 508, the value of which determines the duration for which the transient message group 504 is presented and accessible to a user of the messaging system 100. For example, the group duration parameter 508 could be the duration of a concert, where the transient message group 504 is a collection of content related to that concert. Alternatively, when setting up and creating the transient message group 504, the user (owner user or curator user) can specify the value of the group duration parameter 508.

[0093] Additionally, each short message 502 within a short message group 504 has an associated group participation parameter 510, the value of which determines the duration for which the short message 502 will be accessible within the context of the short message group 504. Therefore, a particular short message group 504 can "expire" and become inaccessible within its context before the short message group 504 itself expires according to the group duration parameter 508. The group duration parameter 508, the group participation parameter 510, and the message receiver identifier 424 each provide input to a group timer 514, which is operable to first determine whether a particular short message 502 of the short message group 504 will be displayed to a specific receiving user, and if so, determine for how long. Note that the short message group 504 also knows the identity of the specific receiving user due to the message receiver identifier 424.

[0094] Therefore, the group timer 514 operatively controls the associated ephemeral message group 504 and the total lifetime of the individual ephemeral messages 502 included within the ephemeral message group 504. In one example, each ephemeral message 502 within the ephemeral message group 504 remains viewable and accessible for a period of time specified by the group duration parameter 508. In another example, within the context of the ephemeral message group 504, a particular ephemeral message 502 may expire based on the group participation parameter 510. Note that even within the context of the ephemeral message group 504, the message duration parameter 506 can still determine the duration for which a particular ephemeral message 502 is displayed to the receiving user. Therefore, the message duration parameter 506 determines the duration for which a particular ephemeral message 502 is displayed to the receiving user, regardless of whether the receiving user views the ephemeral message 502 within or outside the context of the ephemeral message group 504.

[0095] The short-lived timer system 202 can also operatively remove a specific short-lived message 502 from the short-lived message group 504 based on determining that the specific short-lived message 502 has exceeded its associated group participation parameter 510. For example, when the sending user has established a group participation parameter 510 for 24 hours from the date of publication, the short-lived timer system 202 will remove the relevant short-lived message 502 from the short-lived message group 504 after the specified twenty-four hours. The short-lived timer system 202 also operates to remove the short-lived message group 504 when the group participation parameter 510 for each short-lived message 502 within the short-lived message group 504 has expired, or when the short-lived message group 504 itself has expired according to the group duration parameter 508.

[0096] In certain use cases, the creator of a specific ephemeral message group 504 can specify an indefinite group duration parameter 508. In this case, the expiration of the group participation parameter 510 for the last remaining ephemeral message 502 within the ephemeral message group 504 will determine when the ephemeral message group 504 itself expires. In this case, adding a new ephemeral message 502 with a new group participation parameter 510 to the ephemeral message group 504 effectively extends the lifetime of the ephemeral message group 504 to a value equal to the group participation parameter 510.

[0097] In response to the short-lived timer system 202 determining that a short-lived message group 504 has expired (e.g., is no longer accessible), the short-lived timer system 202 communicates with the messaging system 100 (and, in particular, the messaging client 104) to cause the markers (e.g., icons) associated with the relevant short-lived message group 504 to no longer be displayed in the user interface of the messaging client 104. Similarly, when the short-lived timer system 202 determines that the message duration parameter 506 for a particular short-lived message 502 has expired, the short-lived timer system 202 causes the messaging client 104 to no longer display the markers (e.g., icons or text identifiers) associated with the short-lived message 502.

[0098] Use multi-source geolocation data to determine location.

[0099] Figure 6 The location data source 600 of mobile device 602 is shown according to some examples. According to some examples, mobile device 602 is... Figure 1 The client device 102. According to some examples, the mobile device 602 is... Figure 11The wearable electronic device 602 is in the form of glasses 1100, which may be augmented reality (AR) glasses. The mobile device 602 communicates with a positioning system, which includes Global Navigation Satellite System (GNSS) satellites 604, a host device 605, a wireless device 608, and a Pedestrian Dead Retrieval (PDR) sensor 610. The positioning system provides location data 612, 614, 616, 618 or enables the mobile device 602 to determine location data 612, 614, 616, 618. For example, location data 614 from the host device 605 may be Auxiliary GNSS (AGNSS) data, Internet Protocol (IP) location, the location of the host device 605, or the location of the host device 605 with an estimated distance from the mobile device 602 to the host device, etc. Location data 612, 614, 616, 618 are further discussed below with reference to Table 2.

[0100] GNSS satellite 604 is one or more satellites with which mobile device 602 communicates to determine location data 612. Mobile device 602 uses the differences in reception times from different GNSS satellites 604 and the known positions of GNSS satellites 604 to determine location data 612. In some examples, the GNSS data is the known position of GNSS satellite 604. Additionally, according to some examples, host device 605 is configured to determine its own location data 612 from GNSS satellite 604.

[0101] According to some examples, host device 605 is a paired smartphone device or companion device that provides services to mobile device 602. In some examples, host device 605 scans and collects data from nearby wireless devices (such as access points (APs) of IEEE wireless networks, 3GPP wireless networks, or next-generation base stations (BSs)), determines the location of the AP or BS, and provides one or more of the locations of the AP or BS to mobile device 602 as location data 614. Host device 605 determines the location of the AP or BS by performing a lookup of the location of the AP or BS in database 120, requesting the location of the AP or BS from a server device, or determining the location of the AP or BS based on the known location of host device 605.

[0102] Wireless device 608 can be configured to operate according to one or more communication standards such as IEEE 802, 3GPP, LTE, Advanced LTE, 5G communication, etc. Low energy consumption Wireless devices, such as wireless devices 608 and mobile devices 602, are used to perform operations. In some examples, wireless device 608 is a 3GPP BS, 5G BS, IEEE AP, or IEEE non-AP station. Wireless device 608 and mobile device 602 are configured to operate according to one or more communication protocols to determine the location of mobile device 602. For example, the communication protocol may be IEEE 802.11az, WiFi Location Services (WFPS), a proprietary protocol, or another communication protocol used for location determination. Wireless device 608 may be multiple devices. For example, wireless device 608 may be two IEEE 802.11az APs that perform triangulation methods together with mobile device 602 to determine the location of mobile device 602.

[0103] The PDR sensor 610 is part of the mobile device 602 and generates position data 618 based on the motion of the mobile device 602. The PDR sensor 610 includes sensors such as a gyroscope and generates position data 618 to estimate the distance and direction of movement of the mobile device 602.

[0104] Mobile device 602 sends request 620 to a positioning system, such as GNSS satellite 604, host device 605, wireless device 608, or PDR sensor 610, via communications 622, 624, 626, and 628, respectively. In response to request 620, the positioning system, such as GNSS satellite 604, host device 605, wireless device 608, or PDR sensor 610, sends communications 622, 624, 626, and 628, respectively, including location data 612, 614, 616, and 618. In some examples, location data 612, 614, 616, and 618 are sent without request 620.

[0105] Mobile device 602 sends a request 620 to the positioning system or to a module within mobile device 602 for location data 612, 614, 616, and 618. Table 1 provides the characteristics of the positioning system. In Table 1, the characteristics include accuracy, latency, power consumption, indoor / outdoor location, and usage conditions. These characteristics are for the location data indicating the positioning system. These characteristics may differ for different types of location data.

[0106] Table 1 lists the conditions or prerequisites necessary for using the positioning system or for using the positioning system more efficiently in terms of power consumption or other operational characteristics. The conditions of use in Table 1 include an antenna with a high-quality signal or high signal-to-noise ratio, Bluetooth connectivity, availability of the wireless protocol, and prior positioning. Other conditions of use not listed in Table 1 include the presence of host device 605 or paired mobile device, an application running on host device 605 to respond to or service mobile device 602, and indoor or outdoor status. Additional conditions of use may be used. Examples of conditions of use are given below. Mobile device 602 requires a significant amount of power to receive the signal and determine location data 612 from the GNSS satellite 604 signal. Less power is required if the GNSS satellite 604 signal is strong. To reduce power consumption, mobile device 602 avoids using GNSS satellite 604 unless the antenna used to receive the GNSS satellite 604 signal indicates a high signal-to-noise ratio. In some examples, mobile device 602 avoids using the GNSS satellite 604 signal unless mobile device 602 is outdoors to increase the variation in the GNSS satellite 604 signal, which would result in a high signal-to-noise ratio.

[0107] According to some examples, mobile device 602 is a low-power device that relies on a battery.

[0108]

[0109] Referring to Table 1, in some examples, the positioning system GNSS satellite 604 has the following characteristics: the determined location data 612 is GNSS data; it has higher accuracy compared to some other positioning systems; it has higher latency because it takes a relatively long time to obtain positioning and determine or receive location data 612; it requires more power compared to some other positioning systems; the mobile device 602 needs to be outdoors to receive GNSS satellite 604 signals and reduce power consumption when determining location data 612; and the usage condition is an antenna with a high signal-to-noise ratio when receiving GNSS satellite 604 signals. Additional characteristics of GNSS satellite 604 include: no need for internet or Bluetooth. TM The connection; the mobile device 602 requires a GNSS receiver 720 and a GNSS receiver module 727; and the AGNSS data required to obtain the location data 612 is time-consuming and can be obtained from the host device 605 or the GNSS satellite 604.

[0110] Referring to Table 1, in some examples, host device 605 has the following characteristics for AGNSS data: high accuracy, because the data is used in conjunction with GNSS satellite 604; low power consumption, because AGNSS data and requests 620 are processed using technologies such as Bluetooth LE. TMTransmitted using the low-energy wireless protocol; moderate latency for receiving AGNSS data, as the low-energy wireless protocol has higher latency than other wireless protocols; using Bluetooth LE. TM Low power requirements; AGNSS data can be provided by host device 605 indoors or outdoors, although host device 605 may be better able to collect AGNSS data outdoors; and, requires mobile device 602 and host device 605 to communicate via, for example, Bluetooth. TM The device communicates via a wireless connection and may require software modules or applications to run on host device 605 to provide services to mobile device 602.

[0111] Referring to Table 1, in some examples, host device 605 has the following characteristics for other location data 614: variable accuracy, because host device 605 can provide location data 614 in several different ways with different accuracies as described herein; variable latency, because host device 605 can use high-power wireless connectivity or low-power wireless connectivity; variable power usage, because host device 605 can use high-power wireless connectivity or low-power wireless connectivity; host device 605 can be connected to mobile device 602 indoors or outdoors; and, host device 605 is required to connect to mobile device 602 via a wireless connection, and may require software modules or applications to run on host device 605 to provide services to mobile device 602.

[0112] In some examples, wireless device 608 has the following characteristics for WFPS location data 616: it is based on triangulation of signal strength or time of flight when packets are sent and received between two or more wireless devices 608 and mobile device 602; some of the communication protocols used have high accuracy; there is moderate latency based on packets being sent and received between mobile device 602 and wireless device 608; low power consumption; typically, the protocol 616 for determining WFPS location data is only available indoors; and there is a requirement for the availability of the wireless communication protocol. In some examples, information about the location of wireless device 608 is required to receive or determine location data 616. For example, some WFPS location data 616 requires the location of an access point (AP), and the AP's location is stored in a database accessible via the Internet. The AP's database may include billions of mapped wireless networks, which may be referred to as WiFi networks. Storing information about mapped WiFi networks on mobile device 602 is not feasible due to storage, processing, and updating requirements. Access to the Internet can provide the information required to perform WFPS without requiring large amounts of storage. In some examples, host device 605 provides mobile device 602 with information about the mapped WiFi network. According to some examples, host device 605 is client device 102.

[0113] In some examples, wireless device 608 uses other protocols to determine location data 616 or enables mobile device 602 to determine location data 616. In some examples, wireless device 608 is used to receive or determine other types of location data 616. For example, location protocols for 5G networks, IEEE 802.11az, proprietary protocols, etc., are used to determine location data 616. In some examples, wireless device 608 is configured to operate according to wireless communication protocols such as IEEE 802.11, IEEE 802.11az, IEEE 802.11be, etc. According to some examples, wireless device 608 is used as an access point (AP), station (STA), or base station.

[0114] In some examples, accessing the PDR sensor 610 to determine the PDR location data 618 has the following characteristics: the location data 618 from the PDR sensor 610 can be used to detect movement of the mobile device 602; the accuracy of the PDR location data 618 is relatively low because it is based on dead reckoning; the latency is low because the PDR sensor 610 is part of the mobile device 602; the power requirements are low because the PDR sensor 610 requires less power to operate compared to other positioning devices such as the GNSS receiver 720; the PDR sensor 610 operates both indoors and outdoors; and the location data 612 needs to be supplemented because it only provides the offset from the last known location in terms of distance and direction, and the PDR sensor 610 requires strong or high signal-to-noise ratio conditions to operate correctly.

[0115] According to some examples, when the PDR sensor 610 detects motion, that motion is then used to determine whether the position has changed. For example, the PDR sensor 610 may detect motion that instructs the mobile device 602 to move to the left and then to the right to bring it to the same position. According to some examples, the accuracy of the PDR varies depending on the known wearer gait and stride length calibration, as well as the activity to be determined, such as walking, running, etc.

[0116] A location source provides location data 612, 614, 616, and 618 to mobile device 602, wherein location data 612, 614, 616, and 618 indicate data related to the location of mobile device 602. In some examples, location data 612, 614, 616, and 618 include one or more location data components as described in Table 2. According to some examples, location data 612, 614, 616, and 618 are two-dimensional (D), 3D, 4D, or 5D. For example, altitude and location are not included in some location data 612, 614, 616, and 618.

[0117]

[0118]

[0119] According to some examples, the PDR sensor 610 provides two-dimensional (2D) offset, heading, and steps from the starting position. In some examples, the PDR sensor 610 operates continuously and is therefore useful for filling gaps between updates from other positioning systems that require more power or have higher latency.

[0120] In some examples, location data 612, 614, 616, and 618 are not sent after the positioning system has received request 620. For example, a location source such as PDR sensor 610 may not be operating properly, and therefore it may not respond to request 620. The positioning system may not respond with location data 612, 614, 616, and 618 because one of its requirements has not been met. See the requirements column in Table 1 for example. Furthermore, internet access or quality may be too low for a location source such as wireless device 608 to function. According to some examples, host device 605 does not provide location data 614 unless mobile device 602 is paired with host device 605. For example, host device 605 and mobile device 602 may not have Bluetooth. TM The connection, or the quality of the wireless connection, may be too poor to transmit the data used for pairing mobile device 602 with host device 605. In some examples, host device 605 provides location data 614 from another source. For example, host device 605 uses wireless device 608 or GNSS satellite 604 to determine its own location and then sends location data 614 indicating the location of host device 605 to mobile device 602. Host device 605 may use other location sources to determine its location and send that location in the location data 614 to mobile device 602. In some examples, host device 605 sends an estimate of how far mobile device 602 is from host device 605 to mobile device 602, so that mobile device 602 can use this estimate to determine its location based on the estimated distance between mobile device 602 and host device 605, and the location of host device 605. According to some examples, the estimate of how far mobile device 602 is from host device 605 is based on the delay in the wireless communication between mobile device 602 and host device 605. This estimate is based on the strength of the Received Signal Strength Indicator (RSSI) of the received signal and the power indication of the transmitted signal. The mobile device 602 or the host device 605 estimates the distance based on the transmit power used to transmit the signal and the RSSI, where RSSI is the power of the received signal.

[0121] Host device 605 sends data to mobile device 602 to assist it in performing operations on GNSS satellite 604. For example, host device 605 sends almanac information to mobile device 602 for performing GPS estimation, so that mobile device 602 does not need to download almanac information from GNSS satellite 604. In some examples, host device 605 sends other information, such as information about access points in an IEEE 802 network or base stations in a 3GPP or 5G network.

[0122] In some examples, location sources provide location data 612, 614, 616, and 618, which provide the location of mobile device 602 regardless of its orientation. Additionally, according to some examples, a location source, as part of mobile device 602, provides orientation information to mobile device 602. In some examples, PDR sensor 610 provides additional location data, including the orientation of mobile device 602. In some examples, mobile device 602 uses location data 612, 614, 616, and 618 in response to changes in geographic location and uses other devices to determine the orientation of mobile device 602.

[0123] Figure 7 A system 700 for determining location using multi-source geographic location data is shown, based on some examples. Figure 7 The image shows a mobile device 602 having a display 702 and peripheral devices 704. The display 702 is a display as described herein, and the peripheral devices 704 include sensors, speakers, a PDR sensor 610, a GNSS receiver 720, wireless components 723, etc., as described herein. According to some examples, the mobile device 602 is a client device 102. According to some examples, the mobile device 602 is... Figure 11 The glasses 1100 can be AR glasses. According to some examples, the mobile device 602 is a wearable device.

[0124] In some examples, location data 612 is received by GNSS receiver 720 and processed by GNSS receiver module 727. In some examples, location data 614 from host device 605 is received by wireless component 723 and processed by wireless component module 718. In some examples, location data 616 from wireless device 608 is received by wireless component 723 and processed by wireless component module 718. In some examples, location data 618 is generated by PDR sensor 610 and processed by PDR sensor module 728.

[0125] Application module 706 is an application such as an AR application or other application using the current location 708 of mobile device 602. Application module 706 accesses the current location 708, accuracy 709, timestamp 732, and other associated data, and determines whether the current location 708 is sufficient. If the current location 708 is insufficient, application module 706 submits an update location request 734 to location module 722. Update location request 734 optionally indicates the priority 736 of needing to update the current location 708. Update location request 734 optionally includes indications of the required accuracy 738 and freshness 740 for the current location 708.

[0126] Example application module 706 is in Figure 11 The application of projection direction on the optical elements 1143 and 1144 of the AR glasses. Current location 708 may include multiple fields or data associated with current location 708, such as accuracy 709 and timestamp 732, or other data such as those shown in the location data components in Table 2. In some examples, application module 706 may be associated with, for example... Figure 11 The camera 1169 is associated with the camera, and when video or photos are captured, the application module 706 can use the current location 708 or send an update location request 734 to the location module 722.

[0127] Wireless component 723 is configured to communicate with host device 605 and wireless device 608 via a wireless connection. The wireless connection can be, for example... Low-speed connections or high-speed communication protocols such as IEEE 802.11, 3GPP, 5G, WiFi, cellular modems, or other communication protocols. In some examples, the wireless component module 718 is configured to perform WFPS with one or more wireless devices 608 to provide location information based on triangulation. In some examples, the wireless devices 608 are two or more APs configured to operate according to IEEE 802.11az to determine the location of the mobile device 602. Other location protocols are associated with 3GPP, and proprietary protocols are available that include other wireless devices 608 located near the mobile device 602 to provide location information, such as home transmitter location systems. In some examples, the wireless component 723 operates using light, where the peripheral device 704 includes a light sensor.

[0128] In some examples, the wireless component module 718 scans for access points (APs) and their addresses, such as Basic Service Set (BSS) identifiers (BSSIDs), signal strength, frequency, and home channel. The wireless component module 718 may perform the scan in response to an update location request 734 sent by the application module 706 to the location module 722. In some examples, the host device 605 performs the scan and sends information or a portion of that information to the mobile device 602. In some examples, the scan does indeed preserve a set of Service Sets (SSIDs) for the wireless device 608. According to some examples, the information sent by the host device 605 to the mobile device 602 includes a list of APs. In some examples, the host device 605 provides an Application Programming Interface (API) to the mobile device 602. For example, the `getGeoLocationFromWFPS()` method, where the mobile device 602 provides an AP token to the host device 605 via the API; and the host device 605 returns the location of the AP corresponding to the AP token to the mobile device 602. The host device 605 may be located at the same location as or nearly at the same location as the mobile device 602. For example, host device 605 could be a smartphone, and mobile device 602 could be AR glasses. In some examples, wireless component module 718 determines the location from host device 605 based on the delay of the wireless signals exchanged between the two wireless devices, and uses the determined location to correct the location given by host device 605. For example, host device 605 sends its current location to mobile device 602, and mobile device 602 determines that it is within a few meters of host device 605.

[0129] The current location 708 can be estimated based on the exchanged light. For example, mobile device 602 exchanges light with another device and uses the delay in receiving the response, as well as the time to process and send the response, to determine the distance to the other device. If there are more than one other device or optical sensor with which mobile device 602 can exchange light, triangulation is used. GNSS receiver 720 communicates with GNSS satellite 604. Location module 722 is a centralized entity for acquiring, managing, and integrating current location 708 data. According to some examples, the services provided by location module 722 are referred to as location services by application module 706.

[0130] In some examples, the location module 722 maintains the current location 708 and schedules updates to the current location 708 so that the current location 708 is available once requested by the application module 706. In some examples, the location module 722 responds to the update location request 734 for the current location 708 by sending a request to cause the update scheduler module 712 to issue a request 620 for location data 612, 614, 616, 618 to the location sources in the location source 724. According to some examples, the update scheduler module 712 issues the request 620 for location data 612, 614, 616, 618 via the GNSS receiver module 727, the PDR sensor module 728, or the radio component module 718.

[0131] In some examples, when application module 706 requests the current location 708, update scheduler module 712 does not send request 620 to one of the location sources 724. Update scheduler module 712 represents update location request 734 as a deferable work item, which is scheduled to be executed whenever condition 726 indicates that request 620 should be sent to one of the location sources 724. Condition 726 depends on the location source 724 to be used. Condition 726 includes conditions indicating the possibility of obtaining location data 612, 614, 616, 618, conditions indicating the current system load such as display status 716, usage conditions of Table 1, and other conditions 730.

[0132] Other conditions 730 include one or more of the following: WiFi status, wireless status of wireless component 723, user account and pairing status with host device 605, overall system load of mobile device 602, battery status of mobile device 602, other conditions 730 described in conjunction with Tables 2 and 4, status of GNSS receiver 720, status of PDR sensor 610, and speed of mobile device 602, which can be categorized as a mode of transportation such as walking, running, cycling, flying, taking a train, taking a bus, or driving. In some examples, wireless component module 718 is configured to use IP address location to determine location data, which requires internet access, where a server provides location data based on the IP address of mobile device 602. IP address-based location data can be, for example, a location within a university or a city.

[0133] According to some examples, one or more of the location sources 724 may be unavailable some of the time or most of the time. Some location sources 724 are more frequently available than others. For example, location data 614 from paired host device 605 is accurate but not always available because it requires mobile device 602 to be paired with host device 605. In some examples, host device 605 needs to run a software application that provides location information to host device 605. Table 4 provides additional information about location sources 724.

[0134]

[0135]

[0136] According to some examples, when the mobile device 602 is started, the location module 722 attempts to satisfy the initial location update request 734 by querying all location sources 724. In some examples, the location module 722 prioritizes the location sources 724 and selects the highest priority location source that meets condition 726 for sending request 620 to that location source. The location module 722 continues down the priority list until it receives location data 612, 614, 616, 618 that meet the conditions of the location update request 734 (e.g., required accuracy 738 and freshness 740 below a threshold).

[0137] In some examples, after determining the first current location 708, subsequent location data 612, 614, 616, and 618 are integrated as follows. The update scheduler module 712 dynamically adjusts the query rate 737 to the location source 724 based on the speed of the mobile device 602. The update scheduler module 712 uses the PDR sensor 610 via the PDR sensor module 728 to determine whether the mobile device 602 is stationary. For example, if the mobile device 602 is an AR device and the user is using it, the update scheduler module 712 uses the PDR sensor 610 to determine whether the mobile device 602 is stationary but its orientation is changing. In some examples, when the mobile device 602 is stationary but its orientation is changing, the update scheduler module 712 does not update the current location 708.

[0138] In some examples, the update scheduler module 712 increases the query rate 737 based on the speed of the mobile device 602 being used by the user. In some examples, the update scheduler module 712 selects a location source from location source 724 to minimize or reduce the power consumption used to determine the current location 708, wherein the location source 724 meets the conditions required to update the current location 708. For example, the location module 722 may receive an update location request 734 indicating an accuracy of one meter, and the accuracy returned by the IP address location is 100 meters. In this case, the update scheduler module 712 will not use the IP address location.

[0139] In some examples, the update scheduler module 712 executes interleaved requests 620 for location data 612, 614, 616, and 618 to avoid causing a surge in power consumption. In some examples, the update scheduler module 712 schedules request 620 to be executed when the mobile device 602 is not in a sleep or nap interval, which can extend the battery life of the mobile device 602.

[0140] In some examples, the update scheduler module 712 schedules request 620 to be executed while the operating system of mobile device 602 is in a maintenance window. Mobile device 602 is powered on during the maintenance window, so mobile device 602 does not need to be powered on or enter a normal power-on state when request 620 is sent.

[0141] In some examples, the update scheduler module 712 schedules request 620 to one of the location sources 724, where request 620 is sent to the location source when condition 726 for sending request 620 to the location source is met, and the location source provides location data that meets the requirements associated with update location request 734. Since the fusion module 710 fuses the new location data 612, 614, 616, 618 with the current location 708, it determines whether the location source will provide location data that meets the requirements associated with update location request 734 based on the current location 708, update location request 734, and location data 612, 614, 616, 618.

[0142] In some examples, the update scheduler module 712 will send a request 620 to the location source based on the fulfillment of condition 726, and then cancel the request 620 if condition 726 is no longer met. For example, if a user of mobile device 602 begins to interact with mobile device 602 whose display state 716 has been changed to on, the update scheduler module 712 cancels the request 620, where the location source's condition 726 is that the display state 716 is off.

[0143] The fusion module 710 merges two or more location data 612, 614, 616, 618 or the current position 708 with the location data 612, 614, 616, 618 to determine the updated current position 708. According to some examples, the fusion module 710 uses equation (1) to determine the updated current position 708. In some examples, the fusion module 710 is configured to perform... Figure 8 Method 800.

[0144] The current position 708 and position data 612, 614, 616, and 618 are integrated and converted into a common parameter set or common coordinate system before being merged. As described in Table 2, the position data 612, 614, 616, and 618 include one or more components indicating the position.

[0145] Elevation and location are included in some location data 612, 614, 616, 618. Therefore, when some location data 612, 614, 616, 618 include elevation or location while other location data 612, 614, 616, 618 do not, location data 612, 614, 616, 618 may need to be merged or combined.

[0146] The update scheduler module 712 schedules requests 620 that can be sent in the background. As the mobile device 602 moves, it receives location data updates 612, 614, 616, and 618 in the background. Depending on system resource availability, some or all of the location sources 724 will be periodically queried via request 620. When the mobile device 602 powers on, the fusion module 710 is initialized using the first current location 708. Subsequent updates of location data 612, 614, 616, and 618 are fused with the current location 708 to generate an updated current location 708. According to some examples, the update scheduler module 712 is configured to perform... Figure 10 Method 1000. Table 3 shows the location availability of the paired application module on host device 605 and the location availability of application module 706.

[0147]

[0148]

[0149] Figure 8The operation of method 800 for merging locations is illustrated according to some examples. According to some examples, merging module 710 is configured to execute method 800. Method 800 begins at operation 802 with accessing location data. For example, merging module 710 receives location data 612, 614, 616, 618 from update scheduler module 712. In some examples, merging module 710 sends a command to update scheduler module 712 indicating that a first current location 708 is being determined. According to some examples, update scheduler module 712 then sends requests 620 to one or more location sources 724 to obtain location data 612, 614, 616, 618. Method 800 continues to determine the current location at operation 804. For example, if location data 612, 614, 616, 618 are the first location data 612, 614, 616, 618, the fusion module 710 uses location data 612, 614, 616, 618 to determine the current location 708 without fusing them. According to some examples, the fusion module 710 can use the received location data 612, 614, 616, 618, or convert it into different representations. Example representations of location data 612, 614, 616, 618 include Cartesian coordinates, polar coordinates, splines, ellipsoidal coordinates, oblong coordinates, GPS (Global Positioning System) coordinates represented as a combination of latitude and longitude, etc.

[0150] Method 800 continues accessing new location data at operation 806. The fusion module 710 responds to the reception of new location data 612, 614, 616, 618 by accessing the new location data 612, 614, 616, 618. In some examples, the fusion module 710 delays its response until the power-saving mode ends or until the power consumption level of the mobile device 602 drops below a threshold. The method continues to determine whether the new location is valid at operation 808. For example, an estimated speed is determined based on the new location data 612, 614, 616, 618, the current location 708, and the time difference indicated by the timestamps associated with the new location data 612, 614, 616, 618, and the current location 708. If the estimated speed does not exceed a threshold for usage, the new location data is valid; otherwise, the new location data is considered invalid and discarded. Usage is an estimate of the activity being performed by the user of the mobile device 602, such as being stationary, running, cycling, riding a train, riding a bus, walking, flying in an airplane, driving, etc. Based on some examples, usage is determined based on previously estimated speeds. Other methods can be used to determine whether the new location data 612, 614, 616, and 618 are valid.

[0151] Method 800 continues to check whether the new location data is valid at operation 810. If the new location data is invalid, the fusion module 710 returns to operation 806 and waits for or processes the new location data 612, 614, 616, and 618.

[0152] If the new location data is valid, method 800 continues at operation 812 to fuse the new location data with the current location to generate an updated current location. According to some examples, the new location data 612, 614, 616, 618 are fused with the current location 708 based on a weighted sum of the new location data 612, 614, 616, 618 and the current location 708. In some examples, equation (1) is used as shown below to determine the fused or updated current location 708 based on the current location 708 as location 1 and the new location data 612, 614, 616, 618 as location 2.

[0153] Equation (1): x, y, z = ((x1, y1, z1) * (1 / accuracy 1) + (x2, y2, z2) * (1 / accuracy 2)) / (Δ distance), where x, y, z are the coordinates of the merged current position 708 being determined; x1, y1, z1 are the coordinates of position 1, which is the previously determined current position 708; x2, y2, z2 are the coordinates of position 2, which is the new position data 612, 614, 616, 618; accuracy 1 is the accuracy of position 1; accuracy 2 is the accuracy of position 2; and Δ distance is the estimated distance or Euclidean distance that the mobile device 602 has moved between position 1 and position 2. In some examples, "*" is referred to as multiplication or multiplication and represents the mathematical operation of multiplication. x can be called x value; y can be called y value; x1 can be called x1 value; z can be called z value; y1 can be called y1 value; z1 can be called z1 value; z2 can be called z2 value; y2 can be called y2 value; and x2 can be called x2 value.

[0154] Method 800 continues at operation 814 to determine whether the new location data includes one or more optional fields. Based on some examples, location data may include elevation or location as optional location data. Table 2 shown above lists some of the fields that may be included in the location data.

[0155] When the new location data does not include optional fields, method 800 continues at operation 818 to determine whether the optional fields of the current location are valid. For example, if the new location data 612, 614, 616, 618 do not have altitude, but the current location 708 does, then if the altitude is still valid, the mobile device 602 uses the current altitude value from the current location 708 in the merged or updated current location. Whether the altitude is still valid can be determined as follows: if the distance between the new location data 612, 614, 616, 618 and the current location 708 is less than a threshold (where the threshold can be 100 meters or another number such as 1 meter to 10,000 meters), then the previous current altitude value is valid and used as the current altitude value in the updated current location; otherwise, the system sets the altitude value of the updated current location to unknown, which can be represented by a null value. In some examples, the optional fields of the current location are determined to be valid or invalid based on the timestamp 732 of the current location 708. For example, if the current location 708 was determined before a threshold number of minutes had elapsed, the optional data for the current location 708 is not used. In some examples, the term "beyond" is used to indicate a value greater than the threshold. In some examples, other conditions are used to determine whether an optional field is valid. In some examples, a combination of conditions is used to determine whether an optional field is valid. For example, a combination of the distance change from the updated current location to the current location and the time in seconds between determining the updated current location and the current location could be used. The location or current location can be referred to as a location value or current location value.

[0156] In one example, as shown below, the fusion module 710 merges locations such as streets, cities, states, or countries. If new location data 612, 614, 616, 618 do not have a location, but the current location 708 does, the system uses the location from the current location 708 in the updated current location 708 based on the following: If the distance between the new location data 612, 614, 616, 618 and the current location 708 is less than a threshold (where the threshold can be 100 meters or another number such as 1 meter to 10,000 meters), then the mobile device 602 uses the location from the current location 708; otherwise, the mobile device 602 indicates that the location of the merged or updated current location is unknown, which can be represented by a null value. The validity of optional fields can be determined in another way. According to some examples, the threshold for a location is based on the type of location. For example, a street location might have a threshold of 10 meters, while a city location might have a threshold of 1,000 meters.

[0157] If the optional field for the current position is valid, method 800 proceeds to operation 820, updating the current position using the optional field. As described above, the optional field is used for the updated current position. Then, method 800 proceeds to operation 822. If the optional field for the current position is invalid, the optional field for the updated current position is set to unknown or empty as described above. Then, method 800 proceeds to operation 822.

[0158] If the new location data does indeed include optional fields, then method 800 continues in operation 816, and the system uses the optional fields for the updated current location. For example, if the new location data 612, 614, 616, 618 contains altitude, then the system uses that altitude for the updated location data 612, 614, 616, 618. In another example, if the new location data 612, 614, 616, 618 contains location, then the system uses that location for the updated location data 612, 614, 616, 618.

[0159] Method 800 continues from operations 816 and 820 to operation 822, setting the current position to the updated current position. For example, setting the current position 708 to the updated current position determined by executing method 800.

[0160] One or more operations in method 800 may be optional. For example, operations 814 to 820 may be optional. Method 800 may include one or more additional operations. The operations of method 800 may be performed in different orders.

[0161] Figure 9 Example 900 of fused location data is shown based on some examples. In example 900, mobile device 602 moves around and determines current location 708 (e.g., initial location-1 902, fused current location-2 910, fused current location-3 916, and fused current location-4 922) using the method described herein, wherein current location 708 is determined based on received location updates (e.g., initial location-1 902, location update-2 906, location update-3 912, and location update-4 918).

[0162] Example 900 begins with receiving location data and determining an initial location -1 902. For example, operations 802 and 804 of method 800 can be performed on new location data received when mobile device 602 is powered on or when mobile device 602 wakes up from a sleep period. Accuracy -1 904 indicates the accuracy associated with the initial location -1 902.

[0163] Example 900 continues at position update -2 906, where new position data with an associated accuracy of -2908 is received. The fusion module 710 determines the fused current position -2 910. For example, the fusion module 710 can perform method 800 to determine the fused current position -2 910 using equation (1) above, where position 1 is the initial position -1 902 and position 2 is position update -2906. Example 900 continues at position update -3 912, where new position data with an associated accuracy of -3914 is received. The fusion module 710 determines the fused current position -3 916. For example, the fusion module 710 can perform method 800 to determine the fused current position -3 916 using equation (1) above. Example 900 continues at position update -4918, where new position data with an associated accuracy of -4 920 is received. The fusion module 710 determines the fused current position -4922. For example, the fusion module 710 can execute method 800 to determine the fused current position -4 922 using the above equation (1). According to some examples, the initial position -1 902, position update -2 906, position update -3 912, and position update -4 918 are position data 612, 614, 616, and 618 received in response to the update scheduler module 712 sending a request 620.

[0164] Figure 10 A method 1000 for scheduling requests is illustrated according to some examples. In some examples, an update scheduler module 712 is configured to execute method 1000. Method 1000 begins at operation 1002 by querying the PDR. For example, the update scheduler module 712 may periodically query the PDR or query the PDR sensor 610 location data from the PDR sensor module 728 in response to events such as system startup or the end of a sleep command. The location data 618 from the PDR sensor 610 indicates whether the mobile device 602 has moved since the last query. Method 1000 continues to update the query rate at operation 1004. For example, the update scheduler module 712 maintains a query rate 737, which indicates how often a request 620 should be sent to one of the location sources 724. According to some examples, the query rate 737 indicates how often method 1000 should be executed. In some examples, the query rate 737 indicates how often an update to the current location 708 should be performed. In some examples, the query rate 737 is not updated every time the PDR is queried. In one example, if the PDR location data 618 indicates that the speed of the mobile device 602 is faster or slower than the speed indicated by the previous PDR location data 618, the query rate 737 can be increased or decreased.

[0165] Method 1000 continues to determine whether the device is stationary at operation 1008. If the location data 618 from the PDR sensor 610 indicates that the mobile device 602 is stationary, method 1000 returns to operation 1002. A pause or delay may be present, preventing the PDR query operation 1002 from being continuously executed. In some examples, operation 1002 is performed based on an interrupt or a combination of an interrupt and a loop with a pause or delay. For example, an interrupt from the operating system, such as a startup or end of a sleep state, or an interrupt indicating that the PDR sensor 610 has new location data 618 or that the location data 618 has changed.

[0166] If the mobile device 602 is not stationary, method 1000 continues at operation 1010 to determine whether the query rate indicates that an update is needed. For example, if the query rate indicates that not enough time has passed to query new location data 612, 614, 616, method 1000 returns to operation 1002. If enough time has passed, method 1000 continues to operation 1012.

[0167] In some examples, operation 1010 includes an additional test to determine whether the current position 708 is sufficient to satisfy an update position request 734 with priority 736, accuracy 738, and freshness 740. For example, before determining whether to continue operation 1012, it is determined whether the accuracy 709 of the current position 708 is sufficient to satisfy an pending update position request 734 from application module 706 with the requested accuracy 738. If the accuracy 709 is insufficient, method 1000 continues to operation 1012.

[0168] In some examples, operation 1008 includes an additional condition regarding whether mobile device 602 is being worn, before determining whether to proceed to operation 1010. In some examples, if mobile device 602 is not being worn, method 1000 returns to operation 1002. If the system determines that mobile device 602 is being worn, the method proceeds to operation 1012.

[0169] In some examples, as part of operation 1010, the fusion module 710 updates the current location 708 based on new location data 618 from the PDR sensor 610. In some examples, the test of whether the query rate indicates that an update is needed is also based on how much the mobile device 602 has moved based on the location data 618 from the PDR sensor 610.

[0170] If the query rate indicator indicates that sufficient time has elapsed for another request 620 for new location data 612, 614, 616, then method 1000 continues to select a location system at operation 1013.

[0171] The positioning system includes one or more of the following: a location source 724, hardware components as part of a mobile device 602, and external devices required for determining or receiving location data 612, 614, 616, 618. For example, the positioning system 612 for GNSS satellite location data includes one or more of the following: a GNSS satellite 604, a GNSS receiver module 727, and a GNSS receiver 720.

[0172] The location system is selected based on one or more of the following: the default order of location systems, current location 708, location update request 734, current activity associated with mobile device 602, battery power level associated with mobile device 602, condition 726, etc. According to some examples, the order of location systems is selected based on which location system can use the least amount of power to satisfy location request 734. For example, if the accuracy 738 requested by application module 706 is three feet, location data 612, 614, 616, 618 with an accuracy 709 within three feet can be provided, and the location system using the least power is placed as the first location system.

[0173] In some examples, the default selection of the positioning system is used to reduce the computation time used to determine the selection order. In some examples, the default selection of the positioning system is used unless the update location request 734 indicates a value exceeding the thresholds of priority 736, accuracy 738, and freshness 740 associated with the default selection of the positioning system. In some examples, operation 1013 is optional. In some examples, there is a fixed or set number of positioning systems, and the order in which they are tested for whether they should be queried is fixed. In some examples, another module determines the priority used to determine whether to query a positioning system. According to some examples, when the location data 618 from the PDR sensor 610 satisfies the update location request 734, the current location 708 is updated with the location data 618 from the PDR sensor 610, so that method 1000 does not perform operation 1013.

[0174] In the following text, an example order of the positioning systems selected by the update scheduler module 712 is wireless device 608, host device 605, and GNSS satellite 604. Method 1000 continues at operation 1012 to determine whether the conditions for using the first positioning system are met. According to some examples, the first positioning system is wireless device 608. For example, the update scheduler module 712 may query the wireless component module 718 to see if WFPS is available. If WFPS is available, method 1000 continues at operation 1014 to query the first positioning system. For example, a request 620 is sent to the wireless component 723 to perform WFPS with wireless device 608 to obtain location data 616. As described herein, location data 616 is determined using a method performed with one or more other wireless devices using a communication protocol for determining location. When location data 616 is received, the update scheduler module 712 sends location data 616 to the fusion module 710 to update the current location 708. Upon receiving location data 616, the update scheduler module 712 sends location data 616 to the fusion module 710 to update the current location 708. In some examples, method 1000 returns from operation 1014 to operation 1002. In some examples, method 1000 continues from operation 1014 to operation 1016. In some examples, method 1000 continues from operation 1014 to operation 1024.

[0175] Wireless device 608 provides other methods for determining location data 616 as described herein. According to some examples, the different methods each have different conditions and are prioritized by update scheduler module 712.

[0176] If the conditions for using the first positioning system are not met, method 1000 continues at operation 1016 to determine whether the conditions for using the second positioning system are met. In the example, the second positioning system is host device 605, and the condition is whether mobile device 602 is paired with host device 605. For example, update scheduler module 712 may query wireless component module 718 whether mobile device 602 is paired with host device 605. If mobile device 602 is paired with host device 605, method 1000 continues to operation 1018 to query the second positioning system. For example, update scheduler module 712 may send a request 620 for location data 614 to host device 605. When location data 614 is received, update scheduler module 712 sends location data 614 to fusion module 710 to update the current location 708. In some examples, method 1000 continues from operation 1018 back to operation 1002. In some examples, method 1000 continues from operation 1018 to operation 1020. In some examples, method 1000 continues from operation 1018 to operation 1024.

[0177] If the conditions for using the second positioning system are not met, method 1000 continues at operation 1020 to determine whether the conditions for using the third positioning system are met. For example, if mobile device 602 is not paired with host device 605, method 1000 continues at operation 1020 to determine whether the conditions for the third positioning system are met. For example, update scheduler module 712 determines whether the display is off. If the conditions for using the third positioning system are met, method 1000 continues at operation 1022 to query the third positioning system. For example, if the display is off, method 1000 continues to query GNSS. For example, update scheduler module 712 queries GNSS receiver module 727 to send request 620 to GNSS receiver module 727 to determine location data 612. When location data 612 is determined, update scheduler module 712 sends location data 612 to fusion module 710 to update the current location 708. In some examples, method 1000 continues from operation 1018 back to operation 1002. In some examples, method 1000 continues from operation 1022 to operation 1024. In some examples, an additional positioning system is present.

[0178] If the display is not turned off, method 1000 proceeds to operation 1024 to change the condition. In some examples, method 1000 continues from operations 1014, 1018, and 1022 to operation 1024, which changes the condition. In some examples, operation 1024 changes the condition based on whether location data 612, 614, and 616 have been obtained. Figure 7 One or more of the conditions 726. For example, if location data 612, 614, 616 is not obtained, the update scheduler module 712 may request that the display status 716 be changed to off, request the wireless component module 718 to pair with the host device 605, or request the wireless component module 718 to scan for wireless devices 608 that will perform WFPS with the mobile device 602.

[0179] In some examples, the conditions for using the corresponding positioning system operations 1014, 1018, and 1022, listed in operations 1012, 1016, and 1020 respectively, may be different and may include additional conditions as described herein. In some examples, the conditions listed in operations 1012, 1016, and 1020 may include additional conditions, such as the required accuracy prior to using one or more of the location sources 724. In some examples, the conditions listed in operations 1012, 1016, and 1020 may include additional or different conditions prior to using the corresponding location sources of operations 1014, 1018, and 1022.

[0180] One or more operations of method 1000 may be optional. For example, operations 1020 and 1022 may be optional. Method 1000 may include one or more additional operations. The operations of method 1000 may be performed in different orders. In some examples, method 1000 minimizes battery and power usage while maintaining the current position 708 with acceptable accuracy 709 and / or freshness 740.

[0181] Figure 11 Examples of wearable electronic devices in the form of electronically enabled glasses 1100 are shown, according to some examples. The wearable electronic device is an example of a mobile device 602, which is an example form of an eyewear item comprised of electronically enabled glasses 1100, and can also operate within a network system for transmitting image and video content along with associated location information. Figure 11 A front perspective view of glasses 1100 is shown. In some examples, glasses 1100 are referred to as AR glasses. Mobile device 602 may be referred to as AR mobile device 602. Glasses 1100 may include a frame 1132 made of any suitable material, such as plastic or metal (including any suitable shape memory alloy). Frame 1132 may have a front element 1133, which may include a first lens or left lens, a display or optical element holder 1136, and a second lens or right lens, a display or optical element holder 1137 connected by a bridge 1138. Front element 1133 further includes a left end portion 1141 and a right end portion 1142. A first optical element or left optical element 1144 and a second optical element or right optical element 1143 may be disposed within the respective left optical element holder 1136 and right optical element holder 1137. Each of optical elements 1143 and 1144 may be a lens, a display assembly, or a combination thereof. In some examples, for instance, glasses 1100 are provided with an integrated near-eye display mechanism that enables, for example, the display to a user of a preview image of visual media captured by the camera device 1169 of glasses 1100.

[0182] Frame 1132 further includes a left or temple piece 1146 and a right or temple piece 1147, which are coupled to the respective left and right ends 1141 and 1142 of the front piece 1133 by any suitable means such as hinges (not shown), to be coupled to or rigidly or fixably attached to the front piece 1133, thereby becoming integral with the front piece 1133. Each of temple pieces 1146 and 1147 may include a first portion 1151 coupled to the respective end 1141 or 1142 of the front piece 1133 and any suitable second portion 1152 for coupling to the user's ear, such as a bend or bow. In one example, the front piece 1133 may be formed from a single piece of material to have a monolithic or integral construction. In one example, the entire frame 1132 may be formed from a single piece of material to have a monolithic or integral construction.

[0183] The eyeglasses 1100 may include a computing device such as a computer 1161, which may be of any suitable type to be carried by the frame 1132, and in one example, the computing device has a suitable size and shape to be at least partially disposed within one of the temple members 1146 and 1147. In one embodiment, the computer 1161 has a size and shape similar to that of one of the temple members 1146 and 1147, and is therefore disposed, if not completely, almost completely within the structure and boundaries of such temple members 1146 and 1147.

[0184] In one embodiment, computer 1161 may be arranged in both temple members 1146 and 1147. Computer 1161 may include one or more processors with memory, wireless communication circuitry, and a power supply. Computer 1161 includes low-power circuitry, high-speed circuitry, position circuitry, and a display processor. Various other examples may include these elements in different configurations or integrated in different ways. Additional details regarding aspects of computer 1161 may be found in the description below.

[0185] The computer 1161 further includes a battery 1162 or other suitable portable power source. In one example, the battery 1162 is disposed in one of the temple members 1146 or 1147. Figure 11 In the illustrated eyeglasses 1100, battery 1162 is shown disposed in the left temple 1146 and electrically coupled to the remainder of computer 1161 disposed in the right temple 1147 via connector 1174. One or more input and output devices may include connectors or ports (not shown) suitable for charging battery 1162 and accessible from the outside of frame 1132, wireless receivers, transmitters or transceivers (not shown), or combinations of such devices.

[0186] The glasses 1100 include a digital camera device 1169. Although two camera devices 1169 are depicted, other examples contemplate the use of a single or additional (i.e., more than two) camera devices 1169. For ease of description, various features associated with the camera device 1169 will be further described with reference only to a single camera device 1169; however, it should be understood that, in suitable examples, these features may apply to two camera devices 1169.

[0187] In various examples, in addition to the camera device 1169, the glasses 1100 may include any number of input sensors or peripheral devices. The front member 1133 is provided with: an outward-facing, forward-facing, front or outer surface 1166 facing forward or away from the user when the glasses 1100 is mounted on the user's face; and an opposing inward-facing, rearward-facing, rear or inner surface 1167 facing the user's face when the glasses 1100 is mounted on the user's face. Such sensors may include: an inward-facing video sensor or digital imaging module, for example, which may be mounted or positioned on or within the inner surface 1167 of the front member 1133 or at other locations on the frame 1132 facing the user's camera device 1169; and an outward-facing video sensor or digital imaging module, for example, which may be mounted or positioned on or within the outer surface 1166 of the front member 1133 or at other locations on the frame 1132 facing away from the user's camera device 1169. Such sensors, peripheral devices, or peripheral equipment may additionally include biometric sensors, position sensors, accelerometers, or any other such sensors.

[0188] The glasses 1100 also include an example of a camera device control mechanism or user input mechanism, which includes a camera device control button mounted on the frame 1132 for tactile or manual engagement by the user. The camera device control button provides a dual-modal or single-action mechanism because it can be set by the user between only two states: engaged and disengaged. In this example, the camera device control button is in the disengaged state by default, and can be pressed by the user to set it to the engaged state. When the pressed camera device control button is released, it automatically returns to the disengaged state.

[0189] In other examples, the single-action input mechanism can alternatively be provided by, for example, a touch-sensitive button including a capacitive sensor mounted adjacent to the surface of the frame 1132 to detect the presence of a user's finger, so as to set the touch-sensitive button to an engaged state when the user touches a corresponding point on the outer surface 1166 of the frame 1132. It should be understood that the camera device control button and the capacitive touch button described above are merely two examples of tactile input mechanisms for single-action control of the camera device 1169, and other examples may employ different single-action tactile control arrangements.

[0190] Computer 1161 is configured to perform the methods described herein, such as methods 800, 1000, and 1300. According to some examples, computer 1161 is an example of mobile device 602. In some examples, computer 1161 is coupled to one or more antennas for receiving signals from GNSS and a circuit system for processing the signals, wherein the antennas and circuit system are housed in glasses 1100. In some examples, computer 1161 is coupled to one or more wireless antennas and circuit systems for transmitting and receiving wireless signals, wherein the antennas and circuit systems are housed in glasses 1100. In some examples, glasses 1100 houses multiple sets of antennas and circuit systems. In some examples, the antennas and circuit systems are configured according to, for example, Bluetooth. TM Bluetooth Low Energy TM It operates using communication protocols such as IEEE 802 and IEEE 802.11az. In some examples, the PDR sensor is housed within the glasses 1100. For instance, the PDR sensor 610 is housed within the glasses 1100 and coupled to the computer 1161.

[0191] In some examples, application module 706 monitors the current location 708 and provides contextual notifications based on the current location 708. Contextual notifications may include indications of relevant landmarks, presentation of options for hearing about museum exhibits in front of the user on mobile device 602, and so on. In some examples, application module 706 may reside on host device 605, and host device 605 may monitor the current location 708 of mobile device 602 and provide notifications based on the current location 708. In some examples, host device 605 assumes that mobile device 602 is within a few meters or several meters of host device 605 and provides notifications to mobile device 602 based on the assumed location of host device 605.

[0192] In some examples, the landmark application will launch when the mobile device 602 is determined to be within a landmark's threshold. This landmark application could be an application module 706 based on a landmark such as the Eiffel Tower. In some examples, the threshold is 30 to 50 meters. In some examples, the landmark application will terminate or provide an option to terminate once the mobile device 602 moves away from the landmark.

[0193] Figure 12 The diagram illustrates location tracking 1200 according to some examples. The Y-coordinate 1202 is along a vertical axis from -10 meters to 40 meters, and the x-coordinate 1204 is along a horizontal axis from 0 meters to 70 meters. Each of the letters A through Z indicates the current heading and position 708 of the mobile device 602. According to some examples, the current position 708 is determined based on position data 618 from the PDR sensor 610. The position data 618 includes indications of changes in heading and changes in distance from previous distances. In some examples, the current position 708A through Z can be determined based on operations 1002 through 1010, where each determination at operation 1010 does not require sending a request 620 to a positioning system external to the PDR sensor 610. The mobile device 602 follows a path from A to Z.

[0194] Figure 13 A method 1300 for determining a device location using multi-source geospatial data is illustrated, based on some examples. Method 1300 begins at operation 1302, where new location data is accessed from one of multiple location sources. According to some examples, the new location data includes a new location and the accuracy of that new location. For example, a mobile device 602 may receive or determine new location data 612, 614, 616, 618.

[0195] Method 1300 continues at operation 1304 to determine the current location and the accuracy of the current location based on the new location and the new location. In some examples, the current location is also determined based on the previous current location and the accuracy of the previous current location. For example, method 800 can be used to determine the current location 708. In some examples, equation (1) is used to determine the current location 708.

[0196] Method 1300 continues at operation 1306, determining the change in location based on the difference between the current location and the previous current location. For example, as combined with Figure 8 As described, mobile device 602 can determine a change in location based on the difference between the updated current location and the previous current location.

[0197] Method 1300 continues at operation 1308, determining whether to use the optional location data field based on the change in location. For example, in some examples, operation 1308 includes one or more of the following: setting the current altitude value with the new altitude value in response to the new location data including the new altitude value; and setting the current altitude value to the old current altitude value based on the change in location being less than a first threshold and the old current altitude value not being an empty altitude value in response to the new location data not including the new altitude value, and setting the current altitude value with an empty altitude value based on the change in location not being less than the first threshold. The first threshold may be a distance such as 100 meters or other distances. Operations 814, 816, 818, and 820 perform operations to determine whether to use the optional fields of location data 612, 614, 616, and 618.

[0198] One or more operations in method 1300 may be optional. For example, operation 1308 may be optional. Method 1300 may include one or more additional operations. The operations of method 1300 may be performed in different orders.

[0199] Machine architecture

[0200] Figure 14This is a schematic representation of machine 1400, within which instructions 1408 (e.g., software, program, application, app, or other executable code) can be executed to cause machine 1400 to perform any or more of the methods discussed herein. For example, instructions 1408 can cause machine 1400 to perform any or more of the methods described herein. Instructions 1408 transform a general, unprogrammed machine 1400 into a specific machine 1400 programmed to perform the described and illustrated functions in the described manner. Machine 1400 can operate as a standalone device or can be coupled (e.g., networked) to other machines. In a networked deployment, machine 1400 can operate as a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. Machine 1400 may include, but is not limited to: server computers, client computers, personal computers (PCs), tablet computers, laptop computers, netbooks, set-top boxes (STBs), personal digital assistants (PDAs), entertainment media systems, cellular phones, smartphones, mobile devices, wearable devices (e.g., smartwatches), smart home devices (e.g., smart appliances), other smart devices, web devices, network routers, network switches, network bridges, or any machine capable of sequentially or otherwise executing instructions 1408 specifying actions to be taken by machine 1400. Furthermore, although only a single machine 1400 is shown, the term "machine" should also be considered as a collection of machines that individually or jointly execute instructions 1408 to perform any or more of the methods discussed herein. For example, machine 1400 may include client device 102 or any of several server devices forming part of message transceiver server system 108. In some examples, machine 1400 may also include both client and server systems, wherein certain operations of a particular method or algorithm are performed on the server side and certain operations of a particular method or algorithm are performed on the client side.

[0201] Machine 1400 may include a processor 1402, a memory 1404, and input / output I / O units 1438 that can be configured to communicate with each other via a bus 1440. According to some examples, processor 1402 may be referred to as a computer processor. In examples, processor 1402 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, processor 1406 and processor 1402 that execute instructions 1408. The term "processor" is intended to include multi-core processors, which may include two or more independent processors (sometimes referred to as "cores") capable of executing instructions simultaneously. Although Figure 14 Multiple processors 1402 are shown, but machine 1400 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

[0202] Memory 1404 includes main memory 1412, static memory 1414, and memory cells 1416, accessible by processor 1402 via bus 1440. Main memory 1404, static memory 1414, and memory cells 1416 store instructions 1408 that implement any one or more of the methods or functions described herein. Instructions 1408 may also reside wholly or partially in main memory 1412, in static memory 1414, in machine-readable medium 1418 within memory cells 1416, within at least one processor in processor 1402 (e.g., within the processor's cache memory), or any suitable combination thereof during execution by machine 1400.

[0203] I / O component 1438 may include a wide variety of components for receiving input, providing output, generating output, transmitting information, exchanging information, capturing measurement results, etc. The specific I / O component 1438 included in a particular machine will depend on the type of machine. For example, portable machines such as mobile phones may include touch input devices or other such input mechanisms, while headless server machines are unlikely to include such touch input devices. It should be understood that I / O component 1438 may include... Figure 14Many other components are not shown. In various examples, I / O component 1438 may include user output component 1424 and user input component 1426. User output component 1424 may include visual components (e.g., a display such as a plasma display panel (PDP), a light-emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), acoustic components (e.g., a speaker), haptic components (e.g., a vibration motor, a resistance mechanism), other signal generators, etc. User input component 1426 may include alphanumeric input components (e.g., a keyboard, a touchscreen configured to receive alphanumeric input, an optical keyboard, or other alphanumeric input components), point-based input components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or other pointing instruments), haptic input components (e.g., a physical button, a touchscreen or other haptic input component that provides positioning and force for touch or touch gestures), audio input components (e.g., a microphone), etc.

[0204] In other examples, I / O component 1438 may include biometric component 1428, motion component 1430, environmental component 1432 or positioning component 1434, and various other components. For example, biometric component 1428 includes components for detecting expressions (e.g., hand expressions, facial expressions, voice expressions, body posture, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweating, or brain waves), and identifying people (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or EEG-based recognition). Motion component 1430 includes accelerometer components (e.g., accelerometers), gravity sensor components, and rotation sensor components (e.g., gyroscopes).

[0205] The environmental component 1432 includes, for example, one or more camera devices (with still image / photograph and video capabilities), lighting sensor components (e.g., photometers), temperature sensor components (e.g., one or more thermometers that detect ambient temperature), humidity sensor components, pressure sensor components (e.g., barometers), hearing sensor components (e.g., one or more microphones that detect background noise), proximity sensor components (e.g., infrared sensors that detect nearby objects), gas sensors (e.g., gas detection sensors that detect the concentration of hazardous gases for safety purposes or measure pollutants in the atmosphere), or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment.

[0206] Regarding the camera device, client device 102 may have a camera device system including, for example, a front-facing camera on the front surface of client device 102 and a rear-facing camera on the rear surface of client device 102. The front-facing camera may be used, for example, to capture still images and videos (e.g., "selfies") of the user of client device 102, and then the still images and videos may be enhanced using the aforementioned enhancement data (e.g., filters). The rear-facing camera may be used, for example, to capture still images and videos in a more conventional camera device mode, and similarly enhance these images using enhancement data. In addition to the front-facing and rear-facing cameras, client device 102 may also include a 360° camera for capturing 360° photos and videos.

[0207] Furthermore, the camera system of the client device 102 may include dual rear cameras (e.g., a main camera and a depth-sensing camera), or even include triple, quadruple, or quintuple rear camera configurations on the front and rear sides of the client device 102. For example, these multiple camera systems may include wide-angle cameras, ultra-wide-angle cameras, telephoto cameras, macro cameras, and depth sensors.

[0208] The position component 1434 includes a position sensor component (e.g., a GPS receiver component), an altitude sensor component (e.g., an altimeter or barometer that detects air pressure and from which altitude can be obtained), an orientation sensor component (e.g., a magnetometer), etc.

[0209] A wide variety of technologies can be used to implement communication. I / O component 1438 also includes communication component 1436, which is operable to couple machine 1400 to network 1420 or device 1422 via appropriate coupling or connection. For example, communication component 1436 may include a network interface component or other suitable device that interfaces with network 1420. In further examples, communication component 1436 may include wired communication components, wireless communication components, cellular communication components, near field communication (NFC) components, etc. Components (e.g.) (low power consumption) The device 1422 may be any peripheral device (e.g., a peripheral device coupled via USB) that provides communication via other modes.

[0210] Furthermore, the communication component 1436 may detect identifiers or include components operable to detect identifiers. For example, the communication component 1436 may include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting one-dimensional barcodes such as Universal Product Code (UPC) barcodes, multi-dimensional barcodes such as Quick Response (QR) codes, Aztec codes, Data Matrix, Dataglyph, MaxiCode, PDF417, UltraCode, UCC RSS-2D barcodes, and other optical codes) or an acoustic detection component (e.g., a microphone for identifying audio signals from the tag). Additionally, various information can be obtained via the communication component 1436, such as location via Internet Protocol (IP) geolocation, etc. Location methods include signal triangulation and NFC beacon signals that can indicate a specific location.

[0211] Various memories (e.g., main memory 1412, static memory 1414, and the memory of processor 1402) and storage units 1416 may store one or more sets of instructions and data structures (e.g., software) implemented or used by any one or more of the methods or functions described herein. These instructions (e.g., instruction 1408) cause various operations to implement the disclosed examples when executed by processor 1402.

[0212] Instructions 1408 can be sent or received over network 1420 via a transmission medium using a network interface device (e.g., a network interface component included in communication component 1436) and using any of several known transmission protocols (e.g., Hypertext Transfer Protocol (HTTP)). Similarly, instructions 1408 can be sent or received via a transmission medium coupled to device 1422 (e.g., peer-to-peer coupling).

[0213] Software Architecture

[0214] Figure 15This is a block diagram 1500 illustrating a software architecture 1504 that can be installed on any one or more of the devices described herein. The software architecture 1504 is supported by hardware such as a machine 1502 including a processor 1520, memory 1526, and I / O components 1538. In this example, the software architecture 1504 can be conceptualized as a stack of layers, where each layer provides specific functionality. The software architecture 1504 includes layers such as an operating system 1512, libraries 1510, frameworks 1508, and applications 1506. Operationally, application 1506 activates API calls 1550 via the software stack and receives messages 1552 in response to API calls 1550.

[0215] Operating system 1512 manages hardware resources and provides public services. Operating system 1512 includes, for example, a kernel 1514, services 1516, and drivers 1522. Kernel 1514 acts as an abstraction layer between the hardware layer and other software layers. For example, kernel 1514 provides functions such as memory management, processor management (e.g., scheduling), component management, networking, and security settings. Services 1516 can provide other public services to other software layers. Driver 1522 is responsible for controlling or interfacing with the underlying hardware. For example, driver 1522 may include a display driver, a camera driver, etc. or Low-power drives, flash drives, serial communication drives (e.g., USB drives), Drivers, audio drivers, power management drivers, etc.

[0216] Library 1510 provides common low-level infrastructure used by application 1506. Library 1510 may include system library 1518 (e.g., the C standard library), which provides functions such as memory allocation, string manipulation, and mathematical functions. Additionally, library 1510 may include API library 1524, such as media libraries (e.g., libraries for supporting the rendering and manipulation of various media formats, such as Moving Picture Experts Group-4 (MPEG4), Advanced Video Coding (H.264 or AVC), Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio codecs, Joint Picture Experts Group (JPEG or JPG) or Portable Web Graphics (PNG)), graphics libraries (e.g., OpenGL frameworks for rendering graphic content on a display in two-dimensional (2D) and three-dimensional (3D) formats), database libraries (e.g., SQLite, which provides various relational database functions), web libraries (e.g., WebKit, which provides web browsing capabilities), and so on. Library 1510 may also include various other libraries 1528 to provide application programming interfaces (APIs) to application 1506.

[0217] Framework 1508 provides a common high-level infrastructure used by Application 1506. For example, Framework 1508 provides various graphical user interface (GUI) functions, advanced resource management, and advanced location services. Framework 1508 can provide a wide range of other APIs that can be used by Application 1506, some of which may be specific to a particular operating system or platform.

[0218] In the example, application 1506 may include a home application 1536, a contacts application 1530, a browser application 1532, a book reader application 1534, a geolocation application 1541, a location application 1542, a media application 1544, a messaging application 1546, a game application 1548, and a variety of other applications such as third-party applications 1540. The geolocation application 1541 can perform [further actions]. Figures 6 to 13 And the operations disclosed herein. Application 1506 is a program that performs the functions defined in the program. One or more applications 1506 can be created using various programming languages, such as object-oriented programming languages ​​(e.g., Objective-C, Java, or C++) or procedural programming languages ​​(e.g., C or assembly language). In a particular example, a third-party application 1540 (e.g., used by an entity other than the vendor of a particular platform) uses Android. TM or iOS TM Applications developed using a Software Development Kit (SDK) can be used on platforms such as iOS. TM ANDROID TM, Mobile software running on the phone's mobile operating system or another mobile operating system. In this example, a third-party application 1540 can activate API calls 1550 provided by the operating system 1512 to facilitate the functions described herein.

[0219] Processing components

[0220] Now turn to Figure 16 The diagram illustrates a processing environment 1600, which includes processors 1602, 1606, and 1608 (e.g., a GPU, CPU, or a combination thereof). Processor 1602 is shown coupled to a power supply 1604 and includes modules (permanently configured or temporarily instantiated), namely a wireless component 1610, a fusion component 1612, and a scheduler component 1614. (See also...) Figure 7 The wireless component 1610 is operable with, for example, Figure 7 Other wireless device interfaces are shown as wireless component module 718; fusion component 1612 will... Figure 6 The two location data points 612, 614, 616, and 618 are combined or merged together to generate Figure 7 The current position 708 is determined, and one or more operations of methods 800 and 1300 are performed; and the scheduler component 1614 is operatively configured to update the scheduler module 712 and the operations of methods 1000 and 1300. As shown, processor 1602 is communicatively coupled to both processor 1606 and processor 1608.

[0221] Glossary

[0222] "Carrier signal" refers to any intangible medium capable of storing, encoding, or carrying instructions to be executed by a machine, and includes digital or analog communication signals or other intangible media to facilitate the communication of such instructions. Instructions can be sent or received over a network using a transmission medium via a network interface device.

[0223] "Client device" refers to any machine that interfaces with a communication network to obtain resources from one or more server systems or other client devices. Client devices can be, but are not limited to, mobile phones, desktop computers, laptop computers, portable digital assistants (PDAs), smartphones, tablet computers, ultrabooks, netbooks, laptop computers, multiprocessor systems, microprocessor-based or programmable consumer electronics, game consoles, set-top boxes, or any other communication device that a user can use to access the network.

[0224] "Communication network" refers to one or more parts of a network, which can be an ad hoc network, intranet, extranet, virtual private network (VPN), local area network (LAN), wireless LAN (WLAN), wide area network (WAN), wireless WAN (WWAN), metropolitan area network (MAN), the Internet, a part of the Internet, a part of the Public Switched Telephone Network (PSTN), a POTS (Plain Old-Style Telephone Service) network, a cellular telephone network, a wireless network, etc. A network, other types of networks, or a combination of two or more such networks. For example, a network or part of a network may include a wireless network or a cellular network, and the coupling may be a Code Division Multiple Access (CDMA) connection, a Global System for Mobile Communications (GSM) connection, or other types of cellular or wireless coupling. In this example, the coupling can implement any data transmission technology of various types, such as Single Carrier Radio Transmission (1xRTT), Evolved Data Optimization (EVDO), General Packet Radio Service (GPRS), Enhanced Data Rate Evolution of GSM (EDGE), the 3rd Generation Partnership Project (3GPP) including 3G, fourth-generation wireless (4G) networks, Universal Mobile Telecommunications System (UMTS), High-Speed ​​Packet Access (HSPA), Global Microwave Access Interoperability (WiMAX), Long Term Evolution (LTE) standards, other data transmission technologies defined by various standards setting organizations, other long-distance protocols, or other data transmission technologies.

[0225] A “component” refers to a device, physical entity, or logic having boundaries defined by functional or subroutine calls, branch points, APIs, or other technologies that provide partitioning or modularity for a particular processing or control function. Components can be combined with other components via their interfaces to perform machine processing. A component can be an encapsulated functional hardware unit designed for use with other components, and part of a program that typically performs a specific function related to that function. Components can constitute software components (e.g., code embodied on a machine-readable medium) or hardware components. A “hardware component” is a tangible unit capable of performing certain operations and can be configured or arranged in some physical manner. In various examples, one or more hardware components (e.g., standalone computer systems, client computer systems, or server computer systems) or one or more hardware components (e.g., processors or processor groups) of a computer system can be configured by software (e.g., an application or application portion) to operate to perform certain operations described herein. Hardware components can also be implemented mechanically, electronically, or in any suitable combination thereof. For example, a hardware component can include a dedicated circuit system or logic permanently configured to perform certain operations. Hardware components can be dedicated processors, such as field-programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs). Hardware components can also include programmable logic or circuit systems temporarily configured by software to perform certain operations. For example, a hardware component may include software executed by a general-purpose processor or other programmable processor. Once configured by such software, the hardware component becomes a specific machine (or a specific part of a machine) uniquely tailored to perform the configured function and is no longer a general-purpose processor. It will be appreciated that a decision can be made, for cost and time considerations, whether to implement a hardware component mechanically in a dedicated and permanently configured circuit system or in a temporarily configured (e.g., software-configured) circuit system. Therefore, the phrase “hardware component” (or “hardware-implemented component”) should be understood to include tangible entities, i.e., entities physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a certain way or perform certain operations described herein. Consider the example of a hardware component being temporarily configured (e.g., programmed), without needing to configure or instantiate each hardware component at any given time. For example, in cases where the hardware components include a general-purpose processor that is configured as a dedicated processor via software, this general-purpose processor can be configured as its own distinct dedicated processor (e.g., including different hardware components) at different times. The software accordingly configures one or more specific processors to constitute a particular hardware component at one moment and different hardware components at different times. Hardware components can provide information to and receive information from other hardware components. Therefore, the described hardware components can be considered communicatively coupled.In the presence of multiple hardware components, communication can be achieved through signal transmission between or among the two or more hardware components (e.g., via appropriate circuitry and buses). In examples where multiple hardware components are configured or instantiated at different times, such communication between hardware components can be achieved, for example, by storing information in a memory structure accessible to the multiple hardware components and retrieving information from the memory structure. For example, a hardware component can perform an operation and store the output of that operation in a memory device communicatively coupled to it. Another hardware component can then access the memory device at a subsequent time to retrieve and process the stored output. Hardware components can also initiate communication with input or output devices and can operate on resources (e.g., collections of information). The various operations of the example methods described herein can be performed, at least in part, by one or more processors configured, either temporarily (e.g., by software) or permanently, to perform the relevant operations. Whether temporarily or permanently configured, such processors can constitute processor-implemented components that operate to perform one or more operations or functions described herein. As used herein, "processor-implemented component" refers to a hardware component implemented using one or more processors. Similarly, the methods described herein can be implemented at least in part by processors, where a particular processor or one or more processors are examples of hardware. For example, at least some operations of the methods can be performed by one or more processors 1402 or processor-implemented components. Furthermore, one or more processors can also operate to support the execution of related operations in a “cloud computing” environment or as “Software as a Service” (SaaS) operations. For example, at least some operations can be performed by a group of computers (as an example of machines including processors), where these operations are accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., APIs). The execution of certain operations can be distributed among processors, residing not only within a single machine but also deployed across multiple machines. In some examples, the processor or processor-implemented component can reside in a single geographic location (e.g., within a home environment, office environment, or server cluster). In other examples, the processor or processor-implemented component can be distributed across multiple geographic locations.

[0226] "Computer-readable storage medium" refers to both machine storage media and transmission media. Therefore, these terms include both storage devices / media and carrier / modulated data signals. The terms "machine-readable medium," "computer-readable medium," and "device-readable medium" refer to the same thing and can be used interchangeably in this disclosure. The plural form "computer-readable medium" can be referred to as "a plurality of computer-readable media."

[0227] A "brief message" is a message that is accessible for a limited period of time. Brief messages can be text, images, videos, etc. The access time for a brief message can be set by the message sender. Alternatively, the access time can be a default setting or a setting specified by the recipient. Regardless of the setting method, the message is temporary.

[0228] "Machine storage medium" refers to one or more storage devices and media (e.g., centralized or distributed databases, and associated caches and servers) that store executable instructions, routines, and data. Therefore, this term should be considered to include, but is not limited to, solid-state memory and optical and magnetic media, including memory internal or external to the processor. Specific examples of machine storage media, computer storage media, and device storage media include: non-volatile memory, including, for example, semiconductor memory devices such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), FPGAs, and flash memory devices; disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The terms "machine storage medium," "device storage medium," and "computer storage medium" mean the same thing and may be used interchangeably in this disclosure. The terms "machine storage medium," "computer storage medium," and "device storage medium" expressly exclude carrier waves, modulated data signals, and other such media, at least some of which are covered by the term "signal medium."

[0229] "Non-transitory computer-readable storage medium" refers to a tangible medium capable of storing, encoding, or carrying instructions that can be executed by a machine.

[0230] "Signal medium" means any intangible medium capable of storing, encoding, or carrying instructions executable by a machine, and includes digital or analog communication signals or other intangible media to facilitate the communication of software or data. The term "signal medium" should be considered to include any form of modulated data signal, carrier wave, etc. The term "modulated data signal" means a signal whose characteristics are set or altered in a manner that encodes information in the signal. The terms "transmission medium" and "signal medium" mean the same thing and may be used interchangeably in this disclosure.

Claims

1. A method executed on a device, the method comprising: New location data is accessed from a location source among multiple location sources, the new location data including the new location and the accuracy of the new location; The current location and its accuracy are determined based on the new location, the accuracy of the new location, the previous current location, and the accuracy of the previous current location; The change in location is determined based on the difference between the current location and the previous current location; The new location data does not include the new altitude value; Based on the fact that the change in location has not exceeded a first threshold and the previous current altitude value is not an empty altitude value, the current altitude value is set to the previous current altitude value. as well as If the change in location exceeds the first threshold, the current altitude value is set using the empty altitude value.

2. The method according to claim 1, further comprising: In response to the new location data including the new altitude value, the current altitude value is set using the new altitude value.

3. The method according to claim 1, wherein, The current location includes: x value, y value, and z value; the previous current location includes: x1 value, y1 value, and z1 value; and the new location includes: x2 value, y2 value, and z2 value. Determining the current location further includes: The x value, y value, and z value are determined to be equal to: ((x1 value, y1 value, and z1 value) multiplied by (1 / accuracy of the previous current location) + (x2 value, y2 value, and z2 value) multiplied by (1 / accuracy of the new location)) / (change in location value).

4. The method according to claim 1, wherein, The current location includes: x value, y value, and z value; the previous current location includes: x1 value, y1 value, and z1 value; and the new location includes: x2 value, y2 value, and z2 value. Determining the current location further includes: The x value, y value, and z value are determined based on: the x1 value, y1 value, and z1 value; the x2 value, y2 value, and z2 value; and the accuracy of the previous current location, the accuracy of the new location, and the change in the location value.

5. The method according to claim 1, wherein, The current location indicates the position of the device regardless of its orientation.

6. The method according to any one of claims 1 to 5, further comprising: The speed is determined based on the new location, the previous current location, and the amount of time between the new timestamp and the previous current timestamp; as well as In response to the speed exceeding a threshold for the traffic mode for the device, the new location data is discarded, and other new location data is accessed as the new location data.

7. The method according to claim 6, wherein, The mode of transportation includes at least one of the following: walking, running, driving, cycling, flying, taking a train, and taking a bus.

8. The method according to any one of claims 1 to 5, further comprising: In response to the new location data including a new location value, the current location value is set to the new location value; as well as In response to the new location data not including the new location value, based on the change in location being less than a second threshold and the previous current location value not being an empty current location value, the current location value is set to the previous current location value, and based on the change in location not being less than the second threshold, the current location value is set to the empty current location value.

9. The method according to any one of claims 1 to 5, wherein, The device is an augmented reality (AR) mobile device, and the method further includes: In response to the activation of the device, an initial location is determined from one of the plurality of location sources as the previous current location; and Determine the change in orientation of the AR mobile device.

10. The method according to any one of claims 1 to 5, further comprising: The location source is selected from the plurality of location sources based on one or more first conditions; as well as The scheduler generates a request to generate location data for the selected location source.

11. The method according to claim 10, wherein, The request is a first request, and the method further includes: A second request for access to the location data from the device's application.

12. The method according to claim 10, wherein, The multiple location sources include: Pedestrian Dead Retrieval (PDR) system, wireless signal location system, location from paired host equipment system, and Global Navigation Satellite System (GNSS).

13. The method according to claim 10, wherein, The selection of the location source and the frequency of scheduling requests to generate location data for the selected location source are based on the speed of the device or the wearer of the device.

14. The method of claim 10, wherein, The first condition includes one or more of the following: the device's display status is on or off, the device's WiFi status, the pairing status with the paired host device system, the device's load, the device's indoor or outdoor status, the device's connection status with the Global Navigation Satellite System (GNSS), power availability, the accuracy requirements of the location data, the device's speed, the device's position change independent of changes in the device's orientation, and the dimension of the location data generated by the location source.

15. The method of claim 10, further comprising: In response to the fulfillment of one or more second conditions of the request to generate the location data for the selected location source, the location data is requested from the selected location source, the location data is received from the selected location source, and the received location data is set as the new location data.

16. The method according to claim 15, wherein, The one or more second conditions include one or more of the following: whether the device's display is on or off, the device's WiFi status, the pairing status with the paired host device system, the device's load, whether the device is indoors or outdoors, the device's connection status with the Global Navigation Satellite System (GNSS), the device's power availability, the accuracy requirements of the location data, the device's speed, the change in the device's location independent of changes in the device's orientation, and the dimension of the location data generated by the location source.

17. A system for determining location, comprising: One or more computer processors; A computer-readable medium storing one or more instructions that, when executed by the one or more computer processors, cause the system to perform operations including: New location data is accessed from a location source among multiple location sources, the new location data including the new location and the accuracy of the new location; The current location and its accuracy are determined based on the new location, the accuracy of the new location, the previous current location, and the accuracy of the previous current location; The change in location is determined based on the difference between the current location and the previous current location; In response to the new location data including a new altitude value, the current altitude value is set using the new altitude value; and In response to the new location data not including the new altitude value, Based on the fact that the change in location is less than a first threshold and the previous current altitude value is not an empty altitude value, the current altitude value is set to the previous current altitude value, and The current altitude value is set based on the change in location being no less than the first threshold.

18. The system according to claim 17, wherein, The current location includes: x value, y value, and z value; the previous current location includes: x1 value, y1 value, and z1 value; and the new location includes: x2 value, y2 value, and z2 value. Determining the current location further includes: The x value, y value, and z value are determined to be equal to: ((x1 value, y1 value, and z1 value) × (1 / accuracy of the previous current location) + (x2 value, y2 value, and z2 value) × (1 / accuracy of the new location)) / (change in location value).

19. A non-transitory computer-readable storage medium including instructions that, when processed by a computer, configure the computer to perform operations including: New location data is accessed from a location source among multiple location sources, the new location data including the new location and the accuracy of the new location; The current location and its accuracy are determined based on the new location, the accuracy of the new location, the previous current location, and the accuracy of the previous current location; The change in location is determined based on the difference between the current location and the previous current location; The new location data does not include the new altitude value; Based on the fact that the change in location is less than a first threshold and the previous current altitude value is not an empty altitude value, the current altitude value is set to the previous current altitude value. as well as If the change in location exceeds the first threshold, the current altitude value is set using the empty altitude value.

20. The non-transitory computer-readable storage medium according to claim 19, wherein, The current location includes: x value, y value, and z value; the previous current location includes: x1 value, y1 value, and z1 value; and the new location includes: x2 value, y2 value, and z2 value. Determining the current location further includes: In response to the new location data including the new altitude value, the current altitude value is set using the new altitude value; and The x value, y value, and z value are determined to be equal to: ((x1 value, y1 value, and z1 value) × (1 / accuracy of the previous current location) + (x2 value, y2 value, and z2 value) × (1 / accuracy of the new location)) / (change in location value).

Images