Augmented reality object rendering

Abstract

Various implementations provide a registration table for augmented reality (AR) objects that can provide AR objects to client devices to support various software applications or hardware applications. For example, some implementations provide an AR object registration table that facilitates or enables registration of one or more AR objects associated with one or more locations on a planet.

Description

[0001] Claiming priority

[0002] This patent application claims the benefit of priority to U.S. Application Serial No. 16 / 729,775, filed December 30, 2019, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The implementations described herein relate to augmented reality, and more specifically, but not in a limiting sense, to systems, methods, apparatus, and instructions for registering augmented reality objects that can support mixed reality. Background Technology

[0004] Currently, geospatial databases can be used to provide mobile devices with three-dimensional (3D) geospatial maps of the world. Typically, mobile devices such as smartphones use the Global Positioning System (GPS) to orthogonally associate the device with a coordinate location on Earth, and this coordinate location can be used to query a geospatial database to retrieve 3D map data. For example, given the center point of the coordinate location relative to the mobile device, a geospatial database can be queried to retrieve all data for 3D map objects within a specific radius of the center point, or to retrieve data for the 3D map object closest to the center point. Attached Figure Description

[0005] The various accompanying drawings illustrate only some embodiments of this disclosure and should not be construed as limiting its scope. The drawings are not necessarily drawn to scale. For ease of identification of any particular element or action discussed, one or more of the most significant numerals in the reference numerals refer to the reference numeral that first introduces that element, and the same reference numerals may describe similar components in different views.

[0006] Figure 1 This is a block diagram illustrating an example system for exchanging data (e.g., in relation to AR objects) over a network according to some implementations. The example system may include an augmented reality (AR) object system.

[0007] Figure 2 This is a block diagram illustrating an example logical architecture of an AR object system according to some implementations.

[0008] Figure 3 This is a block diagram illustrating an example of an AR object interactive session service according to some implementations.

[0009] Figure 4 This is a flowchart illustrating an example of session processing performed by an AR object interactive session service according to some implementations.

[0010] Figure 5This is a flowchart illustrating an example of using one or more sorters to provide one or more AR objects to a client device according to some implementations.

[0011] Figure 6 This is a block diagram illustrating an example implementation of an AR object system according to some implementation methods.

[0012] Figures 7 to 13 This is a flowchart illustrating a method related to an AR object registration form according to some implementations.

[0013] Figure 14 This is a block diagram illustrating a representative software architecture that can be used in conjunction with the various hardware architectures described herein.

[0014] Figure 15 This is a block diagram illustrating a machine component, according to some embodiments, capable of reading instructions from a machine-readable medium (e.g., a machine-readable storage medium) and performing any one or more of the methods discussed herein. Detailed Implementation

[0015] Various implementations provide systems, methods, apparatus, and instructions for registering augmented reality objects, which can provide augmented reality objects to client devices to support various software or hardware applications (e.g., mixed reality software applications). For example, some implementations provide an augmented reality (AR) object register that facilitates or enables the registration of one or more AR objects associated with one or more locations on a planet (e.g., worldwide). For example, an AR object register can enable the association between one or more AR objects and one or more locations or physical objects on the planet Earth. In doing so, such an AR object register can allow users to explore AR objects anywhere on Earth using their client devices. Furthermore, the AR object register described herein can also support or enable, for example, the implementation of a space-based (e.g., augmented reality-based) World Wide Web.

[0016] The architecture of some implementations described herein allows scaling to serve AR object registrations related to locations on Earth, and allows scaling to provide or support interactive sessions that enable multiple users (e.g., a large number of users) around the world to interact with registered AR objects. In some implementations, the AR registration table can associate (e.g., unite) topological map data (of the Earth) with AR object data, bringing real-world information into the virtual model, thus enabling scalability of the AR registration table. Furthermore, some implementations implement one or more sorters or sorting mechanisms (e.g., sorting algorithms) to determine (e.g., by filtering or sorting) which AR objects are provided to client devices (e.g., in response to requests / queries for AR objects from client devices). In this way, such implementations can influence which AR objects are displayed or shown by the client device at or around its current coordinate set on a map (e.g., a geospatial map).

[0017] Unlike conventional technologies (e.g., traditional geospatial databases), AR registries with various implementations can better support user interaction with registered AR objects. Furthermore, unlike conventional technologies, AR registries with various implementations do not rely on strict (e.g., tight) geofencing to deliver AR objects to client devices.

[0018] As used herein, AR objects can include virtual objects that can be presented in a view of a real-world environment generated by a client device (e.g., a view presented on a display of a mobile client device), wherein the virtual objects can interact with or enhance real-world physical objects of the real-world environment presented in the view. For example, AR objects can be combined with on-site (e.g., real-time or near-real-time) camera feeds such that when the AR object is presented, the AR object appears to be located in the on-site three-dimensional environment (e.g., the AR object appears to occupy a consistent three-dimensional volume and dynamically changes its aspect in response to camera movement in a manner similar to if the AR object were a real-world physical object). Registered AR objects can include AR objects registered through the embodiments described herein, thereby associating AR objects with a set of coordinates via an AR object registration table. The level of interaction (e.g., user interaction) available for AR objects registered through the embodiments can vary. For example, the AR object can be static and have no level of interaction with the user or the real-world environment. Registered AR objects (e.g., virtual balls) may have one or more available interactions (e.g., rotating, bouncing, throwing, etc.), wherein any changes to the state of the AR object (through those available interactions) are confined (e.g., constrained or restricted) to the user at the client device (e.g., changes to the state of the AR object do not propagate to another user at another client device), and any changes to the state of the AR object do not alter the current initial state of the AR object as stored in the AR object registration table. Registered AR objects (e.g., virtual graffiti) may have one or more available interactions (e.g., drawing, generating, or applying virtual graffiti), wherein any changes to the state of the AR object (through those available interactions) are propagated to another user at another client device (e.g., rendered in a view displayed by another client device) without interaction with other users (i.e., no interactive session is required). Furthermore, registered AR objects may allow two or more users to interact with the registered AR object simultaneously (e.g., in real time) during an interactive session (e.g., rotating, bouncing, or throwing the virtual ball). For example, a first user can throw a virtual ball between one or more other users within the same interactive session, where data is transmitted between users' client devices via the interactive session. Depending on the registered AR object, at the end of the interactive session, the final state of the registered AR object (such as if it was changed by the user's interactions during the interactive session) may or may not be saved to the AR object registry, thus updating the initial state of the AR object for subsequent single-user or multi-user interactive sessions.

[0019] In some implementations, registration of AR objects or sorters can be ephemeral (e.g., accessible for a limited time after the first visit). This ephemeral nature of registration may require users to periodically (e.g., every 24 hours) re-register AR objects or sorters, which can deter or prevent registration abuse (e.g., spam).

[0020] In some implementations, user interactions with a given AR object can be defined by a set of rules (e.g., interaction rules) associated with the AR object. For example, rules for the AR object can determine the availability of interactions with the AR object (e.g., the ability to throw or bounce a virtual ball), or can impose constraints on interactions with the AR object (e.g., interactions with the virtual ball are limited to the client, or the virtual ball can only be thrown this far).

[0021] The following description includes systems, methods, techniques, instruction sequences, and computer program products embodying illustrative embodiments of the present disclosure. In this description, numerous specific details are set forth for illustrative purposes to provide an understanding of various embodiments of the subject matter of the invention. However, it will be apparent to those skilled in the art that embodiments of the subject matter of the invention can be practiced without these specific details. Generally, well-known examples of instructions, protocols, structures, and techniques are not necessarily shown in detail.

[0022] Reference will now be made to embodiments of this disclosure, examples of which are illustrated in the accompanying drawings. However, this disclosure may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein.

[0023] Figure 1 This is a block diagram illustrating an example system 100 for exchanging data (e.g., related to AR objects) via network 106 according to some embodiments. Example system 100 may include an augmented reality (AR) object system. System 100 includes a plurality of client devices 102, each of which hosts a plurality of applications, including client application 104. Each client application 104 is communicatively coupled to other instances of server system 108 via network 106 (e.g., the Internet).

[0024] Therefore, each client application 104 can communicate and exchange data with another client application 104 and with the server system 108 via the network 106. The data exchanged between client applications 104 and between client application 104 and server system 108 includes functions (e.g., commands to call functions) and payload data (e.g., AR objects, text, audio, video, or other multimedia data).

[0025] In some implementations, a particular client application 104 provides one or more augmented reality / mixed reality features for its respective client device 102. For example, the particular client application 104 may represent an augmented reality (AR) client software application or a messaging software application that includes augmented reality / mixed reality features. The particular client application 104 may acquire one or more AR objects (e.g., from an augmented reality (AR) object system having an AR object system 116, hereinafter referred to as AR object system 116) to generate a mixed reality environment (e.g., a real-world environment based on client device 102) that includes one or more AR objects. For example, the particular client application 104 may enable client device 102, such as a smartphone, to capture image frames of a real-world environment (e.g., using a smartphone camera) and generate a view (e.g., on a smartphone display) that presents a real-world environment with one or more AR objects associated with the real-world environment (e.g., augmented by one or more AR objects associated with the real-world environment). Specifically, a particular client application 104 can obtain AR objects from an AR registry (e.g., implemented by the AR object system 116) by requesting or querying one or more AR objects from the AR object system 116, for example, using information associated with the client device 102 (e.g., information about the user of the client device 102, the current coordinate set of the client device 102 (e.g., GPS coordinates), or a specified radius around the client device 102). When one or more AR objects (e.g., the AR object system 116) are obtained, the particular client application 104 can receive data for those AR objects. The data for those AR objects may include, for example: model data for rendering and displaying one or more three-dimensional models (e.g., 3D graphical content) of the obtained AR objects on the client device 102; rule data describing one or more rules that determine the user's interaction with the obtained AR objects through the particular client application 104; or state data describing the initial state of the obtained AR objects (e.g., the initial state of the obtained AR objects that will be rendered by the particular client application 104 on the client device 102).

[0026] Regarding the use of the acquired AR object, a specific client application 104 can display the acquired AR object on the display of client device 102 by determining the positioning of the AR object on the display relative to the real-world environment. The specific client application 104 can do this by performing a process of generating (or constructing) a virtual camera by combining data from various sensors (e.g., image sensors, inertial measurement units (IMUs), and GPS sensors) from client device 102, and then using the virtual camera to position the acquired AR object on the display of client device 102. The specific client application 104 can generate the virtual camera, for example, using a Simultaneous Localization and Mapping (SLAM) or Visual Odometry (VIO) system or method. When the specific client application 104 displays the AR object, a 3D model of the AR object can be rendered and displayed as an overlay on the real-world environment presented by client device 102.

[0027] In some implementations, a particular client application 104 enables a user to register one or more AR objects using an AR registration form (e.g., implemented via AR object system 116) associated with a set of coordinates on a map (e.g., a geospatial map).

[0028] Server system 108 provides server-side functionality to a specific client application 104 via network 106. While some functions of system 100 are described herein as being performed by client application 104 or by server system 108, it will be understood that the location of certain functions within client application 104 and server system 108 is a design choice. For example, it may be technically preferred that certain technologies and functions are initially deployed within server system 108, but later migrated to client application 104 on client device 102 with sufficient processing power.

[0029] Server system 108 supports various services and operations provided to client application 104. Such operations include sending data to client application 104; receiving data from client application 104; and processing data generated by client application 104. As an example, this data may include message content, AR object-related information (e.g., model data, orientation, interaction rules or logic, status information, interactions, session information, etc.), client device information, geolocation information, media annotations and overlays, message content persistence conditions, social network information, and on-site event information. Data exchange within system 100 can be invoked and controlled via functions available through the user interface (UI) of client application 104.

[0030] Now, specifically to server system 108, application programming interface (API) server 110 is coupled to application server 112 and provides a programming interface to application server 112. Application server 112 is communicatively coupled to database server 118, which facilitates access to database 120, where data associated with the operations performed by application server 112 is stored.

[0031] Specifically, the Application Programming Interface (API) server 110 handles the receiving and sending of message data (e.g., commands and message payloads) between the client device 102 and the application server 112. Specifically, the API server 110 provides a set of interfaces (e.g., routines and protocols) that can be invoked or queried by the client application 104 to call functions of the application server 112. The API server 110 exposes various functions supported by the application server 112, including, for example: account registration; login functionality; sending AR object-related information (e.g., model data, orientation, interaction rules or logic, status information, interactions, session information, etc.) from the AR object system 116 to a specific client application 104 via the application server 112; sending AR object-related information (e.g., query or request information, user input information, status information, model data of new AR objects, etc.) from a specific client application 104 to the AR object system 116 via the application server 112; and ... the application server 112 to the AR object system 116 via the application server 112; and sending AR object-related information (e.g., query or request information, user input information, status information, model data of new AR objects, etc.) from the client application 104 to the AR object system 116 via the application server 112; and sending AR object-related information (e.g., query or request information, user input information, status information, model data of new AR objects, etc.) via the application server 112; and sending AR object-related information (e.g., query or request information, user input information, Server 112 sends messages from a specific client application 104 to another client application 104; sends media files (e.g., digital images or videos) from client application 104 to message transceiver server application 114, and makes them accessible to another client application 104; sets up a collection of media content items (e.g., stories); retrieves the friend list of the user of client device 102; retrieves such a collection; retrieves messages and content; adds and removes friends from the social graph; shows the location of friends in the social graph; and handles open application events (e.g., related to client application 104).

[0032] Application server 112 hosts multiple applications, systems, and subsystems, including messaging server application 114, AR object system 116, and social networking system 122. Messaging server application 114 implements various messaging techniques and functions, particularly those related to the aggregation and other processing of media content items (e.g., text and multimedia content items) included in messages received from multiple instances of client application 104. As described herein, media content items from multiple sources can be aggregated into a collection of media content items (e.g., a story or gallery), which can be automatically annotated by various implementations described herein. For example, the collection can be annotated by associating it with captions, geolocation, categories, events, highlighted media content items, etc. The collection of media content items is available to client application 104 accessed by messaging server application 114. Given the hardware requirements for such processing, additional processor- and memory-intensive processing of data can also be performed on the server side by messaging server application 114.

[0033] For a given set of media content, one or more annotations of the given set may represent characteristics of the given set, and those characteristics may include one or more graphic elements (e.g., emojis or emoticons) that, when automatically associated with a given set, can be used in the various implementations described herein. Accessing a given set of media content items may include: one or more annotations of the given set and one or more graphic elements associated with the given set through the various implementations described herein.

[0034] As shown, application server 112 also includes an AR object system 116 that implements one or more aspects of the various implementations described herein, such as AR registration forms and sorter-based AR queries. In this document regarding... Figure 2 More details about AR object system 116 are described.

[0035] Social networking system 122 supports various social networking functions and services, and makes these functions and services available to messaging server application 114 and AR object system 116. To this end, social networking system 122 maintains and accesses an entity graph within database 120. Examples of functions and services supported by social networking system 122 include: identifying specific users of system 100 and other users who are related to or "following" them, as well as identifying other entities and specific users of interest.

[0036] Application server 112 is communicatively coupled to database server 118, which facilitates access to database 120, which stores data associated with operations performed by message server application 114 or AR object system 116.

[0037] Figure 2 This is a block diagram illustrating an example logical architecture for an AR object system 116 according to some embodiments. Specifically, the AR object system 116 is shown as including a data layer 210 and an augmented reality (AR) object service 230, which support various features and functions of the AR object system 116. As shown, the data layer 210 includes a three-dimensional (3D) topology data layer 212, a logical topology data layer 214, a user data layer 216, and an augmented reality (AR) object model data layer 218. Also as shown, the AR object service 230 includes an augmented reality (AR) object interactive session service 232, an augmented reality (AR) object query service 234, and an augmented reality (AR) object registration service 236. For various embodiments, the components and the arrangement of the components of the AR object system 116 may be... Figure 2 The differences are shown. Any component of the AR object system 116 can be implemented using one or more processors (e.g., by configuring one or more computer processors to perform the functions described for that component), and therefore may include one or more processors. Furthermore, according to various embodiments, Figure 2 Any of the components shown can be implemented together or separately within a single machine, database, or device, or can be distributed across multiple machines, databases, or devices. For example, data layer 210 can be implemented by one or more databases (e.g., database 120), and AR object service 230 can be implemented by one or more servers (e.g., application server 112).

[0038] The 3D topology data layer 212 includes data describing the internal representation of the real-world environment. This data may include, but is not limited to, 3D modeling information of the real-world environment and information that associates the 3D modeling information with one or more coordinates (e.g., on a topology map). Queries to the 3D topology data layer 212 may include one or more coordinates on a map (e.g., a topology map) and radius values ​​around the points corresponding to those coordinates. Query results provided by the 3D topology data layer 212 may include one or more 3D model objects falling within a radius centered on one or more coordinates. The data in the 3D topology data layer 212 may originate from one or more data sources, including third-party vendors. Furthermore, the data in the 3D topology data layer 212 may be divided into two or more types, such as lower-resolution data (hereinafter referred to as world data) and higher-resolution data (hereinafter referred to as depth world data). The world data may represent the default ground truth data of the AR object system 116 (which can provide a rapid basis for AR object model placement). Compared to deep world data, world data may have lower accuracy (e.g., approximately 3m) and often lacks indoor data of real-world structures (e.g., buildings, etc.). Deep world data can represent 3D topology data with the highest accuracy (e.g., centimeter-level accuracy) within the AR object system 116 and can include indoor data of real-world structures.

[0039] The logical topology data layer 214 includes data related to logic (e.g., business or operational logic) that can be applied to the data provided by the 3D topology data layer 212. At least a portion of the data provided by the logical topology data layer 214 can be stored in a geospatial vector type format. The two types of data provided by the logical topology data layer 214 can include regional data and geolocation data.

[0040] According to some implementations, the region data (of the logical topology data layer 214) labels or identifies one or more regions in the real-world environment, and may also associate one or more attribute values ​​with those regions. For example, the region data may label / identify regions of the real-world environment based on one or more of the following: state boundaries, county boundaries, city limits, parcel features, or zoning areas. These labeled / identified regions may also be referred to as or considered as regions. In the region data, regions of the real-world environment may be defined by a non-scalar polygon data type. In some implementations, the region data facilitates the geographic partitioning of large real-world environments (e.g., the Earth), which may support the allocation and management of interactive sessions and session-related computing resources (e.g., session servers) by the AR object interactive session service 232 as described herein.

[0041] In some implementations, region data tags or identifies one or more permissions for a given region (e.g., as one or more attribute values ​​for a given region). Permissions for a given region (embodied in permission data) can enable AR object system 116 (e.g., its AR object registration service 236) to determine, for example, whether a given user can register (e.g., place) an AR object of their choice (e.g., a new AR object or an existing AR object) at a location corresponding to a set of coordinates on a map (e.g., a topology map). In this way, region data in some implementations can associate a specific real-world space with one or more permissions that can prevent abuse of AR object system 116. For example, permission data (provided by region data) can prevent a first user representing a first business (e.g., pizza shop #1) from registering an AR object (e.g., an AR object representing a coupon for the first business) at a location corresponding to a second business that is a competitor of the first business (e.g., pizza shop #2). On the other hand, the same permission data can allow a second user, identified as the owner of the second business, to register an AR object of their choice at a location corresponding to the second business. In a similar manner, one or more licenses provided by regional data can control and register AR objects associated with locations corresponding to private residences.

[0042] Furthermore, the permission provided by the regional data for a given area can enable the AR object system 116 (e.g., its AR object registration service 236) to determine whether a given user can register (e.g., associate) the sorter with respect to a location corresponding to a set of coordinates on a map (e.g., a topology map). As described herein, the sorter can determine which AR objects are provided (e.g., displayed) to the client device in response to a request or query for AR objects from the client device.

[0043] Table 1 below shows an example structure of a database table used to store regional data for logical topology data layer 214.

[0044]

[0045] According to some implementations, the geographic location data (of the logical topology data layer 214) includes data for storing and registering AR objects associated with one or more coordinates corresponding to locations on a map (e.g., a topology map), data for storing the registration of a sorter associated with one or more coordinates corresponding to locations on a map, or some combination of both. Specifically, for some implementations, the geographic location data may associate data from the 3D topology data layer 212 (e.g., geospatial data) with model data from the AR object model data layer 218. In this way, the geographic location data can facilitate the registration (e.g., placement) of AR objects associated with a set of coordinates corresponding to locations on a map. For some implementations, the geographic location data associates the center (e.g., centroid) of the AR object with this set of coordinates. The center of the AR object may correspond to the center of the AR object's 3D bounding box. When the AR object is ultimately displayed by a client device, the display position and orientation of the AR object can be determined relative to the center of the AR object. Furthermore, for various implementations, geographic location data facilitates the registration of a sorter associated with a set of coordinates corresponding to a location on a map by associating data (e.g., geospatial data) from the 3D topology data layer 212 with an identifier associated with the sorter.

[0046] According to the implementation method, the geographic location data can be implemented as a database join table. Table 2 below can show an example structure of a database table used to store the geographic location data of the logical topology data layer 214.

[0047]

[0048]

[0049] User data layer 216 includes data associated with users of AR object system 116. The data provided by user data layer 216 may include, but is not limited to: data about which AR objects a given user owns or controls, data about the last state of a given AR object relative to a given user, or data about one or more sessions associated with a given user. Table 3 below may represent an example structure of a database table used to store user data in user data layer 216.

[0050]

[0051] Table 4 below can represent an example structure of a database table used to store data in the user data layer 216 for finding user ownership / control of AR objects.

[0052]

[0053]

[0054] AR object model data layer 218 includes data for one or more AR objects that may be registered using AR object system 116. The data stored by AR object model data layer 218 may include, but is not limited to: model data for generating (e.g., rendering) a visual representation of a 3D model of a given AR object; data describing (e.g., pre-computed) 3D bounding boxes of a given AR object; and rule data describing one or more rules for interacting with a given AR object. As described herein, the center of the 3D bounding box associated with a given AR object can determine how the given AR object is positioned and oriented when displayed by a client device relative to the real-world environment (e.g., how the given AR object is embedded in the real-world space presented by the client device). Furthermore, as described herein, one or more rules associated with a given AR object can determine the level of user interaction available relative to the given AR object. For example, one or more rules for a given AR object can determine whether the AR object is static, whether it has client-device-limited interaction, or whether multi-user interaction is permitted through an interactive session.

[0055] According to the implementation, the AR object model data layer 218 can be implemented as a key-value storage data structure. Table 5 below shows an example structure of a database table used to store data in the AR object model data layer 218.

[0056]

[0057] AR object interactive session service 232 facilitates or manages the operation of interactive sessions (hereinafter referred to as sessions) that enable multi-user interaction with one or more registered AR objects (e.g., groups of AR objects). As described herein, sessions allow the transfer of interaction data between client devices of users interacting jointly with one or more AR objects. In some implementations, when a user requests to interact with one or more given AR objects, AR object interactive session service 232 assigns the user to a session, where the assigned session will handle the user's interaction with one or more given AR objects. Furthermore, in some implementations, when a user requests multiple users to interact together with one or more given AR objects (i.e., requests multi-user interaction), AR object interactive session service 232 assigns the user to a session. According to implementations, AR object interactive session service 232 may use different methods to assign users to a given session (e.g., populating a given session with users), such as prioritizing assignment to users who are friends or assigning based on a first-come, first-served service.

[0058] In response to a request from a user's client device to participate in a session to interact with a collection of AR objects (e.g., facilitating multi-user interaction), the AR object interactive session service 232 can assign the user to an existing session that can serve the request (e.g., a session already operating on a session server), or create and assign the user to a new session (e.g., start a new session on the session server) to serve the request. In this document regarding... Figure 3 and Figure 4 More details are described regarding session assignment and session operations using mapping servers (e.g., world servers) and session servers.

[0059] In some implementations, users are restricted to participating in one session at a time. Users participating in a given session can remain idle (e.g., based on a lack of activity or interaction within the session after a period of time). Furthermore, user participant count limits can be assigned to a given session to ensure that the given session operates as expected by the participating users. User participant count limits may vary from session to session. For example, user participant count limits may be based on the geographic area / region served by different sessions (e.g., an area around a landmark, such as the Washington Monument, may involve more user AR object interactions and therefore have a lower count limit than an area covering a small city park).

[0060] In some implementations, each user's client device participating in a given session shares data about that user's participation in the given session with the client devices of all other users participating in the given session. This data may include, but is not limited to, user input to the given session (e.g., swiping, head tilting, etc.) and changes in the state of AR objects involved in the given session caused by user interactions. In various implementations, data sharing between client devices is facilitated through operations within the given session.

[0061] The state of AR objects involved in a session can be referred to as the session state of the AR objects. The current session state of the AR objects in a session can serve as the "ground truth" of a user's interaction with the AR objects through the session. For a given session involving interaction with one or more given AR objects, the client device of a user participating in (e.g., assigned and involved in) the given session can receive the initial state of each of those given AR objects when the user participates in the session. The client device uses the session to initialize the session state of each of those given AR objects at the client device. The user can then participate in the session by, for example, interacting with one or more of the given AR objects, or by observing another user in the session interacting with one or more of the given AR objects. When a user participates in a given session, the user's client device can work locally and maintain (e.g., store and update) a local copy of the session state of each of the given AR objects at the client device. For example, the client device can update the locally maintained session state of the first AR object in a given session based on the user's interaction with the first AR object. Simultaneously, the client device can update the locally maintained session state of the first AR object in a given session based on session state update data received by the client device regarding the interaction between another user participating in the given session and the first AR object (e.g., session state update data broadcast by other users' client devices to all user client devices through the given session). Depending on the interaction level of the given AR object (e.g., defined by rules associated with the given AR object), the final session state of the given AR object can be stored (e.g., persistently stored) in the AR object system 116 when the given session terminates (e.g., stored for users in the given session in the user data layer 216, or stored for all users via the AR object model data layer 218). For example, the rules of the given AR object may define that the given AR object can be interacted with through a session, and any changes to the session state of the given AR object will eventually be saved in the AR object system 116. Once this final session state is stored in the AR object system 116, the stored session state can be used as the initial / starting state of the given AR object the next time one or more users begin interacting with the given AR object again (e.g., in a new session). By maintaining a local copy of the session state at the client device and saving the final state of a given AR object (where applicable) at the end of the session, various implementations can make the AR object interactive session service 232 scalable, improve the stability of the AR object interactive session service 232, and reduce or avoid / reduce overwrite thrash of the AR object interactive session service 232.

[0062] AR object interactive session service 232 can support multiple simultaneous sessions involving interactions with the same AR object. The sessions supported by AR object interactive session service 232 can operate independently. Therefore, for a given AR object in a given session, access to the state of the given AR object within that session (the session state of the given AR object) can be maintained, preventing access to the session state outside the given session (e.g., by a user not participating in the given session). This means that two independent simultaneous sessions may involve users interacting with the same specific AR object, but each of those independent simultaneous sessions maintains its own session state for that specific AR object. As described herein, session independence allows some implementations to manage (e.g., generate, allocate, and operate) sessions using a mapping server and multiple session servers (e.g., independent session servers responsible for serving sessions based on real-world geographic partitions), which can provide scalability and stability for users. For example, the independent session approach means that implementations can provide one or more users (within a single session) with the experience of seeing an AR object created by a single user relative to a real-world object (e.g., placing a virtual hate on a real-world statue) for a satisfactory amount of time (rather than less than a small amount of time, which would occur if all users were interacting with a given AR object operated to be assigned to the same session or if the session state of a given AR object was shared across multiple sessions). In cases where two or more simultaneous sessions involve one or more identical AR objects, the final session state of these identical AR objects can be merged using a merge rule feature if the final session state of these identical AR objects is to be stored after the simultaneous sessions terminate.

[0063] In this article about Figure 3 and Figure 4 More details about the operation of AR object interactive session service 232 are described.

[0064] AR object query service 234 processes requests or queries from client devices for one or more AR objects from an AR registry implemented by AR object system 116 (e.g., via geographic location data from logical topology data layer 214). Based on the received request / query, AR object query service 234 may determine one or more AR objects (from the AR registry) to be sent back to the client device for use. In this way, AR object query service 234 operates as an AR object surface service, assuming that one or more AR objects provided to the client device by AR object query service 234 (e.g., based on a client request or query) cause (or may cause) the client device to render or display those one or more AR objects on the client device. Requests / queries to AR object query service 234 may be generated by a client application (e.g., 104) on the client device, where the client application may use one or more AR objects provided by AR object query service 234 to render a view relative to the real-world environment (e.g., to provide a mixed reality user experience). As described herein, a request / query may include information associated with a client device, such as information about the user of the client device, the current set of coordinates of the client device (e.g., GPS coordinates), or a specified radius around the client device.

[0065] In some implementations, AR object query service 234 uses one or more sorters to determine which one or more AR objects are sent back to the client device in response to a request / query. Using the sorters, AR object query service 234 can prioritize, filter, or sort AR objects to determine the final set of AR objects to be sent to the client device. For example, AR object query service 234 can determine (e.g., identify) an intermediate / initial set of AR objects from an AR registry table based on a client request / query, and then use the sorters to filter and sort the intermediate / initial set of AR objects to determine the final set of AR objects to be sent to the client device. Alternatively, the sorter can receive the client request / query and generate a sorter-based query that includes one or more parameters for prioritizing, filtering, or sorting AR objects, and the result is provided in response to the sorter-based query. For example, filtering, sorting, or both can be performed on attributes associated with geographic location data from logical topology data layer 214.

[0066] A sorter can be implemented such that it is horizontally scalable. According to an implementation, the AR object query service 234 may have one or more sorters available. The AR object query service 234 may select and use one or more sorters based on a number of factors, including, for example, information provided in a client request / query (e.g., client device geographic location or a specific radius) or user selection or preference (from multiple available sorters). As described herein, the AR object registration service 236 can facilitate the registration of new or existing sorters using the AR object system 116, thereby enabling the availability of those new / existing sorters to be used by the AR object query service 234. Example sorters may include query result limits (e.g., limiting to 25 AR objects). Another example sorter may include an algorithm that selects AR objects based on pseudo-random fairness and then sorts the selected AR objects.

[0067] In some implementations, the AR object query service 234 may use a sorter that accesses bidding data (e.g., provided by a bidding system) to determine the priority of the AR object set. This allows the AR object query service 234 to filter the AR object set based on the determined priority. For example, for a set of AR objects falling within a specific radius centered on a location corresponding to the location of the client device, the sorter may (e.g., in real time) access the bidding data of one or more AR objects (in the AR object set) to determine the priority of those one or more AR objects. The bidding data for each AR object can be accessed by performing a monetization lookup for each AR object.

[0068] Bidding data can be provided by a bidding system, which can be standalone or part of the AR object system 116. In the standalone case, the bidding system can have at least read access to data of the AR object system 116 (e.g., geographic location data of the logical topology data layer 214), which can facilitate bidding for the registration / placement of AR objects.

[0069] Some implementations provide or utilize a bidding system that allows one or more users (e.g., representing a third-party organization) to bid for AR objects of their choice over other AR objects (displays). For example, one or more users can bid to prioritize (e.g., increase priority) their chosen AR objects in association with a coordinate set, a region relative to (e.g., around) the coordinate set, and a marked region (e.g., described by region data from logical topology data layer 214), targeting certain users or certain types of users. For example, for an already registered AR object (an AR object registered in association with a coordinate set corresponding to a location on a map), a user can bid to increase the priority of the registered AR object (e.g., above other AR objects registered / placed relative to the same coordinate set or surrounding locations). By increasing the priority of registered AR objects through winning bids, users can effectively improve the display / presentation / display ranking of registered ARs on client devices. For example, based on a sorter associated with bidding data, the request / query results provided to the client device may include a predetermined number of AR objects with the highest bids. Bidding can include the value offered by the bidding (e.g., monetary value or virtual credit), and can also include a priority value requested through bidding (e.g., the amount of priority increase or actual priority value). In a bidding system, bidding can include a user associating monetary value / virtual credit with a geographic location data record (e.g., the position_id corresponding to the record) in the logical topology data layer 214.

[0070] By using a sorter, AR object system 116 can perform distributed queries of the AR registry to find AR objects. Furthermore, the use of a sorter can improve the user experience by refining which AR objects are presented / displayed to the user on the client device. For example, by using a sorter, AR object query service 234 can allow users to view different types of AR objects based on, for example, the time of year or geographical location. For instance, an AR object a user might want to see in Las Vegas is not necessarily one they would want to see at their family's Thanksgiving dinner.

[0071] More information about operating AR Object Query Service 234 is available in this article. Figure 5 Describe it.

[0072] Through AR object registration service 236, users can manage (e.g., add, remove, or modify) the registration of AR objects associated with one or more coordinates corresponding to locations on a map (e.g., a topology map), the registration of sorters associated with one or more coordinates corresponding to locations on the map, or both. For example, users can use AR object registration service 236 to generate new registrations for AR objects with one or more coordinates corresponding to locations on a map. New registrations can be for AR objects newly added to AR object model data layer 218 or already stored on AR object model data layer 218. As described herein, registering AR objects associated with a set of coordinates can effectively place AR objects at locations corresponding to the set of coordinates (e.g., placing AR objects relative to a real-world map to achieve mixed reality). Depending on the implementation, the registration of AR objects or sorters can be transient.

[0073] In some implementations, AR object registration service 236 uses permission data to determine whether a given user can register a given AR object, a given sorter, or both relative to one or more coordinates on a map. For example, as described herein, region data from logical topology data layer 214 can provide permission data associated with one or more areas of the real-world environment (e.g., marked areas described by the region data). Furthermore, in some implementations, AR object registration service 236 implements one or more rate limits regarding registration requests (e.g., requests to add, remove, or update registrations). For example, a rate limit could restrict a given user to five registrations per day via AR object registration service 236. In another example, a rate limit could restrict a given user to a predetermined number of registrations per day, and the given user must pay for more than the predetermined number of registrations within a day. Through rate limits, some implementations can avoid sending spam registration requests to AR object registration service 236.

[0074] According to the implementation, AR object registration service 236 can allow or facilitate the registration of AR objects, sorters, or both in the public domain (e.g., public registration). For example, a user (e.g., from the public) can construct a new AR object or a new sorter and register this new item via AR object registration service 236.

[0075] In some implementations, AR object registration service 236 stores AR object registrations (with respect to a set of coordinates corresponding to locations on a map) in the geographic location data of logical topology data layer 214 as described herein (e.g., using model_id from Table 2). Similarly, in some implementations, AR object registration service 236 stores the registration of sorters (e.g., with respect to a set of coordinates corresponding to locations on a map) as geographic location data of logical topology data layer 214 as described herein (e.g., using ranking_id from Table 2). Some implementations may facilitate the registration of AR objects or sorters related to attributes of client devices (e.g., the identity of a particular client device, the type of client device, the version of the operating system, etc.) or attributes of client applications (e.g., the identity or version of a particular client application or a particular client application type, such as a web browser, social network, or messaging application).

[0076] Figure 3 This is a block diagram illustrating an example of an AR object interactive session service 232 according to some implementations. Figure 3As shown, the AR object interactive session service 232 includes one or more mapping servers 302 (e.g., world servers) and one or more session servers 304. According to some embodiments, a particular mapping server 302 can identify a session (operating on a particular session server 304) and assign that session to a client device, and a particular session server 304 can operate one or more sessions supporting user interaction with one or more AR objects (e.g., multi-user interaction). According to some embodiments, a user's client device sends a request to use a session to interact with a collection of AR objects (e.g., facilitating multi-user interaction). One or more mapping servers 302 can receive the request, identify a particular session server 304 (hereinafter referred to as the identified session server 304) among the session servers to serve the request, assign the user or client device to a new or existing session operating on the identified session server 304 that can serve the request, and reroute or otherwise redirect the client device to the identified session server 304. According to the implementation, mapping server 302 can determine which session server 304 will serve a given request based on, for example, the coordinate set of the client device, the user's identity, the current load of the session server, and the association between session server 304 and a marked region of the real-world environment (e.g., geopartitioning). For example, mapping server 302 can determine which session server 304 will serve a given request such that multiple simultaneous users interacting with the same set of AR objects are grouped in a way that does not overload any session server 304 while maintaining priority user grouping within the session (e.g., placing users who are friends together in the same session).

[0077] A given session server 304 can operate multiple concurrent sessions (e.g., based on its specific load or capacity). As described herein, a given session maintains its own session state for each AR object involved in the AR object within the given session, and that session state is maintained to be inaccessible outside the given session. The given session server 304 can operate virtual, canonical copies of the session. Once multiple client devices of users participating in (operating on the given session server 304) of the given session have established data connections to the given session, each client device can transmit data (such as user input (e.g., swipes, head tilts, etc.)) or update the session state of AR objects to the given session, and the given session can share data with other client devices connected to the given session. Client devices can use, for example, low-latency, User Datagram Protocol (UDP)-based connections to share data with the given session. After receiving user input from a client device, a given session can validate the user input (e.g., to block or avoid bots or scammers) and can (e.g., using the same low-latency, UDP-based connection) share the validated user input with all other client devices, so that client devices can update their local copies of the session state of their AR objects accordingly based on the validated user input. A given session can also update some session information based on the validated user input.

[0078] Figure 4 This is a flowchart illustrating an example of session processing performed by an AR object interactive session service (e.g., 232) according to some implementations. Initially, a client application operating on client device 404 can cause client device 404 to request / query one or more AR objects from AR object query service 402 (e.g., a request / query based on a set of coordinates corresponding to the current position and radius values ​​of client device 404). At operation 420, client device 404 can download data for one or more AR objects generated by the request / query, which may include model data and rule data for one or more AR objects. Subsequently, a user can interact with one or more AR objects according to one or more rules described by the rule data. Finally, a user can request a session to facilitate multi-user interaction regarding at least one of the one or more AR objects. Therefore, at operation 422, client device 404 can initialize a connection with mapping server 406 (e.g., a world server), which allows client device 404 to send its request for a session. In response to the request, at operation 424, the mapping server 406 may check the AR object interactive session cache 410 to determine whether any existing sessions associated with the at least one AR object exist (client device 404 can be assigned to any existing session) or whether a new session needs to be created for the request. Figure 4 In this context, the AR object interactive session cache 410 can cache information about sessions currently being operated by one or more session servers. Therefore, a session server, such as session server 408, can periodically update the information stored in the AR object interactive session cache 410 (as indicated in operation 428). After mapping server 406 identifies client device 404 and assigns it to a new or existing session, at operation 426, mapping server 406 can redirect client device 404 to the session server (represented by session server 408) operating on the assigned session. Once client device 404 is redirected to session server 408 and a data connection to the assigned session is established, at operation 430, the client device can send its user input to the assigned session (which is shared by the assigned session and other client devices connected to the assigned session), and the client device can receive user input from other user client devices participating in the assigned session. Based on the received user input, client device 404 can update its local copy of the session state of the AR objects involved in the assigned session.

[0079] Figure 5This is a flowchart illustrating an example of providing one or more AR objects to a client device using one or more sorters according to some implementations. Initially, a client application operating on client device 502 can cause client device 502 to request / query one or more AR objects from AR object query service 504 (e.g., based on a set of coordinates and radius values ​​corresponding to the current location of client device 504). Operation 530 can represent client device 502 sending a request / query to AR object query service 504. The request / query can be generated by a user of client device 502 using client device 502 (e.g., a smartphone) to scan their surrounding real-world environment for AR objects. AR object query service 504 can (e.g., based on the set of coordinates provided by the request / query) determine one or more sorters associated with the received request / query. One of the determined sorters can be a sorter that accesses bidding data from bidding system 506 at operation 532 and prioritizes one or more AR objects over others. As described herein, bidding system 506 allows users to bid on prioritization of registered AR objects (e.g., increasing the priority of registered AR objects). At operation 534, AR object query service 504 can query geolocation data 508 to determine an intermediate set of AR objects associated with coordinates within a radius of the current geolocation of client device 502, and then apply one or more determined sorters to the intermediate set of AR objects (e.g., filtering or sorting the intermediate set of AR objects) to arrive at a final set of AR objects. At operation 536, AR object query service 504 can obtain (e.g., retrieve) data for the final set of AR objects, which may include, for example, data from AR object model data 510 and rule data associated with the final set of AR objects. At operation 538, the data for the final set of AR objects is provided to client device 502 and downloaded by client device 502 (as indicated in 512). At operation 540, client device 502 can determine the positioning of the virtual camera relative to the display of client device 502 (as shown in 514), and at operation 542, client device 502 can display a rendered model of one or more AR objects from the final set of AR objects based on the positioned virtual camera (as shown in 516). Subsequently, the user of client device 502 can interact with the AR objects displayed on client device 502.

[0080] Figure 6This is a block diagram illustrating an example implementation of an AR object system 116 according to some embodiments. The AR object system 116 is shown as including an augmented reality (AR) object query module 602, an augmented reality (AR) object interactive session module 604, an augmented reality (AR) object registration form module 606, an augmented reality (AR) object bidding module 608, a three-dimensional (3D) topology data module 610, a logical topology data module 612, a user data module 614, and an augmented reality (AR) object model data module 616. The various modules of the AR object system 116 are configured to communicate with each other (e.g., via a bus, shared memory, or switch). Any one or more of these modules can be implemented using one or more processors 600 (e.g., by configuring one or more such processors 600 to perform the functions described for that module), and thus may include one or more processors of processor 600.

[0081] Any one or more modules described may be implemented using hardware alone (e.g., one or more computer processors of a machine such as machine 1500) or using a combination of hardware and software. For example, any described module of AR object system 116 may physically include an arrangement of one or more processors 600 configured to perform the operations described herein for that module (e.g., a subset of one or more processors of a machine such as machine 1500 or one or more processors of said machine). As another example, any module of AR object system 116 may include software, hardware, or both that configure an arrangement of one or more processors 600 (e.g., one or more processors of a machine such as machine 1500) to perform the operations described herein for that module. Thus, different modules of AR object system 116 may include and configure different arrangements of such processors 600 at different points in time or a single arrangement of such processors 600. Furthermore, any two or more modules of AR object system 116 may be combined into a single module, and the functionality described herein for a single module may be subdivided among multiple modules. Furthermore, according to various implementation methods, modules described herein as being implemented within a single machine, database, or device can be distributed across multiple machines, databases, or devices.

[0082] AR object query module 602 is used to facilitate or realize the AR object query module mentioned in this article. Figure 2 The AR object query service 234 is described in terms of its various aspects, features, or functions. The AR object interactive session module 604 is used to facilitate or implement the aspects described herein. Figure 2The AR object interactive session service 232 is described in terms of its various aspects, features, or functions. The AR object registration form module 606 is used to facilitate or implement the aspects described herein. Figure 2 The AR object registration form service 236 is described in terms of its aspects, features, or functions. In some implementations, the AR object registration form module 606 also supports registration of sorters as described herein. The AR object bidding module 608 is used to facilitate or implement the aspects described herein. Figure 2 The AR object query service 234 describes various aspects, features, or functions of the bidding system. The 3D topology data module 610 is used to facilitate or implement the aspects described in this document. Figure 2 This describes various aspects, features, or functions of the 3D topology data layer 212. The logical topology data module 612 is used to facilitate or implement the aspects described herein. Figure 2 This describes various aspects, features, or functions of the logical topology data layer 214. The user data module 614 is used to facilitate or implement the aspects described herein. Figure 2 The user data layer 216 is described in terms of its various aspects, features, or functions. The AR object model data module 616 is used to facilitate or implement the aspects described herein. Figure 2 Describe the various aspects, features, or functions of the AR object model data layer 218.

[0083] In some implementations, the world server set and session server set are used to implement or operate the AR object interactive session module 604. Additionally, in some implementations, the AR object query module 602 is implemented or operated on a query server set that is separate from the world server set and session server set used to operate the AR object interactive session module 604. This will be discussed below regarding... Figures 7 to 13 The operation of the described method is used to provide more information about modules 602 to 616.

[0084] Figures 7 to 13 This is a flowchart illustrating a method related to an AR object registration form according to some implementations. (The text then repeats itself, so the translation stops here.) Figures 7 to 13The various methods described can be embodied in machine-readable instructions that are executed by one or more computer processors, such that the operation of the methods can be performed partially or wholly by server system 108 or more specifically AR object system 116. Therefore, various methods are described herein by way of example with reference to AR object system 116. At least some of the operations of method 800 can be deployed on various other hardware configurations, and the methods described herein are not intended to be limited to operation by server system 108. Although the steps of the methods described herein can be depicted and described in a certain order, the order in which operations are performed can vary between implementations. For example, one operation may be performed before, after, or simultaneously with another operation. Furthermore, in some implementations, the components described with respect to the methods are merely examples of components that can be used with the methods, and other components may also be utilized.

[0085] Now refer to Figure 7 The diagram illustrates a method 700 for providing AR objects to a client device and processing a session for interacting with the provided AR objects. At operation 702, the AR object query module 602 receives a query from the client device for one or more augmented reality objects, wherein the query may include a current set of coordinates corresponding to the client device's location on a map, and may also include a radius relative to the current set of coordinates (e.g., centered on the location corresponding to the current set of coordinates).

[0086] In response to the query received at operation 702, at operation 704, AR object query module 602 determines (e.g., identifies) a set of augmented reality objects based on the query, and at operation 706, sends the query results to the client device, wherein the query results include result data of the set of augmented reality objects determined by operation 704. Determining the set of augmented reality objects based on the query may include: AR object query module 602 performing a search based on the received query. The set of augmented reality objects can be determined from a plurality of augmented reality objects registered in an augmented reality object registry (e.g., registered via AR object registry module 606) via operation 704. As described herein, based on the result data provided to the client device through the query results, the client device may display (or show) one or more augmented reality objects from the set of augmented reality objects.

[0087] According to an implementation, the result data may include the current stored state of at least one augmented reality object (a state stored on the AR object system 116), wherein the current stored state, once provided to the client device, can determine the initial state of at least one augmented reality object for the user on the client device. The result data may include model data for each augmented reality object in the augmented reality object set. The result data may include location (e.g., positioning) data describing a given set of coordinates on a map for each augmented reality object in the augmented reality object set, at which the given client device will display the augmented reality object when generating an augmented reality view relative to the given set of coordinates. The result data may include orientation data describing a given orientation for each augmented reality object in the augmented reality object set, at which the given client device will display the augmented reality object in the given orientation when generating an augmented reality view including the augmented reality object. Additionally, the result data may include rule data describing a set of interaction rules associated with the augmented reality object set, wherein the set of interaction rules can determine the interactions available to the user (on the first client device) regarding the augmented reality object set. The augmented reality registry of the AR object system 116 can associate a given augmented reality object with one or more interaction rules.

[0088] At operation 708, the AR object interactive session module 604 receives from the client device a request from the user at the client device to interact with at least one augmented reality object in the augmented reality object set (at operation 704, the augmented reality object set is determined at operation 704; and at operation 706, the query result for the augmented reality object set is sent to the client device).

[0089] In response to the request received at operation 708, at operation 710, the AR object interactive session module 604 determines (e.g., identifies) a given session server to serve the request received at operation 708, and at operation 712, assigns the client device to a given session operating on the given session server. The given session can be a new session created by the given session in response to a request, or an existing session involving the same set of AR objects associated with the request. In some implementations, the AR object interactive session module 604 may check a session cache to determine if an existing session already exists for the request. The given session server determined at operation 710 may be associated with a geographic partition of a map containing the location of the client device on the map. As described herein, the given session may facilitate interaction between a user of the client device and at least one augmented reality object. Furthermore, as described herein, the given session may maintain session state regarding at least one augmented reality object for one or more users associated with (e.g., participating in) the given session, wherein the session state may be updated based on the interaction of at least one user with at least one augmented reality object. A given session server can be determined from multiple given session servers (e.g., 304), and a mapping server (e.g., 302) can perform the determination of the given session server. In some implementations, the multiple session servers operate on a first set of computer devices, which are separate from a second set of computer devices operating the mapping servers. In some implementations, assigning a first client device to a given session operating on a given session server includes redirecting the client device from the mapping server to the given session server. Once a given session is assigned to a user, user data can be updated via user data module 614. After assignment, a network connection can be established between the client device and the (assigned) given session on the given session server.

[0090] Now refer to Figure 8 This illustrates a method 800 for providing AR objects to client devices and handling sessions where multiple users interact with the provided AR objects. For some implementations, operations 802 to 806 are respectively similar to those described above. Figure 7The described method 700 comprises operations 702 to 706, and is performed on a first client device (as associated with a first user). At operation 808, the AR object interactive session module 604 receives a request from the first client device for multiple users to interact together with at least one augmented reality object in a set of augmented reality objects (e.g., a multi-user interactive session). (At operation 804, the set of augmented reality objects is determined; and at operation 806, a query result for the set of augmented reality objects is sent to the first client device.) As described herein, a multi-user interactive session can facilitate interaction between multiple users and at least one augmented reality object.

[0091] In response to the request received at operation 808, at operation 810, the AR object interactive session module 604 determines (e.g., identifies) a given session server to serve the request received at operation 808, and at operation 812, assigns the first client device to a given session operating on the given session server. As described herein, the given session server determined at operation 810 may be associated with a geographic partition of a map containing the location of the first client device on the map. Additionally, at operation 814, the AR object interactive session module 604 assigns a second client device associated with a second user to the same given session operating on the same given session server (determined at operation 810), wherein the first user of the first client device and the second user of the second client device are part of a plurality of users who received the session request at operation 808. Furthermore, other users among the plurality of users can be assigned to the same given session on the same given session server in a similar manner.

[0092] Now refer to Figure 9 The diagram illustrates a method 900 for providing AR objects to a client device and handling a session for interacting with the provided AR objects. For some implementations, operations 902 to 912 are similar to those described above. Figure 7 Operations 702 to 712 of the described method 700. At operation 914, upon termination of a given session, the AR object interactive session module 604 stores (or causes to store) a final version of the session state of at least one augmented reality object. As described herein, the final version of the session state of a given augmented reality object can be determined (e.g., adjusted) through interactions with users participating in the given session.

[0093] Now refer to Figure 10The diagram illustrates a method 1000 for registering AR objects to an AR object registry. At operation 1002, the AR object registry module 606 receives a request from a client device associated with a user to register a given augmented reality object associated with a given set of coordinates on a map in the augmented reality object registry.

[0094] In response to the request received at operation 1002, at operation 1004, the AR object registration form module 606 determines, based on permission data, whether the user has permission to register a given augmented reality object associated with a given set of coordinates on a map. In some implementations, the permission data describes the association between at least one set of coordinates on the map and a permission set. The permission data may be associated with a region on the map that contains markers for the given set of coordinates. Therefore, in some implementations, the permission data may be provided by area data accessible through the logical topology data module 612.

[0095] Furthermore, in response to the request received at operation 1002, operation 1006 is performed. At operation 1006, based on determining whether the user has permission, the AR object registration module 606 registers the given augmented reality object associated with a given set of coordinates on the map on the augmented reality object registration form. When doing so, the AR object registration module 606 can designate the user as the owner or controller of the registration.

[0096] Now refer to Figure 11 The diagram illustrates a method 1100 for providing AR objects to a client device based on one or more sorters. At operation 1102, the AR object query module 602 receives a query from the client device for one or more augmented reality objects, wherein the query may include a current set of coordinates corresponding to the client device's location on a map, and may also include a radius relative to the current set of coordinates (e.g., centered on the location corresponding to the current set of coordinates).

[0097] In response to the query received at operation 1102, at operation 1104, AR object query module 602: determines (e.g., identifies) an intermediate set of augmented reality objects based on the query; at operation 1106, determines a set of sorters for the query, wherein at least one sorter in the set is configured to filter or sort the set of augmented reality objects; and at operation 1108, generates a final set of augmented reality objects by applying the set of sorters to the intermediate set of augmented reality objects (e.g., filtering or sorting according to the sorters). Example sorters may include sorters that apply at least one of filters or sorting orders to the set of augmented reality objects.

[0098] Another example sorter may include a sorter that filters an augmented reality object set based on a priority set of the augmented reality object set. Priorities may be provided (or determined) for example, by geographic location data (e.g., provided via logical topology data module 612) or by bidding data associated with one or more augmented reality objects (e.g., provided via AR object bidding module 608). As described herein, a bidding system (e.g., implemented by AR object bidding module 608) may enable users to bid on AR object registrations to adjust (e.g., increase) the priority of that AR object registration. Through AR object bidding module 608, the sorter may: request priority information for the augmented reality object set from the bidding system; and receive priority data from the bidding system describing the priority of at least one of the augmented reality objects in the set.

[0099] The determination of at least one sorter (e.g., identification) can be based on the association between the sorter and a user of the client device (e.g., the user chose to use the sorter or the user registered). The determination of at least one sorter can be based on the current set of coordinates corresponding to the location of the client device. Doing so allows the sorter to be applied to a radius around the client device. The determination of at least one sorter can be based on attributes of the client device, such as the client device's identity or device type. The determination of at least one sorter can be based on at least one of a set of dates (e.g., range) or a set of times. Doing so allows the sorter to be applied based on different parts of the year (e.g., according to the seasons of the year).

[0100] As an alternative to operations 1104 to 1108, in some implementations, in response to a query received at operation 1102, AR object query module 602: determines a set of sorters for the client query (e.g., wherein at least one sorter in the set includes a filter parameter for filtering the set of augmented reality objects or a sorting order parameter for sorting the set of augmented reality objects); and generates (e.g., constructs) a query (a sorter-based query) based on the client query and the set of sorters; and then determines (e.g., identifies) the final set of augmented reality objects based on the sorter-based query.

[0101] At operation 1110, AR object query module 602 sends query results to the client device, wherein the query results include result data of the final augmented reality object set (e.g., the final set determined by operation 1108 or the alternative method). As described herein, the result data of the final augmented reality object set may include various types of data (e.g., location data, model data, orientation data, etc.) of one or more augmented reality objects in the final set.

[0102] Now refer to Figure 12 This illustrates a method 1200 for providing AR objects to a client device based on one or more sorters involved in a bidding system. For some implementations, operations 1202 through 1206 are respectively similar to those described above regarding... Figure 11 The described method 1100 comprises operations 1102 to 1106. At operation 1208, the AR object query module 602 requests priority information from the bidding system (via the AR object bidding module 608) for the intermediate set of augmented reality objects determined at operation 1204. In some implementations, operation 1208 is performed based on at least one of the sorters determined at operation 1206 (e.g., the sorter uses the priority information of the augmented reality objects to filter or sort these objects). At operation 1210, the AR object query module 602 receives priority data (or bidding data) from the bidding system, which describes the priority of at least one augmented reality object in the intermediate set of augmented reality objects.

[0103] In some implementations, operations 1212 and 1214 are similar to those described above. Figure 11 Operations 1108 and 1110 of method 1100 are described herein. As described herein, the priority information obtained via operations 1208 and 1210 enables a sorter applied to the intermediate set of augmented reality objects (via operation 1212) to filter or sort the intermediate set of augmented reality objects.

[0104] Now refer to Figure 13 The diagram illustrates a method 1300 for registering a sorter to a sorter registry (which may be implemented as part of an AR object registry). At operation 1302, the AR object registry module 606 receives a request from a client device associated with a user to register a given sorter on the sorter registry (e.g., associated with a given set of coordinates on a map, a marked area, a specific client device, a client device type, a user, a user type, a time of day, a date, a season, etc.).

[0105] In response to the request received at operation 1302, at operation 1304, the AR object registration form module 606 determines whether a user has permission to register for a given sorter based on permission data. In some implementations, the permission data describes the association between at least one set of coordinates on a map and a set of permissions. The permission data may be associated with a region on the map containing markers for a given set of coordinates. Therefore, in some implementations, the permission data may be provided by area data accessible through the logical topology data module 612.

[0106] Furthermore, in response to the request received at operation 1302, operation 1306 is performed. At operation 1306, based on determining whether the user has permission, the AR object registration module 606 registers a given sorter (e.g., associated with a given set of coordinates on a map, a marked area, a specific client device, client device type, user, user type, time of day, date, season, etc.) on the sorter registration table. When doing so, the AR object registration module 606 can designate the user as the registered owner or controller. A sorter can be registered for use only by the user, or made available for use by other users on the AR object system 116.

[0107] Figure 14 This is a block diagram illustrating example software architecture 1406, which can be used in conjunction with various hardware architectures described herein. Figure 14 This is a non-limiting example of a software architecture, and it should be understood that many other architectures can be implemented to facilitate the functionality described herein. Software architecture 1406 can be implemented in, for example... Figure 15 The execution is performed on the hardware of machine 1500, which includes processor 1504, memory / storage device 1506, and I / O components 1518, etc. A representative hardware layer 1452 is shown, and this representative hardware layer 1452 can represent, for example... Figure 15 The machine 1500. A representative hardware layer 1452 includes a processing unit 1454 having associated executable instructions 1404. The executable instructions 1404 represent executable instructions of the software architecture 1406, including implementations of the methods, components, etc., described herein. Hardware layer 1452 also includes a memory or storage module memory / storage device 1456, which also has the executable instructions 1404. Hardware layer 1452 may also include other hardware 1458.

[0108] exist Figure 14 In the example architecture, software architecture 1406 can be conceptualized as a stack of layers, where each layer provides specific functionality. For example, software architecture 1406 may include layers such as operating system 1402, library 1420, application 1416, and presentation layer 1414. Operationally, application 1416 or other components within a layer can call application programming interface (API) calls 1408 through the software stack and receive a response to API call 1408 in the example form of message 1412. The layers shown are representative in nature, and not all software architectures have all layers. For example, some mobile or dedicated operating systems may not provide a framework / middleware 1418, while other operating systems may provide such a layer. Other software architectures may include additional or different layers.

[0109] Operating system 1402 can manage hardware resources and provide general services. Operating system 1402 may include, for example, kernel 1422, services 1424, and drivers 1426. Kernel 1422 can act as an abstraction layer between the hardware layer and other software layers. For example, kernel 1422 may be responsible for memory management, processor management (e.g., scheduling), component management, networking, security settings, etc. Services 1424 can provide other public services to other software layers. Driver 1426 is responsible for controlling or interfacing with the underlying hardware. For example, depending on the hardware configuration, driver 1426 may include display drivers, camera drivers, etc. Drives, flash drives, serial communication drives (e.g., Universal Serial Bus (USB) drives), Drivers, audio drivers, power management drivers, etc.

[0110] Library 1420 provides common infrastructure used by application 1416 or other components or layers. Library 1420 provides functionality that allows other software components to perform tasks more easily than by directly interfacing with the underlying operating system 1402 functions (e.g., kernel 1422, services 1424, and / or drivers 1426). Library 1420 may include system libraries 1444 (e.g., the C standard library), which provide functions such as memory allocation functions, string manipulation functions, mathematical functions, etc. Furthermore, library 1420 may include API libraries 1446, such as media libraries (e.g., libraries supporting the rendering and manipulation of various media formats such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG), graphics libraries (e.g., the OpenGL framework for rendering 2D and 3D graphics content on a display), database libraries (e.g., SQLite providing various relational database functionalities), network libraries (e.g., WebKit providing web browsing functionality), etc. Library 1420 may also include a wide variety of other libraries 1448 to provide many other APIs to application 1416 and other software components / modules.

[0111] The framework / middleware 1418 (sometimes referred to as middleware) provides a higher level of common infrastructure that can be used by application 1416 or other software components / modules. For example, the framework / middleware 1418 can provide a variety of graphical user interface (GUI) functions, advanced resource management, advanced location services, etc. The framework / middleware 1418 can provide a wide range of other APIs that can be used by application 1416 or other software components / modules, some of which may be specific to a particular operating system 1402 or platform.

[0112] Application 1416 includes either built-in application 1438 or third-party application 1440. Examples of representative built-in applications 1438 may include, but are not limited to: contact applications, browser applications, book reader applications, location applications, media applications, messaging applications, or game applications. Third-party applications 1440 may include those using Android by entities other than the platform-specific vendor. TM or iOS TM Applications developed using a Software Development Kit (SDK) can be used on platforms such as iOS. TM ANDROID TM , Mobile software running on a mobile operating system or other mobile operating systems. Third-party application 1440 may invoke API calls 1408 provided by the mobile operating system (such as operating system 1402) to facilitate the functions described herein.

[0113] Application 1416 can use built-in operating system functions (e.g., kernel 1422, service 1424, or driver 1426), libraries 1420, and frameworks / middleware 1418 to create user interfaces to interact with the system's users. Alternatively or additionally, in some systems, user interaction may occur through a presentation layer such as presentation layer 1414. In these systems, the application / component "logic" can be separated from the user-interacting aspects of the application / component.

[0114] Figure 15 This is a block diagram illustrating components of a machine 1500, according to some embodiments, capable of reading instructions from a machine-readable medium (e.g., a machine-readable storage medium) and performing any or more of the methods discussed herein. Specifically, Figure 15A graphical representation of a machine 1500 in the form of an example computer system is shown, in which instructions 1510 (e.g., software, programs, applications, applets, or other executable code) can be executed to cause the machine 1500 to perform any or more of the methods discussed herein. Thus, instructions 1510 can be used to implement the modules or components described herein. Instructions 1510 transform a general, unprogrammed machine 1500 into a specific machine 1500 programmed to perform the described and illustrated functions in the described manner. In alternative embodiments, machine 1500 operates as a standalone device or can be coupled (e.g., networked) to other machines. In a networked deployment, machine 1500 can operate as a server machine or client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. Machine 1500 may include, but is not limited to, server computers, client computers, personal computers (PCs), tablet computers, laptop computers, netbooks, set-top boxes (STBs), personal digital assistants (PDAs), entertainment media systems, cellular phones, smartphones, mobile devices, wearable devices (e.g., smartwatches), smart home devices (e.g., smart home appliances), other smart devices, networked home appliances, network routers, network switches, network bridges, or any machine capable of sequentially or otherwise executing instructions 1510 specifying actions to be taken by machine 1500. Furthermore, although only a single machine 1500 is shown, the term "machine" should also be considered to include a collection of machines that individually or jointly execute instructions 1510 to perform any or more of the methods discussed herein.

[0115] Machine 1500 may include a processor 1504, a memory / storage device 1506, and an I / O component 1518 that can be configured to communicate with each other, for example, via bus 1520. Memory / storage device 1506 may include memory 1514, such as main memory or other memory storage devices, and storage cells 1516, both of which can be accessed by processor 1504, for example, via bus 1502. Storage cells 1516 and memory 1514 store instructions 1510 embodying any one or more of the methods or functions described herein. Instructions 1510 may also reside wholly or partially within memory 1514, storage cells 1516, at least one of processor 1504 (e.g., within the processor's cache memory), or any suitable combination thereof during execution by machine 1500. Thus, the memory of memory 1514, storage cells 1516, and processor 1504 are examples of machine-readable media.

[0116] I / O component 1518 may include a wide variety of components for receiving input, providing output, generating output, transmitting information, exchanging information, capturing measurement results, etc. The specific I / O component 1518 included in a particular machine 1500 will depend on the type of machine. For example, a portable machine such as a mobile phone may include a touch input device or other such input mechanism, while a headless server machine will likely not include such a touch input device. It should be understood that I / O component 1518 may include... Figure 15 Many other components are not shown. For the purpose of simplifying the following discussion only, the I / O components 1518 are grouped according to function, and this grouping is by no means limiting. In various embodiments, the I / O components 1518 may include output components 1526 and input components 1528. Output components 1526 may include visual components (e.g., displays such as plasma display panels (PDPs), light-emitting diode (LED) displays, liquid crystal displays (LCDs), projectors, or cathode ray tubes (CRTs)), auditory components (e.g., speakers), haptic components (e.g., vibration motors, resistance mechanisms), other signal generators, etc. Input components 1528 may include alphanumeric input components (e.g., keyboards, touchscreens configured to receive alphanumeric input, photoelectric keyboards, or other alphanumeric input components), point-based input components (e.g., mice, touchpads, trackballs, joysticks, motion sensors, or other pointing instruments), haptic input components (e.g., physical buttons, touchscreens or other haptic input components that provide the position or force of a touch or touch gesture), audio input components (e.g., microphones), etc.

[0117] In other embodiments, I / O component 1518 may include biometric identification component 1530, motion component 1534, environmental component 1536, or positioning component 1538, as well as a variety of other components. For example, biometric identification component 1530 may include components for detecting expressions (e.g., hand expressions, facial expressions, voice expressions, body posture, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweating, or brain waves), and identifying a person (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or EEG-based recognition). Motion component 1534 may include accelerometer components (e.g., accelerometers), gravity sensor components, rotation sensor components (e.g., gyroscopes), etc. Environmental component 1536 may include, for example, a lighting sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers that detect ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), a sound sensor component (e.g., one or more microphones that detect background noise), a proximity sensor component (e.g., an infrared sensor that detects nearby objects), a gas sensor (e.g., a gas detection sensor for detecting the concentration of hazardous gases to ensure safety or for measuring pollutants in the atmosphere), or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment. Positioning component 1538 may include a position sensor component (e.g., a Global Positioning System (GPS) receiver component), an altitude sensor component (e.g., an altimeter or barometer from which altitude can be derived), an orientation sensor component (e.g., a magnetometer), etc.

[0118] A wide variety of technologies can be used to implement communication. I / O component 1518 may include communication component 1540, which is operable to couple machine 1500 to network 1532 or device 1520 via coupler 1522 and coupler 1524, respectively. For example, communication component 1540 may include a network interface component or other suitable device that interfaces with network 1532. In other examples, communication component 1540 may include wired communication components, wireless communication components, cellular communication components, near field communication (NFC) components, etc. Components (e.g., (low power consumption) Components and other communication components that provide communication via other means. Device 1520 can be any of another machine or a wide variety of peripheral devices (e.g., peripheral devices coupled via Universal Serial Bus (USB)).

[0119] Furthermore, the communication component 1540 can detect identifiers or include components operable to detect identifiers. For example, the communication component 1540 may include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting one-dimensional barcodes, multi-dimensional barcodes, and other optical codes, such as Universal Product Code (UPC) barcodes for one-dimensional barcodes, and QR codes, Aztec codes, data matrices, dataglyphs, MaxiCodes, PDF417, Ultra Codes, and UCCRSS-2D barcodes) or an acoustic detection component (e.g., a microphone for identifying audio signals of tags). Additionally, various information can be derived via the communication component 1540, such as location via Internet Protocol (IP) geolocation, etc. Location can be determined by signal triangulation or by detecting NFC beacon signals that indicate a specific location.

[0120] Throughout this specification, multiple instances can implement components, operations, or structures described as single instances. Although the various operations of one or more methods are shown and described as independent operations, one or more of the various operations can be performed simultaneously, and the operations are not required to be performed in the order shown. Structures and functions presented as separate components in the example configurations can be implemented as combined structures or components. Similarly, structures and functions presented as single components can be implemented as separate components. These and other variations, modifications, additions, and improvements fall within the scope of the subject matter herein.

[0121] Although an overview of the subject matter of the invention has been described with reference to specific embodiments, various modifications and changes can be made to these embodiments without departing from the broad scope of the embodiments described herein.

[0122] The embodiments illustrated herein have been described in sufficient detail to enable those skilled in the art to practice the disclosed teachings. Other embodiments may be used with and derived from the embodiments illustrated herein, allowing for structural and logical substitutions and changes without departing from the scope of this disclosure. Therefore, the specific embodiments are not intended to be limiting, and the scope of the various embodiments is defined only by the appended claims together with the full scope of their equivalents.

[0123] As used herein, the term “or” can be interpreted as inclusive or exclusive. The terms “a” or “one” should be interpreted as meaning “at least one,” “one or more,” etc. The use of words and phrases such as “one or more,” “at least,” “but not limited to,” or other similar phrases should not be interpreted as implying a narrower intent or requirement where such extended phrases may not exist.

[0124] The boundaries between various resources, operations, components, modules, engines, and data storage are somewhat arbitrary, and specific operations are shown within the context of a particular illustrative configuration. Other allocations of functionality are contemplated and may fall within the scope of various embodiments of this disclosure. Typically, structures and functions presented as separate resources in the example configuration can be implemented as combined structures or resources. Similarly, structures and functions presented as individual resources can be implemented as separate resources. These and other variations, modifications, additions, and improvements fall within the scope of embodiments of this disclosure as expressed by the appended claims. Therefore, the specification and drawings should be considered illustrative rather than restrictive.

[0125] The above description includes systems, methods, apparatuses, instructions, and computer media (e.g., computer machine program products) embodying illustrative embodiments of the present disclosure. In this description, numerous specific details are set forth for illustrative purposes to provide an understanding of various embodiments of the subject matter of the invention. However, it will be apparent to those skilled in the art that embodiments of the subject matter of the invention can be practiced without these specific details. Generally, known examples of instructions, protocols, structures, and techniques are not necessarily shown in detail.

[0126] Glossary

[0127] In this context, "augmented reality object" can refer to a virtual object (e.g., a two-dimensional or three-dimensional virtual object) that can be presented in a view generated by a client device in a real-world environment (e.g., a view presented on the display of a mobile client device), wherein the virtual object can interact with or enhance real-world physical objects in the real-world environment presented in the view. For example, using a smartphone's camera, a user can view the real-world environment around them through the smartphone's display, and the smartphone can enhance the view by displaying (e.g., overlaying) one or more virtual objects (e.g., three-dimensional virtual objects) in the view in conjunction with one or more specific real-world physical objects in the real-world environment. For example, augmented reality objects can be combined with on-site (e.g., real-time or near-real-time) camera feeds such that when the augmented reality object is presented, it appears to be located in the on-site three-dimensional environment (e.g., the augmented reality object appears to occupy a consistent three-dimensional volume and dynamically changes its aspect in response to camera movement in a manner similar to that of an AR object being a real-world physical object). In addition to visual information, client devices can convey other sensory information associated with a specific augmented reality object to the user, such as auditory information (e.g., music) and tactile information.

[0128] In this context, "mixed reality" can refer to the merging of a real-world environment and a virtual-world environment (which may include one or more augmented reality objects) to generate a new visualization via a client device. The new visualization can augment one or more real-world physical objects within the real-world environment. This new visualization can create a new mixed reality environment in which real-world physical objects and augmented reality objects can coexist and interact with each other in real time. Furthermore, in mixed reality, users can use client devices to interact with augmented reality objects in real time.

[0129] In this context, "client device" can refer to any machine that interfaces with a communication network to obtain resources from one or more server systems or other client devices. Client devices can be, but are not limited to, mobile phones, desktop computers, laptop computers, portable digital assistants (PDAs), smartphones, tablet computers, ultrabooks, netbooks, multiprocessor systems, microprocessor-based or programmable consumer electronics, game consoles, set-top boxes, or any other communication device that a user can use to access the network.

[0130] In this context, "communication network" can refer to one or more parts of a network, which can be an ad hoc network, intranet, extranet, virtual private network (VPN), local area network (LAN), wireless LAN (WLAN), wide area network (WAN), wireless WAN (WWAN), metropolitan area network (MAN), the Internet, a part of the Internet, a part of the Public Switched Telephone Network (PSTN), a POTS (Plain Old-Style Telephone Service) network, a cellular telephone network, a wireless network, etc. A network, another type of network, or a combination of two or more such networks. For example, a network or part of a network may include a wireless network or a cellular network, and the coupling may be a Code Division Multiple Access (CDMA) connection, a Global System for Mobile Communications (GSM) connection, or other types of cellular or wireless coupling. In this example, the coupling can implement any of a variety of data transmission technologies, such as Single Carrier Radio Transmission (1xRTT), Evolved Data Optimization (EVDO), General Packet Radio Service (GPRS), Enhanced Data Rate Evolution of GSM (EDGE), including 3G networks, fourth-generation wireless (4G) networks, Universal Mobile Telecommunications System (UMTS), High-Speed ​​Packet Access (HSPA), Global Microwave Access Interoperability (WiMAX), the 3rd Generation Partnership Project (3GPP) for Long Term Evolution (LTE) standards, other data transmission technologies defined by various standards setting organizations, other telematics protocols, or other data transmission technologies.

[0131] In this context, "transient" can describe an item that is accessible for a limited duration. Transient items can be AR objects, text, images, videos, etc. The access duration for a transient item can be set by the item owner or initiator (e.g., a message sender or a user registering an AR object). Alternatively, the access duration can be a default setting or a setting specified by the accessing user (e.g., a recipient or a user attempting to access a registered AR object). Regardless of the setting technique, transient items are temporary.

[0132] In this context, "machine-readable medium" can refer to a component, device, or other tangible medium capable of temporarily or permanently storing instructions and data, and may include, but is not limited to, random access memory (RAM), read-only memory (ROM), buffer memory, flash memory, optical media, magnetic media, cache memory, other types of storage devices (e.g., erasable programmable read-only memory (EEPROM)), and / or any suitable combination of the above. The term "machine-readable medium" should be considered to include a single medium or multiple media capable of storing instructions (e.g., a centralized or distributed database or associated cache memory and server). The term "machine-readable medium" should also be considered to include any medium or combination of media capable of storing machine-executable instructions (e.g., code) such that, when executed by one or more processors of the machine, the instructions cause the machine to perform any or more methods described herein. Therefore, "machine-readable medium" refers to a single storage device or apparatus and a "cloud-based" storage system or storage network comprising multiple storage devices or apparatuses. The term "machine-readable medium" does not include the signal itself.

[0133] In this context, a “component” can refer to a device, physical entity, or logic having boundaries defined by function or subroutine calls, branch points, application programming interfaces (APIs), or other technologies provided for partitioning or modularizing specific processing or control functions. Components can be combined with other components via their interfaces to perform machine processing. A component can be part of a packaged functional hardware unit designed for use with other components, and a program that typically performs a specific function related to that function. Components can constitute software components (e.g., code embedded on a machine-readable medium) or hardware components. A “hardware component” is a tangible unit capable of performing certain operations and can be configured or arranged in some physical manner. In various implementations, one or more computer systems (e.g., standalone computer systems, client computer systems, or server computer systems) or one or more hardware components (e.g., processors or processor groups) of a computer system can be configured by software (e.g., an application or application portion) to operate to perform certain operations as described herein. Hardware components can also be implemented mechanically, electronically, or any suitable combination thereof. For example, a hardware component can include a dedicated circuit system or logic permanently configured to perform certain operations. Hardware components can be dedicated processors, such as field-programmable gate arrays (FPGAs) or application-specific integrated circuits (ASICs). Hardware components can also include programmable logic or circuitry systems that are temporarily configured by software to perform certain operations. For example, a hardware component may include software executed by a general-purpose processor or other programmable processor. Once configured by such software, the hardware component becomes a particular machine (or a specific component of a machine) uniquely tailored to perform the configured function and is no longer a general-purpose processor. It should be understood that the decision to implement a hardware component mechanically, in a dedicated and permanently configured circuitry system, or in a temporarily configured (e.g., software-configured) circuitry system can be driven by cost and time considerations. Therefore, the phrase “hardware component” (or “hardware-implemented component”) should be understood to encompass tangible entities, i.e., entities physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a certain way or perform certain operations described herein. Given the implementation of hardware components being temporarily configured (e.g., programmed), each of the hardware components need not be configured or instantiated at any given time. For example, in cases where the hardware components include a general-purpose processor that is configured as a dedicated processor via software, the general-purpose processor can be configured as a different dedicated processor (e.g., including different hardware components) at different times. The software accordingly configures a particular processor or processor to constitute a specific hardware component at one time and a different hardware component at different times. Hardware components can provide information to and receive information from other hardware components.Therefore, the described hardware components can be considered as communicationally coupled. In the presence of multiple hardware components, communication can be achieved through signal transmission (e.g., via appropriate circuitry and buses) between or among two or more hardware components. In embodiments where multiple hardware components are configured or instantiated at different times, such communication between hardware components can be achieved, for example, by storing information in a memory structure accessible to the multiple hardware components and retrieving information from that memory structure. For example, a hardware component can perform an operation and store the output of that operation in a communicationally coupled memory device. Another hardware component can then access the memory device at a subsequent time to retrieve and process the stored output. Hardware components can also initiate communication with input or output devices and can operate on resources (e.g., collections of information). The various operations of the example methods described herein can be performed at least in part by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors can constitute processor-implemented components that operate to perform one or more operations or functions described herein. As used herein, "processor-implemented component" refers to a hardware component implemented using one or more processors. Similarly, the methods described herein can be implemented at least in part by processors, where one or more specific processors are examples of components. For example, at least some operations of the methods can be performed by one or more processors or processor-implemented components. Furthermore, the one or more processors can also be operated to support the execution of related operations in a “cloud computing” environment or to operate as “Software as a Service” (SaaS). For example, at least some operations can be performed by a group of computers (as an example of a machine including processors), where these operations are accessible via a network (e.g., the Internet) and via one or more suitable interfaces (e.g., application programming interfaces (APIs)). The execution of certain operations can be distributed among processors, residing not only within a single machine but also deployed across multiple machines. In some embodiments, the processor or processor-implemented component can be located in a single geographic location (e.g., in a home environment, office environment, or server farm). In other embodiments, the processor or processor-implemented component can be distributed across multiple geographic locations.

[0134] In this context, "processor" can refer to any circuit or virtual circuit (physical circuitry simulated by logic executed on an actual processor) that manipulates data values ​​according to control signals (e.g., "commands," "opcodes," "machine codes," etc.) and generates corresponding output signals applied to operate the machine. For example, a processor can be a Central Processing Unit (CPU), a Reduced Instruction Set Computing (RISC) processor, a Complex Instruction Set Computing (CISC) processor, a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), a Radio Frequency Integrated Circuit (RFIC), or any combination thereof. A processor can also be a multi-core processor having two or more independent processors (sometimes referred to as "cores") capable of executing instructions simultaneously.

[0135] In this context, a "timestamp" can refer to a sequence of characters or encoded information that identifies when an event occurred—for example, given a date and time of day, sometimes accurate to a fraction of a second.

Claims

1. A method comprising: Receive a query for one or more augmented reality objects from a first client device associated with a first user, the query including a current set of coordinates corresponding to the location of the first client device on a map; as well as In response to the query: The intermediate set of augmented reality objects is determined based on the query, and the intermediate set of augmented reality objects is determined from multiple registered augmented reality objects registered in the augmented reality object registry. A set of sorters is determined for the query, and at least one sorter in the set is used to filter the determined set of augmented reality objects; The final set of augmented reality objects is generated by applying the set of sorters to the intermediate set of augmented reality objects; as well as Send the query result to the first client device, the query result including the result data of the final augmented reality object set. The at least one sorter is used to further filter the intermediate augmented reality object set based on a priority set of the intermediate augmented reality object set, and The priority of the at least one augmented reality object is determined based on bidding data associated with it.

2. The method according to claim 1, wherein, At least one sorter in the set of sorters is used to apply a sorting order to the determined set of augmented reality objects.

3. The method according to claim 1, wherein, At least one sorter in the set of sorters is determined based on the query by a first user associated with the first client device.

4. The method according to claim 3, wherein, Based on the first user's selection of a sorter, at least one sorter from the set of sorters is determined for the query.

5. The method according to claim 1, wherein, Based on the current coordinate set, determine at least one sorter from the sorter set for the query.

6. The method according to claim 1, wherein, Based on the attributes of the first client device, at least one sorter in the sorter set is determined for the query.

7. The method according to claim 1, wherein, At least one sorter in the sorter set is determined for the query based on at least one of the date set or time set.

8. The method according to claim 1, further comprising: The system requests priority information of the intermediate set of augmented reality objects from the bidding system, which facilitates users to bid for the priority of augmented reality objects. as well as Priority data is received from the bidding system, the priority data describing at least one subset of priorities in the priority set of the intermediate augmented reality object set.

9. The method according to claim 1, wherein, The method further includes determining the set of sorters from a plurality of sorters registered in a sorter registry table: Receive a request from a second client device associated with the second user to register a given sorter on the sorter registration table; Based on the permission data, determine whether the second user has permission to register the given sorter on the sorter registration form; and Based on determining whether the second user has permission, the given sorter is registered on the sorter registration table.

10. A non-transitory machine-readable medium storing instructions that, when executed by one or more computer processors, cause the one or more computer processors to perform operations, the operations including: Receive client queries for one or more augmented reality objects from a first client device associated with a first user, the client queries including a current set of coordinates corresponding to the location of the first client device on a map; as well as In response to the client query: A set of sorters is determined for the client query, at least one sorter in the set of sorters includes filtering parameters for filtering the set of augmented reality objects; Generate a sorter-based query based on the client query and the sorter set; The given set of augmented reality objects is determined based on the sorter-based query, and the given set of augmented reality objects is determined from multiple registered augmented reality objects registered in the augmented reality object registry. as well as Send query results to the first client device, the query results including result data of the given augmented reality object set. The filtering parameters are used to: filter based on the priority associated with the augmented reality object, and The priority of the augmented reality object is determined based on bidding data associated with it.

11. The non-transitory machine-readable medium according to claim 10, wherein, At least one sorter in the set of sorters includes a sorting order parameter for sorting the set of augmented reality objects.

12. The non-transitory machine-readable medium according to claim 10, wherein, At least one sorter in the sorter set is determined based on a query made by a first user associated with the first client device against the client.

13. The non-transitory machine-readable medium according to claim 10, wherein, Based on the first user's selection of a sorter, at least one sorter in the sorter set is determined for the client query.

14. The non-transitory machine-readable medium according to claim 10, wherein, At least one sorter in the sorter set is determined based on the current coordinate set in the client query.

15. The non-transitory machine-readable medium according to claim 10, wherein, Based on the attributes of the first client device, a query is performed on the client to determine at least one sorter in the set of sorters.

16. The non-transitory machine-readable medium according to claim 10, wherein, At least one sorter in the sorter set is determined based on a query against the client using at least one of the date set or time set.

17. A system comprising: One or more processors; as well as One or more machine-readable media storing instructions that, when executed by the one or more processors, cause the system to perform operations, the operations including: Receive a query from a client device for one or more augmented reality objects, the query including a current set of coordinates corresponding to the client device's location on a map; and In response to the query: The intermediate set of augmented reality objects is determined based on the query, and the intermediate set of augmented reality objects is determined from multiple registered augmented reality objects registered in the augmented reality object registry. A set of sorters is determined for the query, and at least one sorter in the set is used to filter the determined set of augmented reality objects; The final set of augmented reality objects is generated by applying the set of sorters to the intermediate set of augmented reality objects; and The query results are sent to the client device, and the query results include the result data of the final augmented reality object set. The at least one sorter is used to further filter the intermediate augmented reality object set based on a priority set of the intermediate augmented reality object set, and The priority of the at least one augmented reality object is determined based on bidding data associated with it.

Images