Selectively modifying GUI

Abstract

Aspects of the present disclosure relate to systems and methods for navigating images and AR experiences. The system and method present, by a messaging application, a first plurality of options associated with previously captured content items and a second plurality of options associated with an AR experience in a scrollable area above a live video feed being displayed in a GUI including a viewfinder. The system and method scrolls the first plurality of options and the second plurality of options of the scrollable region together to focus one of the first plurality of options or one of the second plurality of options. In response, the system and method modifies the configuration of the GUI in a first manner or modifies the configuration of the GUI in a second manner.

Description

[0001] Priority Claim

[0002] This application claims the benefit of priority of U.S. Provisional Application No. 63 / 425,407, filed on Nov. 15, 2022, and U.S. Patent Application No. 18 / 498,849, filed on Oct. 31, 2023, each of which is incorporated herein by reference in its entirety. Technical Field

[0003] The present disclosure generally relates to navigating through menu items. Background Art

[0004] Social networking sites are among the most popular accessed sites on the Internet, if not the most popular. Social networks provide a wealth of information about users and their friends. Such information includes a user's current status and the user's interests. Social networking sites provide users with access to previously captured images and videos, as well as the ability to initiate one or more augmented reality (AR) or virtual reality (VR) experiences. Brief Description of the Drawings

[0005] In the drawings, which are not necessarily to scale, like reference numerals may describe like components in different views. To easily identify the discussion of any particular element or action, one or more of the most significant digits in the reference numeral refers to the figure number in which the element was first introduced. Some embodiments are shown by way of example and not limitation in the figures of the drawings, in which:

[0006] Figure 1 is a graphical representation of a networked environment in which the present disclosure may be deployed, according to some examples.

[0007] Figure 2 is a graphical representation of a messaging client application, according to some examples.

[0008] Figure 3 is a graphical representation of a data structure as maintained in a database, according to some examples.

[0009] Figure 4 is a graphical representation of a message, according to some examples.

[0010] Figure 5A and Figure 5B are illustrations of example operations performed by a content and AR experience access system, according to some examples.

[0011] Figure 6A 、 Figure 6B 、 Figure 6C 、 Figure 6D 、Figure 7 , Figure 8A and Figure 8B are illustrative inputs and outputs according to the content of the example embodiment and the description of the AR experience access system.

[0012] Figure 9 is a block diagram showing a representative software architecture that can be used in conjunction with various hardware architectures according to the example embodiment.

[0013] Figure 10 is a block diagram showing components of a machine capable of reading instructions from a machine-readable medium (e.g., a machine-readable storage medium) and performing any one or more of the methods discussed herein. Detailed Description

[0014] The following description includes systems, methods, techniques, instruction sequences, and computer program products embodying illustrative embodiments of the present disclosure. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide an understanding of the various embodiments. However, it will be apparent to one of ordinary skill in the art that the embodiments may be practiced without these specific details. Generally, well-known instruction instances, protocols, structures, and techniques are not necessarily shown in detail.

[0015] Typical systems allow users to share various content items with other users and also provide the ability to apply various AR / VR experiences to the content items. To this end, typical systems provide a search interface or a graphical user interface (GUI) specific to a previously captured content item for the user to browse and select a given content item of interest. To find a given content item of interest, the user typically needs to browse through multiple pages of information, which can take a significant amount of time. Once a given content item is selected, typical systems allow the user to launch a separate editing interface to modify the given content item, such as by applying one or more AR / VR experiences to the given content item. Finding the desired AR / VR experience can also take a significant amount of time because the user must again browse through multiple pages of information to find the AR / VR experience to apply.

[0016] When the user is satisfied with the selected content item and the modifications applied to the content item, the user can access a sharing GUI to select one or more recipients of the modified content item. The need to manually browse through multiple pages of information makes such systems less appealing and less intuitive to use, which increases the overall complexity of the system and reduces its overall efficiency. On resource-constrained devices such as mobile devices, navigating through large numbers of information pages and screens to send a specific content item of interest consumes a significant amount of resources, processing power, and battery, which reduces the overall functionality and efficiency of operating such a device.

[0017] The disclosed embodiments improve the efficiency of using an electronic device by enabling a user to quickly and easily navigate a GUI that includes a single scrollable area that includes both previously captured content items and AR / VR experiences. From the same GUI, the user can select a content item of interest, a modification to apply to the content item of interest, and one or more recipients to which the modified content item of interest is to be sent. This speeds up the way content is shared among users and enables such a system to operate more efficiently in mobile settings and environments, particularly by reducing the number of information pages the user needs to search through and access to share a particular content item. This provides such users with a unique way to select and create content items to share in unique environments and situations, which increases the overall appeal and fun of the system.

[0018] The disclosed techniques display a live video feed received from a camera device of a client device in a viewfinder via a messaging application. The disclosed techniques present a first plurality of options associated with previously captured content items and a second plurality of options associated with an AR experience in a scrollable area above the live video feed being displayed in the viewfinder via the messaging application. The disclosed techniques receive an input associated with scrolling the scrollable area and, in response to the input, scroll the first plurality of options together with the second plurality of options to allow the user to select an item of interest.

[0019] In some examples, the disclosed techniques present a first plurality of options associated with previously captured content items and a second plurality of options associated with an AR experience in a scrollable area above a live video feed being displayed in a GUI that includes a viewfinder. The disclosed techniques scroll the first plurality of options of the scrollable area together with the second plurality of options such that one of the first plurality of options or one of the second plurality of options is in focus. In response to scrolling the first plurality of options of the scrollable area together with the second plurality of options, the disclosed techniques: in response to determining that one of the first plurality of options is currently in focus, modify the configuration of the GUI in a first manner. In response to scrolling the first plurality of options of the scrollable area together with the second plurality of options, the disclosed techniques: in response to determining that one of the second plurality of options is currently in focus, modify the configuration of the GUI in a second manner.

[0020] In this way, the user does not need to browse through multiple different information pages to share and create content items. This improves the overall efficiency of the computing device and reduces the complexity of using the messaging application.

[0021] Networked computing environment

[0022] Figure 1FIG. 0 is a block diagram illustrating an example messaging system 100 for exchanging data (e.g., messages and associated content) over a network. The messaging system 100 includes multiple instances of client devices 102, each instance hosting a number of applications, including a messaging client 104 and other external applications 109 (e.g., third-party applications). Each messaging client 104 is communicatively coupled via a network 112 (e.g., the Internet) to other instances of the messaging client 104 (e.g., hosted on corresponding other client devices 102), a messaging server system 108, and an external application server 110. The messaging client 104 may also communicate with locally hosted third-party applications (also referred to as “external applications” and “external applications”) 109 using an application programming interface (API).

[0023] The client device 102 may operate as a stand-alone device or may be coupled (e.g., networked) to other machines. In a networked deployment, the client device 102 may operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The client device 102 may include, but is not limited to, a server computer, a client computer, a personal computer (PC), a tablet, a laptop, a netbook, a set-top box (STB), a personal digital assistant (PDA), an entertainment media system, a cellular phone, a smartphone, a mobile device, a wearable device (e.g., a smartwatch), a smart home device (e.g., a smart appliance), other smart devices, a web device, a network router, a network switch, a network bridge, or any machine capable of performing the disclosed operations. Additionally, although only a single client device 102 is shown, the term “client device” should also be considered to include a collection of machines that individually or jointly perform the disclosed operations.

[0024] In some examples, the client device 102 may include AR glasses or an AR headset, where virtual content is displayed within the lenses of the glasses while the user views the real-world environment through the lenses. For example, an image may be presented on a transparent display that allows the user to view both the content presented on the display and real-world objects simultaneously.

[0025] The messaging client 104 is capable of communicating with other messaging clients 104 and the messaging server system 108 via the network 112 and exchanging data. The data exchanged between the messaging clients 104 and between the messaging client 104 and the messaging server system 108 includes functionality (e.g., commands to invoke functions) and payload data (e.g., text, audio, video, or other multimedia data).

[0026] The messaging server system 108 provides server-side functionality to a particular messaging client 104 via a network 112. Although certain functions of the messaging system 100 are described herein as being performed by the messaging client 104 or by the messaging server system 108, the location of certain functions within the messaging client 104 or within the messaging server system 108 can be a design choice. For example, it may be technically preferable to initially deploy certain technologies and functions within the messaging server system 108, but later migrate the technologies and functions to the messaging client 104 where the client device 102 has sufficient processing power.

[0027] The messaging server system 108 supports various services and operations provided to the messaging client 104. Such operations include sending data to the messaging client 104, receiving data from the messaging client 104, and processing data generated by the messaging client 104. As an example, the data can include message content, client device information, geographical location information, media enhancements and overlays, message content persistence conditions, social network information, and live event information. Data exchange within the messaging system 100 is invoked and controlled through functions available via the user interface (UI) of the messaging client 104.

[0028] Turning now specifically to the messaging server system 108, an API server 116 is coupled to an application server 114 and provides a programming interface to the application server 114. The application server 114 is communicatively coupled to a database server 120 that facilitates access to a database 126 that stores data associated with messages processed by the application server 114. Similarly, a web server 128 is coupled to the application server 114 and provides a web-based interface to the application server 114. To this end, the web server 128 processes incoming network requests via the Hypertext Transfer Protocol (HTTP) and several other related protocols.

[0029] The API server 116 receives and sends message data (e.g., commands and message payloads) between the client device 102 and the application server 114. Specifically, the API server 116 provides a set of interfaces (e.g., routines and protocols) that can be invoked or queried by the messaging client 104 to invoke the functions of the application server 114. The API server 116 exposes various functions supported by the application server 114, including: account registration; login functionality; sending a message from a particular messaging client 104 to another messaging client 104 via the application server 114; sending a media file (e.g., an image or video) from the messaging client 104 to the messaging server 118 for possible access by another messaging client 104; setting a collection of media data (e.g., a story); retrieving a list of friends of the user of the client device 102; retrieving such a collection; retrieving messages and content; adding and deleting entities (e.g., friends) in an entity graph (e.g., a social graph); locating friends within the social graph; and opening application events (e.g., related to the messaging client 104).

[0030] The application server 114 hosts several server applications and subsystems, including, for example, the messaging server 118, the image processing server 122, and the social network server 124. The messaging server 118 implements several message processing techniques and functions, particularly those related to the aggregation and other processing of the content (e.g., text and multimedia content) included in messages received from multiple instances of the messaging client 104. As will be described in further detail, text and media content from multiple sources can be aggregated into collections of content (e.g., called stories or galleries). These collections are then made available to the messaging client 104. Given the hardware requirements for other processor- and memory-intensive data processing, such processing can also be performed by the messaging server 118 on the server side.

[0031] The application server 114 also includes an image processing server 122 that is dedicated to performing various image processing operations, typically with respect to images or videos within the payload of messages sent from or received at the messaging server 118.

[0032] The image processing server 122 is used to implement the enhancement system 208( Figure 2The scanning function (shown in). The scanning function includes activating and providing one or more augmented reality experiences on the client device 102 when an image is captured by the client device 102. Specifically, the messaging application 104 on the client device 102 can be used to activate the imaging device. The imaging device displays one or more real-time images or videos along with one or more icons or identifiers of one or more augmented reality experiences to the user. The user can select a given identifier among the identifiers to initiate the corresponding augmented reality experience or perform the desired image modification.

[0033] The social network server 124 supports various social networking functions and services and makes these functions and services available to the messaging server 118. To this end, the social network server 124 maintains and accesses the entity graph 308 within the database 126 (as Figure 3 shown). Examples of functions and services supported by the social network server 124 include identifying other users in the messaging system 100 who are related to or being "followed" by a particular user, and also include identifying the interests of a particular user and other entities.

[0034] Returning to the messaging client 104, the features and functions of external resources (e.g., third-party applications 109 or applets) are available to the user via the interface of the messaging client 104. The messaging client 104 receives a user selection of an option for initiating or accessing the features of an external resource such as an external application 109 (e.g., a third-party resource). The external resource can be a third-party application (external application 109) installed on the client device 102 (e.g., a "native application"), or a scaled-down version of a third-party application hosted on the client device 102 or away from the client device 102 (e.g., on an external resource or application server 110) (e.g., an "applet"). The scaled-down version of the third-party application includes a subset of the features and functions of the third-party application (e.g., the full-scale native version of a third-party stand-alone application) and is implemented using a markup language document. In one example, the scaled-down version of the third-party application (e.g., an "applet") is a web-based markup language version of the third-party application and is embedded within the messaging client 104. In addition to using a markup language document (e.g., an *ml file), the applet can include a scripting language (e.g., a.*js file or a.json file) and a style sheet (e.g., a.*ss file).

[0035] In response to receiving a user selection of an option for a feature to initiate or access an external resource (e.g., external application 109), the messaging client 104 determines whether the selected external resource is a web-based external resource or a locally installed external application. In some cases, the external application 109 locally installed on the client device 102 can be initiated independently of and separately from the messaging client 104, for example, by selecting an icon corresponding to the external application 109 on the home screen of the client device 102. A scaled-down version of such an external application can be initiated or accessed via the messaging client 104, and in some examples, any part of the scaled-down external application cannot (or a limited part can) be accessed outside of the messaging client 104. The scaled-down external application can be initiated by receiving a markup language document associated with the scaled-down external application from the external application server 110 via the messaging client 104 and processing such a document.

[0036] In response to determining that the external resource is the locally installed external application 109, the messaging client 104 instructs the client device 102 to initiate the external application 109 by executing locally stored code corresponding to the external application 109. In response to determining that the external resource is a web-based resource, the messaging client 104 communicates with the external application server 110 to obtain a markup language document corresponding to the selected resource. The messaging client 104 then processes the obtained markup language document to present the web-based external resource within the user interface of the messaging client 104.

[0037] The messaging client 104 can notify a user of the client device 102 or other users associated with such a user (e.g., “friends”) of activities occurring in one or more external resources. For example, the messaging client 104 can provide notifications to participants in a conversation (e.g., a chat session) in the messaging client 104 regarding the current or recent use of an external resource by one or more members of a user group. One or more users can be invited to join the external resource of the activity or to launch (in a group of friends) an external resource that has been recently used but is currently inactive. The external resource can provide the ability for participants in the conversation (each using a corresponding messaging client 104) to share items, conditions, statuses, or locations within the external resource with one or more members of the user group in the chat session. The shared item can be an interactive chat card that chat members can use to interact, e.g., to launch the corresponding external resource, view specific information within the external resource, or take the chat members to a specific location or status within the external resource. Within a given external resource, a response message can be sent to a user on the messaging client 104. The external resource can selectively include different media items in the response based on the current context of the external resource.

[0038] The messaging client 104 can present a list of available external resources (e.g., third-party or external applications 109 or applets) to the user to launch or access a given external resource. The list can be presented in a context-sensitive menu. For example, the icons representing different external applications in the external application 109 (or applet) can vary based on how the user launches the menu (e.g., from a conversation interface or from a non-conversation interface).

[0039] System Architecture

[0040] Figure 2 is a block diagram showing additional details regarding the messaging system 100 according to some examples. Specifically, the messaging system 100 is shown to include a messaging client 104 and an application server 114. The messaging system 100 includes several subsystems that are supported on the client side by the messaging client 104 and on the server side by the application server 114. These subsystems include, for example, an ephemeral timer system 202, a collection management system 204, an enhancement system 208, a map system 210, a game system 212, and an external resource system 220.

[0041] The short-lived timer system 202 is responsible for implementing temporary or time-limited access to content by the messaging client 104 and the messaging server 118. The short-lived timer system 202 includes a number of timers that selectively enable access (e.g., for presentation and display) to messages and associated content via the messaging client 104 based on the duration and display parameters associated with a message or set of messages (e.g., a story). Additional details regarding the operation of the short-lived timer system 202 are provided below.

[0042] The collection management system 204 is responsible for managing collections or sets of media (e.g., collections of text, images, video, and audio data). Collections of content (e.g., messages, including images, video, text, and audio) can be organized into "event libraries" or "event stories". Such collections can be made available for a specified period of time (e.g., the duration of the event related to the content). For example, content related to a concert can be made available as a "story" for the duration of that concert. The collection management system 204 can also be responsible for posting an icon notifying the user interface of the messaging client 104 of the existence of a particular collection.

[0043] The collection management system 204 also includes a curation interface 206 that enables a collection manager to manage and curate a particular content collection. For example, the curation interface 206 enables an event organizer to curate a collection of content related to a particular event (e.g., delete inappropriate content or redundant messages). Additionally, the collection management system 204 employs machine vision (or image recognition technology) and content rules to automatically curate content collections. In some examples, compensation may be paid to users for including user-generated content in a collection. In such cases, the collection management system 204 operates to automatically pay such users for the use of their content.

[0044] The enhancement system 208 provides various functions that enable a user to enhance (e.g., annotate or otherwise modify or edit) media content associated with a message. For example, the enhancement system 208 provides functions related to generating and publishing a media overlay for a message to be processed by the messaging system 100. The enhancement system 208 operably provides a media overlay or enhancement (e.g., an image filter) to the messaging client 104 based on the geographical location of the client device 102. In another example, the enhancement system 208 operably provides a media overlay to the messaging client 104 based on other information such as the social network information of a user of the client device 102. The media overlay may include audio and visual content and visual effects. Examples of audio and visual content include pictures, text, logos, animations, and sound effects. Examples of visual effects include color overlays. The audio and visual content or visual effects may be applied to a media content item (e.g., a photo) at the client device 102. For example, the media overlay may include text, graphic elements, or images that may be superimposed over a photo taken by the client device 102. In another example, the media overlay includes a location identification overlay (e.g., Venice Beach), a name of a live event, or a business name overlay (e.g., Beach Café). In another example, the enhancement system 208 uses the geographical location of the client device 102 to identify a media overlay that includes the business name at the geographical location of the client device 102. The media overlay may include other markers associated with the business. The media overlay may be stored in the database 126 and accessed via the database server 120.

[0045] In some examples, the enhancement system 208 provides a user-based publishing platform that enables a user to select a geographical location on a map and upload content associated with the selected geographical location. The user may also specify the circumstances under which a particular media overlay should be provided to other users. The enhancement system 208 generates a media overlay that includes the uploaded content and associates the uploaded content with the selected geographical location.

[0046] In other examples, the augmentation system 208 provides a merchant-based publishing platform that enables a merchant to select a specific media overlay associated with a geographical location via a bidding process. For example, the augmentation system 208 associates the media overlay of the highest bidding merchant with the corresponding geographical location for a predefined amount of time. The augmentation system 208 communicates with the image processing server 122 to obtain an augmented reality experience and presents an identifier of such an experience in one or more user interfaces (e.g., as an icon on a live image or video, or as a thumbnail or icon in an interface dedicated to presenting the augmented reality experience identifier). Once an augmented reality experience is selected, one or more images, videos, or augmented reality graphic elements are retrieved and presented as an overlay on the image or video captured by the client device 102. In some cases, the camera device is switched to a front view (e.g., the front camera device of the client device 102 is activated in response to the activation of a specific augmented reality experience), and the image from the front camera device of the client device 102 starts to be displayed on the client device 102 instead of the image from the rear camera device of the client device 102. One or more images, videos, or augmented reality graphic elements are retrieved and presented as an overlay on the image captured and displayed by the front camera device of the client device 102.

[0047] In other examples, the augmentation system 208 is capable of communicating and exchanging data with another augmentation system 208 on another client device 102 and with a server via the network 112. The data exchanged may include: a session identifier identifying a shared AR session; a transformation between the first client device 102 and the second client device 102 (e.g., multiple client devices 102, including the first device and the second device), which is used to align the shared AR session to a common origin; a common coordinate system; functions (e.g., commands for invoking functions) and other payload data (e.g., text, audio, video, or other multimedia data).

[0048] The augmentation system 208 sends a transformation to the second client device 102 such that the second client device 102 can adjust the AR coordinate system based on the transformation. In this way, the first client device 102 and the second client device 102 synchronize their coordinate systems and frames to display the content in the AR session. Specifically, the augmentation system 208 calculates the origin of the second client device 102 in the coordinate system of the first client device 102. Then, the augmentation system 208 can determine the offset in the coordinate system of the second client device 102 based on the position of the origin in the coordinate system of the second client device 102 from the perspective of the second client device 102. A transformation is generated using the offset such that the second client device 102 generates AR content according to the common coordinate system with the first client device 102.

[0049] The augmentation system 208 can communicate with the client device 102 to establish a separate or shared AR session. The augmentation system 208 can also be coupled to the messaging server 118 to establish an electronic group communication session (e.g., group chat, instant messaging) for the client device 102 in the shared AR session. The electronic group communication session can be associated with a session identifier provided by the client device 102 to obtain access to the electronic group communication session and the shared AR / VR session. In one example, the client device 102 first obtains access to the electronic group communication session and then obtains a session identifier in the electronic group communication session that allows the client device 102 to access the shared AR session. In some examples, the client device 102 is able to access the shared AR / VR session without the help of or communication with the augmentation system 208 in the application server 114.

[0050] The map system 210 provides various geolocation functions and supports the presentation of map-based media content and messages by the messaging client 104. For example, the map system 210 enables the display on a map of: user icons or avatars (e.g., stored in the profile data 316) to indicate the current or past locations of the user's "friends", and media content generated by such friends within the context of the map (e.g., a collection of messages including photos and videos). For example, a message posted by a user from a specific geographical location to the messaging system 100 can be displayed to the user's "friends" within the context of that specific location on the map in the map interface of the messaging client 104. A user can also share his or her location and status information with other users of the messaging system 100 (e.g., using an appropriate status avatar) via the messaging client 104, where the location and status information are similarly displayed within the context of the map interface of the messaging client 104 to selected users.

[0051] The game system 212 provides various game functions within the context of the messaging client 104. The messaging client 104 provides a game interface that presents a list of available games (e.g., web-based games or web-based applications) that can be launched by the user within the context of the messaging client 104 and played with other users of the messaging system 100. The messaging system 100 also enables a specific user to invite such other users to participate in playing a specific game by sending an invitation from the messaging client 104 to the other users. The messaging client 104 also supports both voice and text messaging (e.g., chat) within the game context, provides a leaderboard for the game, and supports the provision of in-game rewards (e.g., game currency and items).

[0052] The external resource system 220 provides an interface for the messaging client 104 to communicate with the external application server 110 to initiate or access external resources. Each external resource (application) server 110 hosts a small-scale version of an application based on a markup language (e.g., HTML5) or an external application (e.g., a game, utility, payment, or ride-sharing application outside of the messaging client 104). The messaging client 104 can initiate the web-based resource by accessing an HTML5 file from an external resource (application) server 110 associated with the web-based resource (e.g., application). In some examples, the applications hosted by the external resource server 110 are programmed in JavaScript using a software development kit (SDK) provided by the messaging server 118. The SDK includes application programming interfaces (APIs) having functions that can be called or activated by the web-based application. In some examples, the messaging server 118 includes a JavaScript library that provides access to certain user data of the messaging client 104 to a given third-party resource. HTML5 is used as an example technology for programming games, but applications and resources programmed based on other technologies can be used.

[0053] To integrate the functionality of the SDK into the web-based resource, the SDK is downloaded from the messaging server 118 by the external resource (application) server 110 or otherwise received by the external resource (application) server 110. Once downloaded or received, the SDK will be included as part of the application code of the web-based external resource. Then, the code of the web-based resource can call or activate certain functions of the SDK to integrate features of the messaging client 104 into the web-based resource.

[0054] The SDK stored on the messaging server 118 effectively provides a bridge between external resources (e.g., third-party or external applications 109 or applets and the messaging client 104). This provides users with a seamless experience of communicating with other users on the messaging client 104 while still retaining the look and feel of the messaging client 104. To bridge the communication between the external resources and the messaging client 104, in some examples, the SDK facilitates the communication between the external resource server 110 and the messaging client 104. In some examples, the WebView JavaScript Bridge running on the client device 102 establishes two one-way communication channels between the external resources and the messaging client 104. Messages are sent asynchronously between the external resources and the messaging client 104 via these communication channels. Each SDK function call is sent as a message and a callback. Each SDK function is implemented by constructing a unique callback identifier and sending a message with that callback identifier.

[0055] By using the SDK, not all information from the messaging client 104 is shared with the external resource server 110. The SDK restricts which information is shared based on the requirements of the external resources. In some examples, each external resource server 110 provides the messaging server 118 with an HTML5 file corresponding to a web-based external resource. The messaging server 118 may add a visual representation (e.g., box art or other graphics) of the web-based external resource in the messaging client 104. Once the user selects the visual representation or indicates that the messaging client 104 accesses the features of the web-based external resource through the GUI of the messaging client 104, the messaging client 104 obtains the HTML5 file and instantiates the resources required to access the features of the web-based external resource.

[0056] The messaging client 104 presents a graphical user interface for an external resource (e.g., a login page or a splash screen). During, before, or after presenting the login page or splash screen, the messaging client 104 determines whether the launched external resource has been previously authorized to access the user data of the messaging client 104. In response to determining that the launched external resource has been previously authorized to access the user data of the messaging client 104, the messaging client 104 presents another graphical user interface of the external resource that includes the functions and features of the external resource. In response to determining that the launched external resource has not been previously authorized to access the user data of the messaging client 104, after a threshold time period (e.g., 3 seconds) of displaying the login page or splash screen of the external resource, the messaging client 104 slides up a menu for authorizing the external resource to access the user data (e.g., animating the menu to emerge from the bottom of the screen to the middle or other part of the screen). The menu identifies the types of user data for which the external resource will be authorized to use. In response to receiving a user selection of an accept option, the messaging client 104 adds the external resource to the list of authorized external resources and enables the external resource to access the user data from the messaging client 104. In some examples, the messaging client 104 authorizes the external resource to access the user data according to the OAuth 2 framework.

[0057] The messaging client 104 controls the types of user data shared with the external resource based on the type of the authorized external resource. For example, access to a first type of user data (e.g., a two-dimensional (2D) avatar of a user with or without different avatar characteristics) is provided to an external resource that includes a full-scale external application (e.g., a third-party or external application 109). As another example, access to a second type of user data (e.g., payment information, a 2D avatar of the user, a 3D avatar of the user, and avatars with various avatar characteristics) is provided to an external resource that includes a scaled-down version of the external application (e.g., a web-based version of a third-party application). Avatar characteristics include different ways of customizing the appearance and feel of the avatar (e.g., different poses, facial features, clothing, etc.).

[0058] The Content and AR Experience Access System 224 allows a user to quickly and easily browse through previously captured content items and AR experiences (or other applications, such as external applications) via a single scrollable area or scroll bar. The Content and AR Experience Access System 224 presents the scrollable area concurrently with the live video feed received from the camera device of the client device 102, and / or presents the scrollable area above the live video feed. The scrollable area includes a first plurality of icons or options for accessing previously captured content items (e.g., images and / or videos captured by the client device 102 and / or received from another user). The scrollable area includes a second plurality of icons or options for accessing different AR experiences. The first plurality of icons or options and the second plurality of icons or options are visually demarcated by an indicator corresponding to the live video feed. The first plurality of icons may be visually distinguishable from the second plurality of icons, e.g., by using a first geometric shape (e.g., square, rectangle, circle, or triangle) and / or color for each of the first plurality of icons, and a second geometric shape (e.g., square, rectangle, circle, or triangle) and / or color for each of the second plurality of icons.

[0059] The Content and AR Experience Access System 224 also presents, separately from the scrollable area, a first icon or option for accessing a complete library or list of all available previously captured content items, and a second icon or option for searching all available AR experiences. That is, the scrollable area may represent or include a subset of all available previously captured content items using only the first plurality of icons or options. The subset may be selected based on the time of day, location, and / or content item type. To view the complete list of previously captured content items, the user may select the first icon or option. Similarly, the scrollable area may represent or include a subset of all available AR experiences using only the second plurality of icons or options. The subset of AR experiences may be selected based on the time of day, location, and / or AR experience type. To view the complete list of AR experiences, the user may select the second icon or option.

[0060] The content and AR experience access system 224 receives input from a user, such as a swipe gesture in a first direction (e.g., left direction), and in response, the content and AR experience access system 224 navigates the scrollable area, or scrolls and animates the scrollable area (such as scrolling in the first direction). The scrollable area includes a focus or a central portion, and different icons among the first plurality of icons or the second plurality of icons are positioned and displayed in the focus or the central portion when the scrollable area is scrolled. When each icon is in focus in the central portion or the focus area, the corresponding content item and / or AR experience is accessed. If the icon in focus corresponds to a previously captured content item, the content and AR experience access system 224 presents the corresponding previously captured content item in the viewfinder presenting the live video feed to replace the live video feed. If the icon in focus corresponds to an AR experience, the content and AR experience access system 224 presents one or more options for launching an AR experience related to the live video feed presented in the viewfinder and / or related to the previously captured content item.

[0061] In some examples, the content and AR experience access system 224 conditionally modifies the configuration of the GUI in which the scrollable area and the live video feed are presented. The conditions can be based on many different factors and criteria, such as which icon is currently in focus. That is, in response to determining that one of the first plurality of options is currently in focus, the content and AR experience access system 224 modifies the configuration of the GUI in a first manner. In response to determining that one of the second plurality of options is currently in focus, the content and AR experience access system 224 modifies the configuration of the GUI in a second manner.

[0062] In some examples, the content and AR experience access system 224 displays, via a messaging application, a live video feed received from a camera device of a client device in the viewfinder. The content and AR experience access system 224 presents a first plurality of options associated with previously captured content items and a second plurality of options associated with AR experiences in a scrollable area above the live video feed being displayed in the viewfinder via the messaging application, and receives input associated with scrolling the scrollable area. In response to receiving the input, the content and AR experience access system 224 scrolls the first plurality of options together with the second plurality of options.

[0063] In some examples, the content and AR experience access system 224 presents an indicator of the live video feed within the scrollable area between the first plurality of options and the second plurality of options. In some aspects, each of the first plurality of options is represented by a first type of visual indicator, and each of the second plurality of options is represented by a second type of visual indicator. In some aspects, the previously captured content item includes at least one of a previously captured image or a previously captured video.

[0064] In some aspects, the previously captured content items associated with the first plurality of options correspond to a subset of the plurality of content items previously stored by the client device. In such a case, the content and AR experience access system 224 presents options for accessing the plurality of content items above the live video feed being displayed in the viewfinder and separate from the scrollable region.

[0065] In some aspects, the content and AR experience access system 224 selects a subset of the plurality of content items from the plurality of content items previously stored by the client device based on one or more criteria. The one or more criteria include the time associated with each of the subset of the plurality of content items, the location associated with each of the subset of the plurality of content items, and / or the type of content item associated with each of the subset of the plurality of content items.

[0066] In some examples, the AR experience associated with the second plurality of options corresponds to a subset of the plurality of AR experiences available on the messaging application. In such a case, the content and AR experience access system 224 presents options for searching the plurality of AR experiences above the live video feed being displayed in the viewfinder and separate from the scrollable region. In some aspects, the options for searching the plurality of AR experiences include a first option. The content and AR experience access system 224 presents a second option for accessing the plurality of content items above the live video feed being displayed in the viewfinder and separate from the scrollable region.

[0067] In some examples, the input includes a swipe gesture in a first direction above the scrollable region. In response to the swipe gesture, the content and AR experience access system 224 scrolls the first plurality of options and the second plurality of options together in the first direction. In some aspects, the content and AR experience access system 224 identifies a single option currently in focus in the first plurality of options based on scrolling the first plurality of options and the second plurality of options together. The content and AR experience access system 224 replaces the display of the live video feed in the viewfinder with the display of a single previously captured content item associated with the single option currently in focus in the first plurality of options. The content and AR experience access system 224 presents a navigation region associated with the single previously captured content item in response to identifying the single option currently in focus in the first plurality of options, the navigation region including one or more options associated with the single previously captured content item.

[0068] In some examples, one or more options include editing options for modifying individual previously captured content items, category-based search options, and / or content sharing options for sending individual previously captured content items to one or more other users. In some aspects, when different content items among the previously captured content items associated with the first plurality of options come into focus based on scrolling the first plurality of options together with the second plurality of options, the content and AR experience access system 224 updates the display of the live video feed in the viewfinder using the display of the different content items among the previously captured content items. In some aspects, the content and AR experience access system 224 presents a first content item among the previously captured content items in the viewfinder at a first aspect ratio associated with the first content item among the previously captured content items. After presenting the first content item among the previously captured content items, the content and AR experience access system 224 presents a second content item among the previously captured content items in the viewfinder at a second aspect ratio associated with the second content item among the previously captured content items.

[0069] In some examples, the input includes a swipe gesture in a second direction above the scrollable region. In such a case, in response to the swipe gesture, the content and AR experience access system 224 scrolls the first plurality of options and the second plurality of options together in the second direction. In some examples, the content and AR experience access system 224 identifies an individual option currently in focus in the second plurality of options based on scrolling the first plurality of options together with the second plurality of options. The content and AR experience access system 224 applies an individual AR experience associated with the individual option currently in focus in the second plurality of options to the live video feed in the viewfinder. The content and AR experience access system 224 presents a navigation region associated with the individual AR experience in response to identifying the individual option currently in focus in the second plurality of options, the navigation region including one or more options associated with the individual AR experience.

[0070] In some examples, one or more options include options for modifying attributes of an individual AR experience, activating an individual AR experience, and / or saving an individual AR experience. In some aspects, the content and AR experience access system 224 receives a swipe gesture in the second direction and determines that the rate of performing the swipe gesture in the second direction exceeds a reference rate. In response to determining that the rate of performing the swipe gesture in the second direction exceeds the reference rate, the content and AR experience access system 224 scrolls the first plurality of options and the second plurality of options together in the second direction until a predetermined termination point between a first option in the first plurality of options and a first option in the second plurality of options. In some aspects, the predetermined termination point corresponds to an indicator of the live video feed that is displayed within the scrollable region between the first plurality of options and the second plurality of options.

[0071] In some examples, the content and AR experience access system 224 presents a first plurality of options associated with previously captured content items and a second plurality of options associated with AR experiences in a scrollable region above a live video feed being displayed in a GUI that includes a viewfinder. The content and AR experience access system 224 scrolls the first plurality of options and the second plurality of options of the scrollable region together such that one of the first plurality of options or one of the second plurality of options is in focus. In response to scrolling the first plurality of options of the scrollable region together with the second plurality of options: in response to determining that one of the first plurality of options is currently in focus, the content and AR experience access system 224 modifies the configuration of the GUI in a first manner, and in response to determining that one of the second plurality of options is currently in focus, modifies the configuration of the GUI in a second manner.

[0072] In some examples, the content and AR experience access system 224 presents an indicator of the live video feed within the scrollable region between the first plurality of options and the second plurality of options. In some examples, each of the first plurality of options is represented by a first type of visual indicator, and each of the second plurality of options is represented by a second type of visual indicator. In some examples, the previously captured content items include at least one of a previously captured image or a previously captured video. In some examples, the previously captured content items associated with the first plurality of options correspond to a subset of a plurality of content items previously stored by the client device. In such a case, the content and AR experience access system 224 presents an option to access the plurality of content items above the live video feed being displayed in the GUI and separate from the scrollable region.

[0073] In some examples, the content and AR experience access system 224 selects a subset of the plurality of content items from the plurality of content items previously stored by the client device based on one or more criteria, the one or more criteria including the time associated with each of the subset of the plurality of content items, the location associated with each of the subset of the plurality of content items, and / or the content item type associated with each of the subset of the plurality of content items. In some aspects, the AR experiences associated with the second plurality of options correspond to a subset of the plurality of AR experiences available on the messaging application. The content and AR experience access system 224 presents an option to search for the plurality of AR experiences above the live video feed being displayed in the GUI and separate from the scrollable region.

[0074] In some examples, the options for searching for multiple AR experiences include a first option, and the content and AR experience access system 224 presents a second option for accessing multiple content items above the live video feed being displayed in the GUI and separate from the scrollable region. In some aspects, the content and AR experience access system 224 receives an input that includes a swipe gesture in a given direction, and in response to the swipe gesture, scrolls the first plurality of options together with the second plurality of options in the given direction.

[0075] In some examples, the content and AR experience access system 224 receives a swipe gesture in a second direction and determines that the rate of performing the swipe gesture in the second direction exceeds a reference rate. In response to determining that the rate of performing the swipe gesture in the second direction exceeds the reference rate, the content and AR experience access system 224 scrolls the first plurality of options together with the second plurality of options in the second direction until a predetermined termination point between a first option in the first plurality of options and a first option in the second plurality of options. In some examples, the predetermined termination point corresponds to an indicator of the live video feed that is displayed within the scrollable region between the first plurality of options and the second plurality of options.

[0076] In some aspects, the content and AR experience access system 224 identifies a single option that is currently in focus in the first plurality of options and replaces the display of the live video feed in the viewfinder with the display of a single previously captured content item associated with the single option that is currently in focus in the first plurality of options. In some aspects, the content and AR experience access system 224 modifies the configuration of the GUI in a first manner by presenting a navigation region associated with one of the first plurality of options that is currently in focus, the navigation region including one or more options associated with a single previously captured content item corresponding to the one of the first plurality of options that is currently in focus.

[0077] In some aspects, one or more options include editing options for modifying individual previously captured content items, category-based search options, and / or content sharing options for sending individual previously captured content items to one or more other users. In some aspects, the content and AR experience access system 224 receives an input selecting an editing option and, in response to receiving the input, presents one or more image editing features corresponding to the editing option over the individual previously captured content item. In some aspects, the editing option includes a caption tool, and the content and AR experience access system 224 presents a text input area over the individual previously captured content item and presents a text attribute selection area over the individual previously captured content item. In some aspects, the editing option includes a graphical element option, and the content and AR experience access system 224 presents a plurality of options for adding a corresponding graphical element over the individual previously captured content item.

[0078] In some examples, before modifying the configuration of the GUI in a first manner, the content and AR experience access system 224 presents capture options associated with a live video feed being displayed in the GUI, where an image or video including the live video feed is stored in response to receiving an input selecting the capture option. In response to modifying the configuration of the GUI in the first manner, the content and AR experience access system 224 removes the display of the capture options.

[0079] In some examples, the content and AR experience access system 224 modifies the configuration of the GUI in a second manner by populating the capture options associated with the live video feed being displayed in the GUI with a representation of a single AR experience corresponding to one of the second plurality of options that is currently in focus and presenting a navigation area associated with one of the second plurality of options that is currently in focus. In one example, the navigation area includes one or more options associated with an individual AR experience, and the one or more options include options for modifying the attributes of the individual AR experience, options for activating the individual AR experience, and options for saving the individual AR experience.

[0080] Data Architecture

[0081] Figure 3 is a schematic diagram showing a data structure 300 that can be stored in the database 126 of the messaging server system 108 according to certain examples. Although the contents of the database 126 are shown as including several tables, it should be understood that the data can be stored in other types of data structures (e.g., other types of data structures are object-oriented databases).

[0082] The database 126 includes message data stored in the message table 302. For any particular message, the message data includes at least message sender data, message recipient (or receiver) data, and a payload. The following references Figure 4 describe additional details regarding information that can be included in a message and that is included in the message data stored in the message table 302.

[0083] The entity table 306 stores entity data and is linked (e.g., by reference) to the entity graph 308 and the profile data 316. Entities for which records are maintained within the entity table 306 can include individuals, corporate entities, organizations, objects, locations, events, etc. Regardless of the entity type, any entity for which the messaging server system 108 stores data can be an identified entity. A unique identifier and an entity type identifier (not shown) are provided for each entity.

[0084] The entity graph 308 stores information regarding relationships and associations between entities. By way of example only, such relationships can be social, professional (e.g., working in a common company or organization), interest-based, or activity-based.

[0085] The profile data 316 stores various types of profile data regarding a particular entity. Based on privacy settings specified by the particular entity, the profile data 316 can be selectively used and presented to other users of the messaging system 100. In the case where the entity is an individual, the profile data 316 includes, for example, a username, a telephone number, an address, settings (e.g., notification and privacy settings), and a user-selected avatar representation (or a collection of such avatar representations) and an associated virtual wardrobe that includes one or more virtual fashion items that can be placed on the avatar. A particular user can then selectively include one or more of these avatar representations within the content of messages transmitted via the messaging system 100 and on a map interface displayed by the messaging client 104 to other users. The collection of avatar representations can include "status avatars" that present graphical representations of statuses or activities that the user can select to convey at a particular time.

[0086] In the case where the entity is a group, in addition to the group name, members, and various settings (e.g., notifications) of the associated group, the profile data 316 of the group can similarly include one or more avatar representations associated with the group.

[0087] The database 126 also stores enhancement data, such as overlays or filters, in the enhancement table 310. The enhancement data is associated with videos (the data of the videos is stored in the video table 304) and images (the data of the images is stored in the image table 312) and is applied to the videos and images.

[0088] The database 126 can also store data related to individual and shared AR sessions. This data can include data transmitted between the AR session client controller of the first client device 102 and another AR session client controller of the second client device 102, as well as data transmitted between the AR session client controller and the augmentation system 208. The data can include data for establishing a common coordinate system for a shared AR scene, transformations between devices, session identifiers, images depicting bodies, skeletal joint positions, wrist joint positions, feet, etc.

[0089] In one example, a filter is an overlay that is displayed as an overlay on an image or video during presentation to a receiving user. Filters can be of various types, including filters selected by a sending user from a set of filters presented to the sending user by the messaging client 104 when the sending user is composing a message. Other types of filters include location-based filters (also known as geo-filters), which can be presented to the sending user based on a geographical location. For example, a location-based filter specific to a nearby or specific location can be presented by the messaging client 104 within the user interface based on geographical location information determined by the global positioning system (GPS) unit of the client device 102.

[0090] Another type of filter is a data filter, which can be selectively presented to the sending user by the messaging client 104 based on other input or information collected by the client device 102 during the message creation process. Examples of data filters include the current temperature at a specific location, the current speed of the sending user's travel, the battery life of the client device 102, or the current time.

[0091] Other augmentation data that can be stored in the image table 312 includes augmented reality content items (e.g., corresponding to an applied augmented reality experience). Augmented reality content items or augmented reality items can be real-time special effects and sounds that can be added to an image or video.

[0092] As described above, the augmented data includes augmented reality content items, overlays, image transformations, AR images, and similar items related to modifications that can be applied to image data (e.g., video or images). This includes real-time modifications that modify the image when it is captured using the device sensors of the client device 102 (e.g., one or more camera devices) and then display the modified image on the screen of the client device 102. This also includes modifications to stored content (e.g., video clips in a library that can be modified). For example, in a client device 102 capable of accessing multiple augmented reality content items, a user can use a single video clip with multiple augmented reality content items to see how different augmented reality content items will modify the stored clip. For example, by selecting different augmented reality content items for the content, multiple augmented reality content items applying different pseudo-random movement models can be applied to the same content. Similarly, real-time video capture can be used with the shown modifications to show how the video images currently captured by the sensors of the client device 102 will modify the captured data. Such data can be simply displayed on the screen without being stored in memory, or the content captured by the device sensors can be recorded and stored in memory with or without modification (or both). In some systems, a preview function can show how different augmented reality content items will look within different windows of the display. This can, for example, enable multiple windows with different pseudo-random animations to be viewed simultaneously on the display.

[0093] Accordingly, systems that use data of augmented reality content items and various other such transformation systems that use the data to modify content can involve the detection of objects (e.g., faces, hands, bodies, cats, dogs, surfaces, objects, etc.) in video frames, the tracking of such objects as they leave the field of view, enter the field of view, and move around in the field of view, and the modification or transformation of such objects when tracking them. In various examples, different methods can be used to implement such transformations. Some examples can involve: generating a three-dimensional mesh model of one or more objects; and using the transformation and animated textures of the model within the video to implement the transformation. In other examples, tracking of points on the object can be utilized to place an image or texture (which can be two-dimensional or three-dimensional) at the tracked location. In yet another example, neural network analysis of video frames can be used to place an image, model, or texture in the content (e.g., an image or video frame). Accordingly, augmented reality content items refer both to the images, models, and textures used to create transformations in the content and to the additional modeling and analysis information required to implement such transformations using object detection, tracking, and placement.

[0094] Real-time video processing can be performed using any kind of video data (e.g., video streams, video files, etc.) stored in the memory of any kind of computerized system. For example, a user can load video files and save them in the memory of a device, or can use the sensors of the device to generate a video stream. In addition, computer animation models can be used to process any object, such as a person's face and parts of a person's body, an animal, or a non-biological object (e.g., a chair, a car, or other object).

[0095] In some examples, when a specific modification is selected along with the content to be transformed, the computing device identifies the element to be transformed and then, if the element to be transformed exists in a video frame, detects and tracks the element to be transformed. The element of the object is modified according to the request for the modification, thereby transforming the frame of the video stream. For different types of transformations, the transformation of the frame of the video stream can be performed by different methods. For example, for a frame transformation that mainly involves changing the form of the element of an object, the feature points for each element of the object are calculated (e.g., using an Active Shape Model (ASM) or other known methods). Then, for each element in at least one element of the object, a grid based on the feature points is generated. This grid is used for the subsequent stage of tracking the element of the object in the video stream. During the tracking process, the mentioned grid for each element is aligned with the position of each element. Then, additional points are generated on the grid. A first set of points is generated for each element based on the request for the modification, and a second set of points is generated for each element based on the first set of points and the request for the modification. Then, the frame of the video stream can be transformed by modifying the element of the object based on the first set of points, the second set of points, and the grid. In such a method, the background of the modified object can also be changed or distorted by tracking and modifying the background of the modified object.

[0096] In some examples, a transformation that uses elements of an object to change some regions of the object can be performed by calculating feature points for each element of the object and generating a mesh based on the calculated feature points. Points are generated on the mesh, and then various regions based on the points are generated. Then, the elements of the object are tracked by aligning the regions for each element with the positions of each element in at least one element, and the properties of the regions can be modified based on a modification request, thereby transforming the frames of the video stream. Depending on the specific modification request, the properties of the mentioned regions can be transformed in different ways. Such modifications can involve: changing the color of the region; removing at least some parts of the region from the frame of the video stream; including one or more new objects in the region based on the modification request; and modifying or distorting the elements of the region or the object. In various examples, any combination of such modifications or other similar modifications can be used. For some models to be animated, some feature points can be selected as control points for determining the entire state space of options for model animation.

[0097] In some examples of using computer animation models to transform image data using face detection, a specific face detection algorithm (e.g., Viola-Jones) is used to detect faces in the image. Then, an active shape model (ASM) algorithm is applied to the face region of the image to detect face feature reference points.

[0098] Other suitable face detection methods and algorithms can be used. For example, in some examples, landmarks are used to locate features, which represent distinguishable points that exist in most of the images under consideration. For example, for face landmarks, the position of the left eye pupil can be used. If the initial landmark is not recognizable (e.g., if a person wears an eye patch), secondary landmarks can be used. Such landmark identification processes can be used for any such object. In some examples, a set of landmarks forms a shape. The coordinates of the points in the shape can be used to represent the shape as a vector. A similarity transformation (allowing translation, scaling, and rotation) that minimizes the average Euclidean distance between the shape points is used to align one shape with another. The average shape is the average of the aligned training shapes.

[0099] In some examples, the search for landmarks starts from the average shape aligned with the position and size of the face determined by the global face detector. Then, such a search repeats the following steps: a tentative shape is proposed by adjusting the positions of the shape points through template matching of the image texture around each point, and then the tentative shape is made to conform to the global shape model until convergence occurs. In some systems, individual template matching is unreliable, and the shape model pools the results of weak template matches to form a stronger overall classifier. The entire search is repeated at each level of an image pyramid from coarse resolution to fine resolution.

[0100] The transformation system can capture an image or video stream on a client device (e.g., client device 102) and perform complex image manipulations locally on the client device 102 while maintaining an appropriate user experience, computation time, and power consumption. The complex image manipulations can include size and shape changes, mood transformations (e.g., changing a face from a frown to a smile), state transformations (e.g., aging a subject, reducing apparent age, changing gender), style transformations, application of graphical elements, and any other suitable image or video manipulations implemented by a convolutional neural network that has been configured to execute effectively on the client device 102.

[0101] In some examples, a computer animation model for transforming image data can be used by the system, in which a user can use the client device 102 with a neural network to capture an image or video stream of the user (e.g., a selfie), and the neural network operates as part of a messaging client 104 operating on the client device 102. The transformation system operating within the messaging client 104 determines the presence of a face within the image or video stream and provides a modification icon associated with the computer animation model to transform the data image, or the computer animation model can be present in association with the interfaces described herein. The modification icon includes a change that can be used as a basis for modifying the user's face within the image or video stream as part of a modification operation. Once the modification icon is selected, the transformation system initiates a process to transform the user's image to reflect the selected modification icon (e.g., generate a smiling face on the user). Once the image or video stream is captured and a specified modification is selected, the modified image or video stream can be presented in a graphical user interface displayed on the client device 102. The transformation system can implement a complex convolutional neural network on a portion of the image or video stream to generate and apply the selected modification. That is, the user can capture an image or video stream, and once the modification icon has been selected, the modified result can be presented in real time or near real time. Additionally, when a video stream is being captured and the selected modification icon remains toggled, the modification can be persistent. Machine learning neural networks can be used to implement such modifications.

[0102] A graphical user interface presenting modifications performed by a transformation system can provide additional interaction options to a user. Such options can be based on an interface for initiating content capture and selection of a particular computer animation model (e.g., initiated from a content creator user interface). In various examples, after an initial selection of a modification icon, the modification can be persistent. The user can switch the modification on or off by tapping or otherwise selecting the face being modified by the transformation system and store it for later viewing or browsing other areas of the imaging application. In cases where multiple faces are modified by the transformation system, the user can globally switch the modification on or off by tapping or selecting a single face modified and displayed within the graphical user interface. In some examples, individual faces within a group of multiple faces can be modified separately, or such modifications can be switched separately by tapping or selecting an individual face or a sequence of faces displayed within the graphical user interface.

[0103] The story sheet 314 stores data regarding a collection of messages and associated image, video, or audio data, which are compiled into a collection (e.g., a story or a library). The creation of a particular collection can be initiated by a particular user (e.g., each user maintaining a record in the entity table 306). A user can create a "personal story" in the form of a collection of content that has been created and sent / broadcast by that user. To this end, the user interface of the messaging client 104 can include user-selectable icons to enable the sending user to add specific content to his or her personal story.

[0104] Collections can also constitute "live stories", which are collections of content from multiple users created manually, automatically, or using a combination of manual and automatic techniques. For example, a "live story" can constitute a curated stream of user-submitted content from various locations and events. Users whose client devices have location services enabled and are at a common location event at a particular time can be presented, for example, via the user interface of the messaging client 104, with the option to contribute content to a particular live story. A live story can be identified to a user by the messaging client 104 based on his or her location. The end result is a "live story" told from a community perspective.

[0105] Another type of content collection is referred to as a "location story", which enables users whose client devices 102 are located within a particular geographical location (e.g., on a college or university campus) to contribute to a particular collection. In some implementations, contributing to a location story may require secondary authentication to verify that the end user belongs to a particular organization or other entity (e.g., is a student on a university campus).

[0106] As mentioned above, the video table 304 stores video data which, in one example, is associated with messages for which records are maintained in the message table 302. Similarly, the image table 312 stores image data which is associated with messages for which the message data is stored in the entity table 306. The entity table 306 can associate various enhancements from the enhancement table 310 with the various images and videos stored in the image table 312 and the video table 304.

[0107] The data structure 300 can also store training data for training one or more machine learning techniques (models) to generate 2D bounding boxes. The training data can include multiple training videos and corresponding ground truth bounding boxes. The images and videos can include a mixture of various real-world objects which can occur in different real-world environments such as a home or different rooms in a home. One or more machine learning techniques or models can be trained to extract features of the received input images or videos and establish a relationship between the extracted features and the 2D bounding boxes of the real-world objects depicted in the images or videos. Once trained, the machine learning technique can receive new images or videos and can estimate 2D bounding boxes for the newly received images or videos.

[0108] Data communication architecture

[0109] Figure 4 is a schematic diagram showing the structure of a message 400 according to some examples, which is generated by the messaging client 104 for transmission to another messaging client 104 or the messaging server 118. The content of a particular message 400 is used to populate the message table 302 which is stored in the database 126 and is accessible by the messaging server 118. Similarly, the content of the message 400 is stored in the memory as "in-transit" or "in-flight" data of the client device 102 or the application server 114. The message 400 is shown to include the following example components:

[0110] ● Message identifier 402: A unique identifier that identifies the message 400.

[0111] ● Message text payload 404: The text to be generated by the user via the user interface of the client device 102 and included in the message 400.

[0112] ● Message image payload 406: Image data captured by the camera device component of the client device 102 or retrieved from the memory of the client device 102 and included in the message 400. The image data for a sent or received message 400 can be stored in the image table 312.

[0113] ● Message Video Payload 408: Video data that is captured by a camera device component or retrieved from a memory component of the client device 102 and included in the message 400. The video data for a sent or received message 400 can be stored in the video table 304.

[0114] ● Message Audio Payload 410: Audio data that is captured by a microphone or retrieved from a memory component of the client device 102 and included in the message 400.

[0115] ● Message Enhancement Data 412: Enhancement data (e.g., filters, stickers, or other annotations or enhancements) that represents an enhancement to be applied to the message image payload 406, message video payload 408, or message audio payload 410 of the message 400. The enhancement data 412 for a sent or received message 400 can be stored in the enhancement table 310.

[0116] ● Message Duration Parameter 414: A parameter value that indicates, in seconds, the amount of time that the content of the message (e.g., message image payload 406, message video payload 408, message audio payload 410) will be presented to a user or made accessible to the user via the message client 104.

[0117] ● Message Geolocation Parameter 416: Geolocation data (e.g., latitude coordinates and longitude coordinates) associated with the content payload of the message. Multiple message geolocation parameter 416 values can be included in the payload, with each of these parameter values being associated with a content item included in the content (e.g., a specific image within the message image payload 406, or a specific video within the message video payload 408).

[0118] ● Message Story Identifier 418: An identifier value that identifies one or more content collections (e.g., "stories" identified in the story table 314) associated with a specific content item in the message image payload 406 of the message 400. For example, the identifier value can be used to associate each of multiple images within the message image payload 406 with multiple content collections.

[0119] ● Message Tag 420: Each message 400 can be tagged with multiple tags, with each of the multiple tags indicating a theme of the content included in the message payload. For example, in the case where a specific image included in the message image payload 406 depicts an animal (e.g., a lion), a tag value indicating the relevant animal can be included within the message tag 420. The tag values can be manually generated based on user input, or can be automatically generated using, for example, image recognition.

[0120] ● Message sender identifier 422: An identifier (e.g., a messaging system identifier, an email address, or a device identifier) that indicates the user of the client device 102 on which the message 400 is generated and from which the message 400 is sent.

[0121] ● Message recipient identifier 424: An identifier (e.g., a messaging system identifier, an email address, or a device identifier) that indicates the user of the client device 102 to which the message 400 is addressed.

[0122] The content (e.g., values) of the respective components of the message 400 can be pointers to positions in a table in which content data values are stored. For example, the image value in the message image payload 406 can be a pointer to a position within the image table 312 (or the address of a position within the image table 312). Similarly, the values within the message video payload 408 can point to data stored within the video table 304, the values stored within the message enhancement data 412 can point to data stored within the enhancement table 310, the values stored within the message story identifier 418 can point to data stored within the story table 314, and the values stored within the message sender identifier 422 and the message recipient identifier 424 can point to user records stored within the entity table 306.

[0123] Figure 5A FIG. is a flowchart showing example operations of the content and AR experience access system 224 during an execution process 500 according to some examples. The process 500 can be embodied as computer-readable instructions executed by one or more processors such that the operations of the process 500 can be performed in part or in whole by functional components of the messaging server system 108; thus, the process 500 is described below by way of example with reference to the messaging server system 108. However, in other embodiments, at least some of the operations of the process 500 can be deployed on various other hardware configurations. Accordingly, the process 500 is not intended to be limited to the messaging server system 108 and can be implemented in whole or in part by any other component. Some or all of the operations of the process 500 can be parallel, unordered, or completely omitted.

[0124] At operation 502, as discussed above and below, the content and AR experience access system 224 displays, via a messaging application, a live video feed received from a camera device of the client device in a viewfinder.

[0125] At operation 503, as discussed above and below, the content and AR experience access system 224 presents, via the messaging application, in a scrollable area above the live video feed being displayed in the viewfinder, a first plurality of options associated with previously captured content items and a second plurality of options associated with AR experiences.

[0126] At operation 504, as discussed above and below, the content and AR experience access system 224 receives input associated with scrolling a scrollable region.

[0127] At operation 505, in response to receiving the input, the content and AR experience access system 224 scrolls a first plurality of options together with a second plurality of options (as discussed above and below).

[0128] Figure 5B FIG. is a flowchart showing example operations of the content and AR experience access system 224 in performing process 501 according to some examples. Process 501 may be embodied as computer-readable instructions executed by one or more processors such that the operations of process 501 may be performed in part or in whole by functional components of the messaging server system 108; thus, process 501 is described below by way of example with reference to the messaging server system 108. However, in other embodiments, at least some of the operations of process 501 may be deployed on various other hardware configurations. Accordingly, process 501 is not intended to be limited to the messaging server system 108 and may be implemented in whole or in part by any other component. Some or all of the operations of process 501 may be parallel, unordered, or completely omitted.

[0129] At operation 512, as discussed above and below, the content and AR experience access system 224 presents, via a messaging application, a first plurality of options associated with previously captured content items and a second plurality of options associated with an AR experience in a scrollable region above a live video feed being displayed in a GUI including a viewfinder.

[0130] At operation 513, as discussed above and below, the content and AR experience access system 224 scrolls the first plurality of options of the scrollable region together with the second plurality of options to bring one of the first plurality of options or one of the second plurality of options into focus.

[0131] At operation 514, as discussed above and below, in response to scrolling the first plurality of options of the scrollable region together with the second plurality of options: in response to determining that one of the first plurality of options is currently in focus, the content and AR experience access system 224 modifies the configuration of the GUI in a first manner.

[0132] At operation 515, as discussed above and below, in response to scrolling the first plurality of options of the scrollable region together with the second plurality of options: in response to determining that one of the second plurality of options is currently in focus, the content and AR experience access system 224 modifies the configuration of the GUI in a second manner.

[0133] Figure 6A 、Figure 6B , Figure 6C , Figure 6D , Figure 7 , Figure 8A and Figure 8B illustrate illustrative inputs and outputs of content and AR experience access system 224 according to some examples. As Figure 6A shown, GUI 600 is presented by AR experience access system 224 on client device 102. GUI 600 includes viewfinder 611, which displays a live video feed captured by a front or rear camera device of client device 102. GUI 600 includes capture option 620. In response to receiving a user selection of capture option 620, AR experience access system 224 stores an image or video corresponding to the live video feed. For example, if capture option 620 is held down, a set of frames of the live video feed begins to be captured and stored as a video until capture option 620 is released. If capture option 620 is tapped, a single image shown in viewfinder 611 is captured and stored. The captured and stored video and / or image forms part of a set of previously captured images / videos.

[0134] In some examples, first icon 622 is presented on the same line as capture option 620 and second icon 624. First icon 622, when selected (e.g., tapped by a user), instructs AR experience access system 224 to present a full-screen GUI that overlays viewfinder 611 and includes a complete list of a complete library of previously captured content items. In some cases, first icon 622 does not cause all previously captured content items to be presented, but rather causes a subset of content items that meet one or more criteria to be presented. For example, in response to receiving an input selecting first icon 622, AR experience access system 224 may present images / videos shared with one or more other users and / or images / videos captured and stored at a specified time interval or specified location. From the GUI of the presented previously captured content items, a given content item can be selected to be shared with one or more other users / friends and / or modified (edited).

[0135] In response to receiving a user selection of second icon 624, AR experience access system 224 presents a GUI that overlays viewfinder 611 and includes a search or exploration interface for accessing a complete list of available AR experiences or external applications. From the GUI of the AR experience, AR experience access system 224 can receive search parameters for accessing or identifying one or more AR experiences that meet the search criteria. A given AR experience can be selected and used to modify or enhance the live video feed presented in viewfinder 611.

[0136] In some examples, the GUI 600 includes a scrollable region 610 that provides quick and easy access to a mix of content item types. For example, the scrollable region 610 provides quick and easy access to a collection or subset of previously captured content items and a collection or subset of AR experiences. A collection or subset of previously captured content items can be selected by retrieving or identifying a quantity (e.g., 25 or fewer) of previously captured content items from a full library of content items available in response to receiving an input selecting the first icon 622. A collection of previously captured content items can be formed by searching the full library of content items for content items that meet one or more criteria, such as content items of a particular type, content items captured at a particular date, time, and / or location, and / or content items shared with one or more other users. A collection or subset of AR experiences can be selected by retrieving or identifying a quantity (e.g., 25 or fewer, which can be the same or a different quantity than that used to select the collection of previously captured content items) of AR experiences from a full library of AR experiences searchable in response to receiving an input selecting the second icon 624. A collection of AR experiences can be formed by searching the full library of AR experiences for AR experiences that meet one or more criteria, such as content items of a particular type, content items captured at a particular date, time, and / or location, and / or content items shared with one or more other users.

[0137] The scrollable region 610 uses a first set of icons or options 612 to represent each of the subsets of previously captured content items. The first set of icons or options 612 can present a first visual indicator (e.g., geometric shape and / or color). The scrollable region 610 uses a second set of icons or options 614 to represent each of the subsets of AR experiences. The second set of icons or options 614 can present a second visual indicator different from the first visual indicator (e.g., geometric shape and / or color). The first set of icons or options 612 can be separated or divided from the second set of icons or options 614 by an indicator 616. The indicator 616 can be an option or representation of a live video feed presented in the viewfinder 611. The indicator 616 can be presented with a third visual attribute different from the first visual indicator and the second visual indicator (e.g., geometric shape and / or color).

[0138] The AR experience access system 224 receives an input associated with the scrollable region 610 to adjust or control which option is currently in focus. As Figure 6AAs shown, when the indicator 616 is at the center of the scrollable region 610, the indicator 616 is currently in focus. In this case, the viewfinder 611 presents the content corresponding to the indicator 616 (e.g., a live video feed received from the imaging device of the client device 102). The input can be a swiping gesture, which is achieved by detecting the placement of a finger or digit of the user's hand on the region of the GUI 600 corresponding to the scrollable region 610 (where the scrollable region 610 is presented) and detecting the movement or drag of the finger or digit in a given direction (e.g., left or right).

[0139] In response to the AR experience access system 224 detecting that a swiping gesture is being made in a first direction (e.g., a finger or digit is being dragged across the GUI 600 towards the left), the AR experience access system 224 animates and scrolls the first set of options 612 and the second set of options 614 (along with the indicator 616) towards the first direction (e.g., towards the left). The speed or rate at which the icons in the scrollable region 610 scroll can correspond to the speed or rate at which the swiping gesture is made. As the scrollable region 610 scrolls to the left, different options corresponding to different AR experiences in the options 614 are brought into focus (e.g., into the focus region or the center region of the scrollable region 610).

[0140] For example, as Figure 6B shown in the GUI 601 of, based on the swiping gesture made in the first direction, the viewfinder 611 has scrolled to the left. A given option 632 corresponding to a given AR experience in the options 614 is brought into focus. The given option 632 in the options 614 that is in focus is shown by a white border placed around the option 632. The capture option 620 is updated to reflect the AR experience corresponding to the given option 632 that is in focus. For example, the visual representation 634 fills or populates the capture option 620 to represent or identify the AR experience corresponding to the given option 632 that is in focus.

[0141] In some examples, in response to determining that a given option 632 in focus corresponds to an AR experience, the AR experience access system 224 also presents a navigation bar or menu 636. The menu 636 may include one or more options or features associated with the AR experience, such as a recommendation option, a shopping option, a play option, a save option, a close option, and a search option. When the recommendation option is selected, additional AR experiences are provided to the user for selection and activation. The shopping option provides the user with a list of other available AR experiences that the user can purchase and activate. The play option allows the user to instruct the AR experience access system 224 to initiate the AR experience by modifying the live video feed presented in the viewfinder 611 with AR elements of the AR experience corresponding to the visual representation 634. The save option allows the user to save the AR experience corresponding to the visual representation 634 in a separate library of saved AR experiences. The search option presented in the menu 636, when selected, may perform the same or a similar function as the second icon 624. Other menu options may be presented in another area of the viewfinder 611, such as options to add music to the live video feed, perform a replay, capture a short video, etc.

[0142] In some cases, in response to detecting that a swipe gesture is being made in a second direction (e.g., a finger or pointer is dragged across the GUI 601 towards the right), the AR experience access system 224 animates and scrolls the first set of options 612 and the second set of options 614 (along with the indicator 616) in the second direction (e.g., towards the right). That is, when one of the options 614 is in focus, the AR experience access system 224 may receive an input including a swipe gesture for scrolling the scrollable area 610 towards the options 614. The speed or rate of execution of the gesture may be measured and compared to a reference or threshold. If the speed or rate of execution of the gesture exceeds the reference or threshold, the AR experience access system 224 scrolls the options 612 and the options 614 in the second direction of the swipe gesture at the speed or rate of the gesture, but the AR experience access system 224 automatically stops scrolling the options 612 and the options 614 at a specified point (e.g., the indicator 616). In such a case, when automatically stopping the scrolling of the scrollable area 610, the AR experience access system 224 automatically presents the live video feed in the viewfinder 611. The AR experience access system 224 may receive additional inputs, including additional swipe gestures in the second direction, for bringing the options 612 into focus.

[0143] In some examples, as Figure 6CAs shown in GUI 603, in response to determining that a given option 632 in focus corresponds to an AR experience and in response to receiving a user selection of the visual representation 634, the AR experience access system 224 also presents a context - relevant AR experience 635. The context - relevant AR experience 635 may include one or more options or features corresponding to the relevant AR experience, such as an AR experience developed by the same developer as the AR experience associated with the visual representation 634 and / or having content similar to the AR experience associated with the visual representation 634. The AR experience access system 224 may receive an input selecting one of the AR experiences from the context - relevant AR experience 635 and may automatically launch the selected AR experience to enhance the live video feed in the viewfinder 611 with the corresponding AR object. The context - relevant AR experience 635 may be presented as a small icon above the visual representation 634 (as shown in GUI 603).

[0144] In some cases, the context - relevant AR experience 635 may be presented on the same line as the visual representation 634 (as Figure 6D shown in GUI 604). For example, GUI 604 presents a visual representation 634 corresponding to the AR experience navigated to and brought into focus in the scrollable area 610. GUI 604 presents a set of related AR experiences 651 in the same line as the visual representation 634. The AR experience access system 224 may receive an input that slides left / right along the line of the related AR experiences 651 to replace the visual representation 634 in focus with a different representation of one of the related AR experiences 651. The AR experience access system 224 may also present a set of options 650 for adjusting the parameters of the AR experience associated with the visual representation 634 in focus. The set of options 650 may include a camera device control option for switching between the front - facing camera device and the rear - facing camera device from which the live video feed is received, a dual - camera device activation option for activating both the front - facing camera device and the rear - facing camera device of the client device 102, a timer option for activating a timer to start / stop recording a video corresponding to the live video feed, an import media option for adding media to the AR experience for enhancing the live video feed, and so on.

[0145] Returning to Figure 6A, in response to detecting that a swipe gesture is being made in a second direction (e.g., a finger or pointer is being dragged across the GUI 600 towards the right), the AR experience access system 224 animates and scrolls the first set of options 612 and the second set of options 614 (along with the indicator 616) towards the second direction (e.g., towards the right). The speed or rate at which the icons in the scrollable area 610 scroll can correspond to the speed or rate at which the swipe gesture is made. As the scrollable area 610 scrolls to the right, different options corresponding to different previously captured content items in the options 612 are brought into focus (e.g., into the focus area or the central area of the scrollable area 610). When each of the options 612 enters the focus area, the AR experience access system 224 retrieves the corresponding previously captured content item and replaces the rendering of the live video feed in the viewfinder 611 with the retrieved previously captured content item. Different previously captured content items can be stored in different aspect ratios. The viewfinder 611 renders them according to the respective aspect ratios of the previously captured content items. Specifically, when the first option in the options 612 is brought into focus and corresponds to a first previously captured content item captured in a first aspect ratio, the viewfinder 611 is updated to display the first previously captured content item in the first aspect ratio. As the scrollable area 610 continues to scroll in the same direction, the adjacent second option in the options 612 is brought into focus. The second previously captured content item captured in a second aspect ratio and corresponding to the second option currently in focus in the options 612 is presented in the viewfinder 611 in the second aspect ratio.

[0146] For example, as Figure 7 shown in the GUI 700 of, the viewfinder 611 has scrolled to the right, and a given option corresponding to a previously captured content item in the options 612 is brought into focus. The current option 612 in focus is shown by a white border placed around the option 612. The viewfinder 611 is updated to present the previously captured content item 720. The AR experience access system 224 can present a navigation menu 710 that has one or more options related to the previously captured content item 720 corresponding to the option currently in focus.

[0147] The options in the navigation menu 710 may include an edit option, a story option, and a send option. The edit option, when selected, instructs the AR experience access system 224 to present an edit interface for modifying one or more attributes of a previously captured content item 720. For example, in response to receiving a selection of the edit option, a set of edit features 730 may be presented. In some cases, even if the edit option is not selected and in response to the AR experience access system 224 determining that the option currently in focus in the scrollable region 610 corresponds to a previously captured content item 720, the set of edit features 730 may be presented. In some examples, in response to determining that the option currently in focus in the scrollable region 610 corresponds to a previously captured content item 720, the AR experience access system 224 removes the capture option 620 from the display. This is because the capture option 620 cannot be used to perform any function when presenting a previously captured content to the viewfinder 611. On the other hand, when the AR experience access system 224 determines that the option currently in focus in the scrollable region 610 corresponds to an AR experience, the capture option 620 remains displayed to enable applying the AR experience to a live video feed that can be captured.

[0148] In response to receiving a selection of the story option, the AR experience access system 224 may automatically send or share the previously captured content item 720 with various friends or preselected friends of the user on the messaging client 104. In response to receiving a selection of the send option, the AR experience access system 224 may present a user interface for the user to select one or more designated recipients to whom the previously captured content item 720 is to be sent.

[0149] In some examples, the edit feature 730 includes a caption input feature 732. In response to receiving an input selecting the caption input feature 732, the AR experience access system 224 presents Figure 8A a set of GUIs 801, 802, and 803 as shown. Specifically, GUI 801 presents the previously captured content item 720 similar to GUI 700. In this case, the navigation menu 710 only includes the send option. The AR experience access system 224 may receive an input selecting the caption input feature 732 from GUI 801. In response, the AR experience access system 224 presents GUI 802. In GUI 802, a text input area is presented together with the previously captured content item 720 and above the previously captured content item 720. Additionally, the caption input feature 732 is shown above the previously captured content item 720 without any unselected edit features. Additional edit features for caption input (e.g., in the form of a slider) are presented to control the text attributes of the input caption corresponding to the text input in the text input area.

[0150] Once the user adds the desired text with the corresponding text attributes in the text input area, the AR experience access system 224 presents the GUI 803, which allows the user to specify a location within the previously captured content item 720 where the explanatory text corresponding to the text in the text input area is presented. The GUI 803 now includes additional options for: adding to the story, saving the modified previously captured content item 720 with the explanatory text, and / or sending the modified previously captured content item 720 to one or more specified recipients. In this way, the user can quickly and easily select the desired previously captured content item from the scrollable area 610 and edit the desired content item within the same GUI. All of this can be done without navigating to a separate content editing interface and selecting the desired content to be edited within that specific interface. In this way, the efficiency of the entire system is improved, and the number of information pages that need to be searched and navigated through to perform content editing is reduced.

[0151] In some examples, the editing feature 730 includes a sticker or graphic element feature. In response to receiving an input to select the sticker or graphic element feature, the AR experience access system 224 presents Figure 8B a set of GUIs 804, 805, 806, and 807 as shown. Specifically, the GUI 804 presents the previously captured content item 822 (e.g., the content item 720) similar to the GUI 700. The previously captured content item 822 corresponds to the icon 821 that is currently in focus in the scrollable area 610. The AR experience access system 224 can receive an input to select the sticker or graphic element feature from the GUI 804. In response, the AR experience access system 224 presents the GUI 805. In the GUI 805, a set of graphic elements or stickers is presented together with the previously captured content item 822 and is presented above the previously captured content item 822. The GUI 805 includes a search input area that allows the user to enter a search string to identify one or more matching stickers or graphic elements. For example, the GUI 806 can present a set of matching stickers or graphic elements 823.

[0152] Once the user selects a specific sticker or graphic element from the set of matching stickers or graphic elements 823, the AR experience access system 224 presents the GUI 807, which allows the user to specify a location within the previously captured content item 822 where the selected sticker or graphic element is to be placed. For example, the AR experience access system 224 can receive an input that specifies the location 830 within the previously captured content item 822 where the selected graphic element is to be added. The AR experience access system 224 places and modifies the previously captured content item 822 to include the selected graphic element at the specified location.

[0153] The GUI 807 includes options for: adding to a story, saving a previously captured content item 822 with explanatory text and / or sending the modified previously captured content item 822 to one or more specified recipients. In this way, the user can quickly and easily select a desired previously captured content item from the scrollable area 610 and edit the desired content item within the same GUI. All of this can be done without navigating to a separate content editing interface and selecting the desired content to be edited from within that particular interface. In this way, the efficiency of the entire system is increased and the number of information pages that need to be searched and navigated through to perform content editing is reduced.

[0154] Return reference Figure 7 , in response to detecting that a swipe gesture is being made in a first direction (e.g., a finger or stylus is being dragged across the GUI 600 towards the left), the AR experience access system 224 animates and scrolls the first set of options 612 and the second set of options 614 (along with the indicator 616) in the first direction (e.g., towards the left). That is, when one of the options 612 is in focus, the AR experience access system 224 can receive an input including a swipe gesture that is used to scroll the scrollable area 610 towards the options 614. The speed or rate of execution of the gesture can be measured and compared to a reference or threshold. If the speed or rate of execution of the gesture exceeds the reference or threshold, the AR experience access system 224 scrolls the options 612 and the options 614 in the first direction of the swipe gesture at the speed or rate of the swipe gesture, but the AR experience access system 224 automatically stops scrolling the options 612 and the options 614 at a specified point (e.g., the indicator 616). In this case, when automatically stopping the scrolling of the scrollable area 610, the AR experience access system 224 automatically presents a live video feed in the viewfinder 611. The AR experience access system 224 can receive additional input, including an additional swipe gesture in the first direction, which is used to bring the options 614 into focus.

[0155] Machine architecture

[0156] Figure 9is an illustrative representation of a machine 900 within which instructions 908 (e.g., software, program, application, applet, app, or other executable code) can be executed to cause the machine 900 to perform any one or more of the methods discussed herein. For example, the instructions 908 can cause the machine 900 to perform any one or more of the methods described herein. The instructions 908 transform the general, unprogrammed machine 900 into a particular machine 900 programmed to perform the functions described and shown in the manner described. The machine 900 can operate as a stand-alone device or can be coupled (e.g., networked) to other machines. In a networked deployment, the machine 900 can operate in the capacity of a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 900 can include, but is not limited to: server computers, client computers, personal computers (PCs), tablet computers, laptop computers, netbooks, set-top boxes (STBs), personal digital assistants (PDAs), entertainment media systems, cellular telephones, smartphones, mobile devices, wearable devices (e.g., smart watches), smart home devices (e.g., smart appliances), other smart devices, web appliances, network routers, network switches, network bridges, or any machine capable of sequentially or otherwise executing the instructions 908 specifying the actions to be taken by the machine 900. Moreover, although only a single machine 900 is shown, the term "machine" shall also be taken to include a collection of machines that individually or jointly execute the instructions 908 to perform any one or more of the methods discussed herein. For example, the machine 900 can include a client device 102 or any one of a number of server devices forming part of a messaging server system 108. In some examples, the machine 900 can also include both a client system and a server system, where certain operations of a particular method or algorithm are executed on the server side and certain operations of a particular method or algorithm are executed on the client side.

[0157] The machine 900 can include a processor 902, a memory 904, and input / output (I / O) components 938 that can be configured to communicate with each other via a bus 940. In an example, the processor 902 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) can include, for example, a processor 906 and a processor 910 that execute the instructions 908. The term "processor" is intended to include multi-core processors that can include two or more independent processors (sometimes referred to as "cores") that can execute instructions simultaneously. AlthoughFigure 9 Multiple processors 902 are shown, but machine 900 can include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

[0158] Memory 904 includes main memory 912, static memory 914, and storage unit 916, all of which are accessible by processor 902 via bus 940. Main memory 904, static memory 914, and storage unit 916 store instructions 908 embodying any one or more of the methods or functions described herein. The instructions 908 may also reside, completely or partially, within main memory 912, within static memory 914, within a machine-readable medium within storage unit 916, within at least one of the processors within processor 902 (e.g., within a cache memory of the processor), or within any suitable combination thereof during execution by machine 900.

[0159] I / O components 938 can include various components that receive input, provide output, generate output, transmit information, exchange information, capture measurement results, etc. The specific I / O components 938 included in a particular machine will depend on the type of the machine. For example, a portable machine (e.g., a mobile phone) may include a touch input device or other such input mechanism, while a headless server machine will likely not include such a touch input device. It should be understood that I / O components 938 can include Figure 9 many other components not shown. In various examples, I / O components 938 can include user output components 924 and user input components 926. User output components 924 can include visual components (e.g., a display such as a plasma display panel (PDP), a light-emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), acoustic components (e.g., speakers), haptic components (e.g., a vibration motor, a resistance mechanism), other signal generators, etc. User input components 926 can include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, an optical keyboard, or other alphanumeric input components), point-based input components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or other pointing instruments), haptic input components (e.g., physical buttons, a touch screen that provides the location and force of a touch or touch gesture, or other haptic input components), audio input components (e.g., a microphone), etc.

[0160] In other examples, the I / O component 938 can include a biometric component 928, a motion component 930, an environmental component 932, or a location component 934, as well as a variety of other components. For example, the biometric component 928 includes components for detecting expressions (e.g., hand expressions, facial expressions, voice expressions, body postures, or eye tracking), measuring biometric signals (e.g., blood pressure, heart rate, body temperature, sweating, or brain waves), identifying people (e.g., voice recognition, retina recognition, facial recognition, fingerprint recognition, or electroencephalogram-based recognition), etc. The motion component 930 includes an acceleration sensor component (e.g., an accelerometer), a gravity sensor component, and a rotation sensor component (e.g., a gyroscope).

[0161] The environmental component 932 includes, for example, one or more camera devices (with still image / photo and video capabilities), a lighting sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers for detecting ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an acoustic sensor component (e.g., one or more microphones for detecting background noise), a proximity sensor component (e.g., an infrared sensor for detecting nearby objects), a gas sensor (e.g., a gas detection sensor for detecting the concentration of hazardous gases for safety or measuring pollutants in the atmosphere), or other components that can provide an indication, measurement, or signal corresponding to the surrounding physical environment.

[0162] Regarding the camera device, the client device 102 can have a camera device system that includes, for example, a front camera device on the front surface of the client device 102 and a rear camera device on the rear surface of the client device 102. The front camera device can be used, for example, to capture still images and videos of the user of the client device 102 (e.g., "selfies"), and then the still images and videos can be enhanced using the above-described enhancement data (e.g., filters). The rear camera device can be used, for example, to capture still images and videos in a more traditional camera device mode, where the images are similarly enhanced using the enhancement data. In addition to the front camera device and the rear camera device, the client device 102 can also include a 360° camera device for capturing 360° photos and videos.

[0163] Furthermore, the camera device system of the client device 102 can include a dual rear camera device (e.g., a primary camera device and a depth sensing camera device), or even a triple, quadruple, or quintuple rear camera device configuration on the front and rear sides of the client device 102. For example, these multi-camera device systems can include a wide-angle camera device, an ultra-wide-angle camera device, a telephoto camera device, a macro camera device, and a depth sensor.

[0164] The position component 934 includes a position sensor component (e.g., a GPS receiver component), an altitude sensor component (e.g., an altimeter or barometer that detects air pressure, from which altitude can be obtained), an orientation sensor component (e.g., a magnetometer), and the like.

[0165] A variety of techniques can be used to implement communication. The I / O component 938 also includes a communication component 936 that is operable to couple the machine 900 to the network 920 or the device 922 via a respective coupling or connection. For example, the communication component 936 can include a network interface component that interfaces with the network 920 or other suitable devices. In another example, the communication component 936 can include a wired communication component, a wireless communication component, a cellular communication component, a near field communication (NFC) component, a Bluetooth component (e.g., Bluetooth low energy), a Wi-Fi component, and other communication components for providing communication via other modalities. The device 922 can be another machine or any of a variety of peripheral devices (e.g., a peripheral device coupled via USB).

[0166] In addition, the communication component 936 can detect indicators or include components operable to detect indicators. For example, the communication component 936 can include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting one-dimensional barcodes such as Universal Product Code (UPC) barcodes, multi-dimensional barcodes such as Quick Response (QR) codes, Aztec codes, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, UCC RSS-2D barcodes, and other optical codes), or an acoustic detection component (e.g., a microphone for identifying an audio signal of a tag). In addition, various information can be derived via the communication component 936, such as a position obtained via Internet Protocol (IP) geolocation, a position obtained via Wi-Fi signal triangulation, a position obtained via detecting an NFC beacon signal that can indicate a specific location, and the like.

[0167] Various memories (e.g., main memory 912, static memory 914, and the memory of the processor 902) and the storage unit 916 can store one or more sets of instructions and data structures (e.g., software) implemented or used by any of the methods or functions described herein. When executed by the processor 902, these instructions (e.g., instruction 908) cause the various operations to implement the disclosed examples.

[0168] Instructions 908 can be sent or received over network 920 via a network interface device (e.g., the network interface component included in communication component 936) using a transmission medium and any one of a number of well-known transmission protocols (e.g., Hypertext Transfer Protocol (HTTP)). Similarly, instructions 908 can be sent or received using a transmission medium via a coupling (e.g., a peer-to-peer coupling) to device 922.

[0169] Software architecture

[0170] Figure 10 FIG. 1000 is a block diagram illustrating a software architecture 1004 that can be installed on any one or more of the devices described herein. The software architecture 1004 is supported by hardware, such as a machine 1002 that includes a processor 1020, a memory 1026, and I / O components 1038. In this example, the software architecture 1004 can be conceptualized as a stack of layers, where each layer provides a specific function. The software architecture 1004 includes the following layers, such as an operating system 1012, libraries 1010, frameworks 1008, and applications 1006. In operation, an application 1006 activates an API call 1050 through the software stack and receives a message 1052 in response to the API call 1050.

[0171] The operating system 1012 manages hardware resources and provides common services. The operating system 1012 includes, for example, a kernel 1014, services 1016, and drivers 1022. The kernel 1014 serves as an abstraction layer between the hardware and other software layers. For example, the kernel 1014 provides functions such as memory management, processor management (e.g., scheduling), component management, networking, and security settings. The services 1016 can provide other common services for other software layers. The drivers 1022 are responsible for controlling or interfacing with the underlying hardware. For example, the drivers 1022 can include a display driver, a camera driver, a Bluetooth or Bluetooth Low Energy driver, a flash driver, a serial communication driver (e.g., a USB driver), a WI-FI driver, an audio driver, a power management driver, etc.

[0172] Library 1010 provides common low-level infrastructure used by application 1006. Library 1010 may include system library 1018 (e.g., C standard library), which provides functions such as memory allocation functions, string manipulation functions, mathematical functions, etc. In addition, library 1010 may include API library 1024, such as media libraries (e.g., libraries for supporting the presentation and manipulation of various media formats, such as Moving Picture Experts Group-4 (MPEG4), High Efficiency Video Coding (H.264 or AVC), Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio codec, Joint Photographic Experts Group (JPEG or JPG), or Portable Network Graphics (PNG)), graphics libraries (e.g., OpenGL framework for presenting graphical content in two-dimensional (2D) and three-dimensional (3D) on a display), database libraries (e.g., SQLite that provides various relational database functions), web libraries (e.g., WebKit that provides web browsing functions), etc. Library 1010 may also include various other libraries 1028 to provide many other APIs to application 1006.

[0173] Framework 1008 provides common high-level infrastructure used by application 1006. For example, framework 1008 provides various Graphical User Interface (GUI) functions, high-level resource management, and high-level location services. Framework 1008 may provide a wide range of other APIs that can be used by application 1006, some of which may be specific to a particular operating system or platform.

[0174] In an example, application 1006 may include home application 1036, contacts application 1030, browser application 1032, book reader application 1034, location application 1042, media application 1044, messaging application 1046, game application 1048, and various other applications such as external application 1040. Application 1006 is a program that executes functions defined in a program. One or more of the applications in application 1006 can be created using various programming languages, such as object-oriented programming languages (e.g., Objective-C, Java, or C++) or procedural programming languages (e.g., C or assembly language). In a particular example, external application 1040 (e.g., an application developed using the ANDROID or IOS Software Development Kit (SDK) by an entity other than the vendor of a particular platform) can be mobile software running on a mobile operating system such as IOS, ANDROID, WINDOWS Phone, or other mobile operating systems. In this example, external application 1040 can activate API call 1050 provided by operating system 1012 to facilitate the functions described herein.

[0175] Glossary

[0176] "Carrier signal" means any intangible medium that can store, encode, or carry instructions executed by a machine and includes digital or analog communication signals or other intangible media that facilitate the communication of such instructions. Instructions can be sent or received over a network via a network interface device using a transmission medium.

[0177] "Client device" means any machine that interfaces with a communication network to obtain resources from one or more server systems or other client devices. A client device can be, but is not limited to, a mobile phone, desktop computer, laptop computer, portable digital assistant (PDA), smartphone, tablet computer, ultrabook, netbook, laptop, multiprocessor system, microprocessor-based or programmable consumer electronics, game console, set-top box, or any other communication device that a user can use to access a network.

[0178] "Communication network" means one or more portions of a network, which can be an ad hoc network, intranet, extranet, virtual private network (VPN), local area network (LAN), wireless LAN (WLAN), wide area network (WAN), wireless WAN (WWAN), metropolitan area network (MAN), the Internet, a portion of the Internet, a portion of the public switched telephone network (PSTN), a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, a Wi-Fi network, other types of networks, or a combination of two or more such networks. For example, a network or a portion of a network can include a wireless network or a cellular network, and the coupling can be a code division multiple access (CDMA) connection, a global system for mobile communications (GSM) connection, or other types of cellular or wireless couplings. In this example, the coupling can implement any data transfer technology among various types of data transfer technologies, such as single-carrier radio transmission technology (1xRTT), evolved data optimized (EVDO) technology, general packet radio service (GPRS) technology, enhanced data rate for GSM evolution (EDGE) technology, 3rd Generation Partnership Project (3GPP) including 3G, 4th Generation Wireless (4G) network, universal mobile telecommunications system (UMTS), high-speed packet access (HSPA), worldwide interoperability for microwave access (WiMAX), long term evolution (LTE) standard, other data transfer technologies defined by various standards-setting organizations, other long-distance protocols, or other data transfer technologies.

[0179] "Component" refers to a logical, device, or physical entity with boundaries defined by function or subroutine calls, branch points, APIs, or other technical definitions that provide partitioning or modularization for a particular processing or control function. Components can be combined with other components via their interfaces to perform machine processing. A component can be an encapsulated functional hardware unit designed to be used with other components and can be part of a program that typically performs a specific function among related functions.

[0180] Components can form software components (e.g., code implemented on a machine-readable medium) or hardware components. A "hardware component" is a tangible unit capable of performing certain operations and can be configured or arranged in some physical manner. In various examples, one or more computer systems (e.g., stand-alone computer systems, client computer systems, or server computer systems) or one or more hardware components of a computer system (e.g., a processor or a group of processors) can be configured by software (e.g., an application or a portion of an application) to operate to perform certain operations described herein as a hardware component.

[0181] A hardware component can also be implemented mechanically, electronically, or any suitable combination thereof. For example, a hardware component can include dedicated circuitry or logic permanently configured to perform certain operations. A hardware component can be a dedicated processor, such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). A hardware component can also include programmable logic or circuitry that is temporarily configured by software to perform certain operations. For example, a hardware component can include software executed by a general-purpose processor or other programmable processor. Once configured by such software, the hardware component becomes a particular machine (or a particular component of a machine) uniquely customized to perform the configured function and is no longer a general-purpose processor. It should be recognized that the decision of whether to implement a hardware component mechanically in dedicated and permanently configured circuitry or in temporarily configured (e.g., software-configured) circuitry can be made based on cost and time considerations. Thus, the phrase "hardware component" (or "hardware-implemented component") should be understood to include a tangible entity, i.e., an entity that is physically constructed, permanently configured (e.g., hard-wired), or temporarily configured (e.g., programmed) to operate in some manner or perform certain operations described herein.

[0182] Consider an example where a hardware component is temporarily configured (e.g., programmed) without configuring or instantiating every hardware component at any given time. For example, in a case where a hardware component includes a general-purpose processor that is configured by software to become a dedicated processor, the general-purpose processor can be configured at different times to be respective different dedicated processors (e.g., including different hardware components). The software accordingly configures one or more specific processors to form a specific hardware component at one time and different hardware components at different times.

[0183] Hardware components can provide information to other hardware components and receive information from other hardware components. Thus, the described hardware components can be considered to be communicatively coupled. In a case where multiple hardware components are present simultaneously, communication can be achieved by signal transmission between two or more of the hardware components (e.g., via appropriate circuitry and buses). In an example where multiple hardware components are configured or instantiated at different times, such communication between the hardware components can be achieved, for example, by storing information in a memory structure accessible to the multiple hardware components and retrieving the information from the memory structure. For example, one hardware component can perform an operation and store the output of the operation in a memory device to which it is communicatively coupled. Then, another hardware component can access the memory device at a subsequent time to retrieve the stored output and process the stored output. Hardware components can also initiate communication with input devices or output devices and can operate on resources (e.g., a collection of information).

[0184] The various operations of the example methods described herein can be performed, at least in part, by one or more processors temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors may constitute processor-implemented components that operate to perform one or more of the operations or functions described herein. As used herein, "processor-implemented components" refers to hardware components implemented using one or more processors. Similarly, the methods described herein can be at least in part processor-implemented, where a particular one or more processors are examples of hardware. For example, at least some of the operations of the method can be performed by one or more processors 902 or processor-implemented components. Additionally, the one or more processors may also operate to support the execution of the relevant operations in a "cloud computing" environment or as "software as a service" (SaaS) operations. For example, at least some of the operations can be performed by a group of computers (as examples of machines including processors), where the operations are accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., APIs). The execution of certain operations of the method can be distributed among the processors, not residing only within a single machine but being deployed across several machines. In some examples, the processor or processor-implemented components may be located in a single geographical location (e.g., within a home environment, an office environment, or a server farm). In other examples, the processor or processor-implemented components may be distributed across several geographical locations.

[0185] "Computer-readable storage medium" refers to both machine storage media and transmission media. Thus, these terms include both storage devices / media and carrier / modulated data signals. The terms "machine-readable medium", "computer-readable medium", and "device-readable medium" mean the same thing and may be used interchangeably in this disclosure.

[0186] "Ephemeral message" refers to a message that is accessible for a limited duration. Ephemeral messages can be text, images, video, etc. The access time for an ephemeral message can be set by the message sender. Alternatively, the access time can be a default setting or a setting specified by the recipient. Regardless of the setting technique, the message is transient.

[0187] "Machine storage medium" means a single or multiple storage devices and media (e.g., centralized or distributed databases, and associated caches and servers) that store executable instructions, routines, and data. Thus, the term should be considered to include, without limitation, solid-state memory as well as optical and magnetic media, including memory internal or external to a processor. Specific examples of machine storage media, computer storage media, and device storage media include: non-volatile memory, including, for example, semiconductor memory devices such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), FPGA, and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The terms "machine storage medium", "device storage medium", and "computer storage medium" mean the same thing and may be used interchangeably in this disclosure. The terms "machine storage medium", "computer storage medium", and "device storage medium" expressly exclude carrier waves, modulated data signals, and other such media, at least some of which are covered by the term "signal medium".

[0188] "Non-transitory computer-readable storage medium" means a tangible medium that can store, encode, or carry instructions executable by a machine.

[0189] "Signal medium" means a medium that can store, encode, or carry instructions executable by a machine and includes digital or analog communication signals or other intangible media that facilitate the communication of software or data. The term "signal medium" should be considered to include any form of modulated data signal, carrier wave, etc. The term "modulated data signal" refers to a signal that sets or changes one or more of its characteristic sets to encode information therein. The terms "transmission medium" and "signal medium" mean the same thing and may be used interchangeably in this disclosure.

[0190] Without departing from the scope of this disclosure, changes and modifications can be made to the disclosed examples. These and other changes or modifications are intended to be included within the scope of this disclosure as expressed in the appended claims.

Claims

1. A method, comprising: Presenting, via a messaging application, a first plurality of options associated with previously captured content items and a second plurality of options associated with an augmented reality (AR) experience in a scrollable region above a live video feed being displayed in a graphical user interface (GUI) including a viewfinder; Scrolling the first plurality of options of the scrollable region together with the second plurality of options to bring one of the first plurality of options or one of the second plurality of options into focus; And In response to scrolling the first plurality of options of the scrollable region together with the second plurality of options: Modifying the configuration of the GUI in a first manner in response to determining that the one of the first plurality of options is currently in focus; And Modifying the configuration of the GUI in a second manner in response to determining that the one of the second plurality of options is currently in focus.

2. The method according to claim 1, further comprising: Presenting an indicator of the live video feed within the scrollable region between the first plurality of options and the second plurality of options.

3. The method according to any one of claims 1 to 2, wherein Each of the first plurality of options is represented by a first type of visual indicator, and each of the second plurality of options is represented by a second type of visual indicator.

4. The method according to any one of claims 1 to 3, wherein The previously captured content items include at least one of a previously captured image or a previously captured video.

5. The method according to any one of claims 1 to 4, wherein, The previously captured content items associated with the first plurality of options correspond to a subset of a plurality of content items previously stored by the client device, and the method further comprises: Presenting, above the live video feed being displayed in the GUI and separately from the scrollable region, an option for accessing the plurality of content items.

6. The method according to claim 5, further comprising: Selecting the subset of the plurality of content items from the plurality of content items previously stored by the client device based on one or more criteria, the one or more criteria including a time associated with each of the subset of the plurality of content items, a location associated with each of the subset of the plurality of content items, and a content item type associated with each of the subset of the plurality of content items.

7. The method according to any one of claims 1 to 6, wherein The AR experience associated with the second plurality of options corresponds to a subset of a plurality of AR experiences available on the messaging application, and the method further comprises: Presenting, above the live video feed being displayed in the GUI and separately from the scrollable region, an option for searching the subset of the plurality of AR experiences.

8. The method according to claim 7, wherein The option for searching the subset of the plurality of AR experiences includes a first option, and the method further comprises presenting, above the live video feed being displayed in the GUI and separately from the scrollable region, a second option for accessing a plurality of content items.

9. The method according to any one of claims 1 to 8, further comprising: Receiving an input including a swipe gesture in a given direction, wherein in response to the swipe gesture, the first plurality of options and the second plurality of options are scrolled together in the given direction.

10. The method according to claim 9 further comprises: Receiving a sliding gesture in a second direction; Determining that a rate of execution of the sliding gesture in the second direction exceeds a reference rate; And In response to determining that the rate of execution of the sliding gesture in the second direction exceeds the reference rate, scrolling the first plurality of options together with the second plurality of options in the second direction until a predetermined termination point, the predetermined termination point being between a first option of the first plurality of options and a first option of the second plurality of options.

11. The method according to any one of claims 1 to 10, wherein The predetermined termination point corresponds to an indicator of the live video feed, and the indicator is displayed within the scrollable area between the first plurality of options and the second plurality of options.

12. The method according to any one of claims 1 to 11 further comprises: Identifying a single option currently in focus among the first plurality of options; And Using a display of a single previously captured content item associated with the single option currently in focus among the first plurality of options to replace the display of the live video feed in the viewfinder.

13. The method according to any one of claims 1 to 12, wherein Modifying the configuration of the GUI in the first manner further comprises: Presenting a navigation area associated with the one currently in focus among the first plurality of options, the navigation area including one or more options associated with a single previously captured content item corresponding to the one currently in focus among the first plurality of options.

14. The method according to claim 13, wherein, The one or more options include an edit option for modifying the single previously captured content item, a category-based search option, and a content sharing option for sending the single previously captured content item to one or more other users.

15. The method according to any one of claims 1 to 14 further comprises: Receiving an input selecting the edit option; And In response to receiving the input, presenting one or more image editing features corresponding to the edit option above the single previously captured content item.

16. The method according to claim 15, wherein, The edit option includes a caption tool, and the method further comprises: presenting a text input area above the single previously captured content item and presenting a text attribute selection area above the single previously captured content item, and wherein the edit option includes a graphic element option, and the method further comprises: presenting a plurality of options for adding corresponding graphic elements above the single previously captured content item.

17. The method according to any one of claims 1 to 16 further comprises: Before modifying the configuration of the GUI in the first manner, presenting a capture option associated with the live video feed being displayed in the GUI, wherein an image or video including the live video feed is stored in response to receiving an input selecting the capture option; and In response to modifying the configuration of the GUI in the first manner, removing the display of the capture option.

18. The method according to any one of claims 1 to 17, wherein Modifying the configuration of the GUI in the second manner further comprises: Populate capture options associated with the live video feed being displayed in the GUI with a representation of a separate AR experience corresponding to the one of the second plurality of options that is currently in focus; and Present a navigation area associated with the one of the second plurality of options that is currently in focus, the navigation area including one or more options associated with the separate AR experience, the one or more options including an option for modifying an attribute of the separate AR experience, an option for activating the separate AR experience, and an option for saving the separate AR experience.

19. A system, comprising: A processor configured to perform operations, the operations including: Present, via a messaging application, in a scrollable area above a live video feed being displayed in a graphical user interface (GUI) including a viewfinder, a first plurality of options associated with previously captured content items and a second plurality of options associated with augmented reality (AR) experiences; Scroll the first plurality of options of the scrollable area together with the second plurality of options to bring one of the first plurality of options or one of the second plurality of options into focus; and In response to scrolling the first plurality of options of the scrollable area together with the second plurality of options: In response to determining that one of the first plurality of options is currently in focus, modify the configuration of the GUI in a first manner; and In response to determining that one of the second plurality of options is currently in focus, modify the configuration of the GUI in a second manner.

20. A non-transitory machine-readable storage medium, the non-transitory machine-readable storage medium including instructions that, when executed by one or more processors of a machine, cause the machine to perform operations, the operations including: Present, via a messaging application, in a scrollable area above a live video feed being displayed in a graphical user interface (GUI) including a viewfinder, a first plurality of options associated with previously captured content items and a second plurality of options associated with augmented reality (AR) experiences; Scroll the first plurality of options of the scrollable area together with the second plurality of options to bring one of the first plurality of options or one of the second plurality of options into focus; And In response to scrolling the first plurality of options of the scrollable area together with the second plurality of options: In response to determining that one of the first plurality of options is currently in focus, modify the configuration of the GUI in a first manner; And In response to determining that one of the second plurality of options is currently in focus, modify the configuration of the GUI in a second manner.

Images