Automatic tagging of content items

Abstract

Aspects of the present disclosure relate to a system for tagging content items. The system generates a content item for display through a messaging application. The system receives, through a messaging application, an input to select a visual element and determines that the visual element visually represents a plurality of users. The system adds a visual element to the content item. In response to adding the visual element to the content item and based on determining that the visual element represents the plurality of users, the system links the content item with the visual element to one or more respective accounts on a messaging application of one or more of the plurality of users.

Description

[0001] Priority Claim

[0002] This application claims the benefit of U.S. Patent Application No. 18 / 071,452, filed Nov. 29, 2022, the entire content of which is incorporated herein by reference. Technical Field

[0003] The present disclosure generally relates to tagging content using a messaging application. Background Art

[0004] Messaging applications allow users to communicate with each other in a variety of different ways. Users can send chat messages to one or more other users. Users can also access certain group pages, view those pages, and post content to those pages. Some messaging applications allow users to link content items to other users. Brief Description of the Drawings

[0005] In the drawings (which are not necessarily drawn to scale), like numerals may describe similar components in different views. To easily identify the discussion of any particular element or act, one or more of the most significant digits in the reference numeral refers to the figure number in which the element is first introduced. Some non-limiting examples are shown in the figures of the drawings, in which:

[0006] Figure 1 is a graphical representation of a networked environment in which the present disclosure may be deployed, in accordance with some examples.

[0007] Figure 2 is a graphical representation of a messaging client application, in accordance with some examples.

[0008] Figure 3 is a graphical representation of a data structure maintained in a database, in accordance with some examples.

[0009] Figure 4 is a graphical representation of a message, in accordance with some examples.

[0010] Figures 5 to 8 is a graphical representation of an output of a content tagging system, in accordance with some examples.

[0011] Figure 9 is a flowchart showing an example operation of a content tagging system, in accordance with some examples.

[0012] Figure 10 is a graphical representation of a machine in the form of a computer system within which a set of instructions may be executed to cause the machine to perform any one or more of the methods discussed herein.

[0013] Figure 11 A block diagram showing a software architecture in which an example can be implemented. Detailed implementation

[0014] The following description includes systems, methods, techniques, instruction sequences, and computer program products embodying illustrative examples of the present disclosure. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide an understanding of the various examples. However, it will be apparent to those skilled in the art that the examples may be practiced without these specific details. Generally, well-known instruction instances, protocols, structures, and techniques are not necessarily shown in detail.

[0015] Messaging applications typically enable end users to communicate with each other in a variety of different ways. Sometimes users create content items and add various graphical elements to the content items. The graphical elements can be associated with multiple other users, so the user who creates the content item needs to manually identify the other users to link the content to these users. For example, instead of sending an image with stickers to each of the users individually, a user can add many stickers that the users are interested in to an image, and the user can type the names and accounts of each of the users. In response, the user can then easily search for and access the image with the stickers. Identifying users who may be interested in the stickers and then having to manually search the users' accounts is very time-consuming and inefficient. As a result, many content items are not tagged and not linked to users who may be interested in the content items. This makes users unaware that they are associated with certain content items and results in missed opportunities for conversations and participation on the messaging application. Browsing through many information pages to find users and corresponding accounts who may be interested in the stickers can be frustrating and extremely inefficient.

[0016] The disclosed technology solves these technical problems by providing a system that automatically links content items having visual elements associated with multiple users to the corresponding accounts of the multiple users. Specifically, the disclosed technology generates a content item for display by a messaging application and receives an input selecting a visual element by the messaging application. The disclosed technology determines that the visual element visually represents multiple users, such as a sticker that includes and simultaneously depicts the corresponding avatars of multiple users. The disclosed technology adds the visual element to the content item and, in response to adding the visual element to the content item and based on determining that the visual element represents multiple users, links the content item with the visual element to the corresponding accounts on the messaging application of the multiple users.

[0017] In these ways, the techniques described herein improve the efficiency of using an electronic device and the overall experience of a user when using the electronic device. Additionally, by providing a fast way to automatically tag content items that may be of interest to multiple users, the total amount of system resources required to complete a task is also reduced because the number of information pages that need to be accessed and viewed is decreased.

[0018] Networked computing environment

[0019] Figure 1 is a block diagram showing an example messaging system 100 for exchanging data (e.g., messages and associated content) over a network. Messaging system 100 includes multiple instances of client devices 102, each of the multiple instances hosting a number of applications, including a messaging client 104 and other external applications 109 (e.g., third-party applications). Each messaging client 104 is communicatively coupled via a network 112 (e.g., the Internet) to other instances of messaging client 104 (e.g., hosted on corresponding other client devices 102), a messaging server system 108, and an external app server 110. Messaging client 104 can also communicate with locally hosted third-party applications (also referred to as "external applications" and "external apps") 109 using an application programming interface (API). Each user of client device 102 has a corresponding account of messaging client 104 on messaging system 100.

[0020] In some examples, client device 102 can include AR glasses or an AR head-mounted device in which virtual content is displayed within the lenses of the glasses while the user views the real-world environment through the lenses. For example, an image can be presented on a transparent display, which allows the user to view both the content presented on the display and real-world objects simultaneously.

[0021] Messaging client 104 (sometimes referred to as the client application) is capable of communicating and exchanging data with other messaging clients 104 and with the messaging server system 108 via network 112. The data exchanged between messaging clients 104 and between messaging client 104 and the messaging server system 108 includes functionality (e.g., commands to activate functionality) and payload data (e.g., text, audio, video, or other multimedia data).

[0022] The messaging server system 108 provides server-side functionality to a particular messaging client 104 via a network 112. Although certain functions of the messaging system 100 are described herein as being performed by the messaging client 104 or by the messaging server system 108, the location of certain functions within the messaging client 104 or within the messaging server system 108 can be a design choice. For example, it may be technically preferable to initially deploy certain technologies and functions within the messaging server system 108, but later migrate the technologies and functions to the messaging client 104 where the client device 102 has sufficient processing power.

[0023] The messaging server system 108 supports various services and operations provided to the messaging client 104. Such operations include sending data to the messaging client 104, receiving data from the messaging client 104, and processing data generated by the messaging client 104. By way of example, the data can include message content, client device information, geographical location information, media enhancements and overlays, message content persistence conditions, social network information, and live event information. Data exchange within the messaging system 100 is invoked and controlled through functions available via the user interface of the messaging client 104.

[0024] Turning now specifically to the messaging server system 108, an API server 116 is coupled to an application server 114 and provides a programming interface to the application server 114. The application server 114 is communicatively coupled to a database server 120 that facilitates access to a database 126 that stores data associated with messages processed by the application server 114. Similarly, a web server 128 is coupled to the application server 114 and provides a web-based interface to the application server 114. To that end, the web server 128 processes incoming network requests via the Hypertext Transfer Protocol (HTTP) and several other related protocols.

[0025] The API server 116 receives and sends message data (e.g., commands and message payloads) between the client device 102 and the application server 114. Specifically, the API server 116 provides a set of interfaces (e.g., routines and protocols) that can be invoked or queried by the messaging client 104 to activate the functions of the application server 114. The API server 116 discloses various functions supported by the application server 114, including: account registration; login functionality; sending messages from a particular messaging client 104 to another messaging client 104 via the application server 114; sending media files (e.g., images or videos) from the messaging client 104 to the messaging server 118 for possible access by another messaging client 104; setting a collection of media data (e.g., a story); retrieving a friend list of the user of the client device 102; retrieving such a collection; retrieving messages and content; adding and deleting entities (e.g., friends) in an entity graph (e.g., a social graph); locating friends in the social graph; and opening application events (e.g., related to the messaging client 104).

[0026] The application server 114 hosts several server applications and subsystems, including, for example, the messaging server 118, the image processing server 122, and the social network server 124. The messaging server 118 implements several message processing techniques and functions, particularly those related to the aggregation and other processing of the content (e.g., text and multimedia content) included in messages received from multiple instances of the messaging client 104. As will be described in further detail, text and media content from multiple sources can be aggregated into collections of content (e.g., called stories or galleries). These collections are then made available to the messaging client 104. Given the hardware requirements for other processor - and memory - intensive data processing, such processing of data can also be performed by the messaging server 118 on the server side.

[0027] The application server 114 also includes an image processing server 122 that is dedicated to performing various image processing operations, typically with respect to images or videos within the payload of messages sent from or received at the messaging server 118.

[0028] The image processing server 122 is used to implement the enhancement system 208( Figure 2The scanning function (shown in the figure). The scanning function includes activating and providing one or more AR experiences on the client device 102 when an image is captured by the client device 102. Specifically, the messaging client 104 on the client device 102 can be used to activate the imaging device. The imaging device displays one or more real-time images or videos and one or more icons or identifiers of one or more AR experiences to the user.

[0029] The social network server 124 supports various social networking functions and services and makes these functions and services available to the messaging server 118. To this end, the social network server 124 maintains and accesses the entity graph 308 in the database 126 (as Figure 3 shown in the figure). Examples of functions and services supported by the social network server 124 include identifying other users with whom a particular user in the messaging system 100 has a relationship or whom the particular user is "following", and also include identifying the interests of a particular user and other entities.

[0030] Returning to the messaging client 104, the features and functions of external resources (e.g., third-party applications 109 or mini-programs) are available to the user via the interface of the messaging client 104. The messaging client 104 receives a user selection of an option for initiating or accessing the features of an external resource (e.g., a third-party resource) such as the external application 109. The external resource can be a third-party application (external application 109) installed on the client device 102 (e.g., a "local application"), or a scaled-down version of a third-party application hosted on the client device 102 or away from the client device 102 (e.g., on an external resource or application server 110) (e.g., a "mini-program"). The scaled-down version of the third-party application includes a subset of the features and functions of the third-party application (e.g., the full-scale native version of a third-party stand-alone application) and is implemented using a markup language document. In one example, the scaled-down version of the third-party application (e.g., a "mini-program") is a web-based markup language version of the third-party application and is embedded in the messaging client 104. In addition to using a markup language document (e.g., an *ml file), the mini-program can include a scripting language (e.g., a.*js file or a.json file) and a stylesheet (e.g., an *ss file).

[0031] In response to receiving a user selection of an option for a feature to initiate or access an external resource (external app 109), the messaging client 104 determines whether the selected external resource is a web-based external resource or a locally installed external application. In some cases, an external application 109 locally installed on the client device 102 can be initiated independently of and separately from the messaging client 104, for example, by selecting an icon corresponding to the external application 109 on the home screen of the client device 102. A scaled-down version of such an external application can be initiated or accessed via the messaging client 104, and in some examples, no part (or a limited part) of the scaled-down external application can be accessed outside the messaging client 104. The scaled-down external application can be initiated by receiving a markup language document associated with the scaled-down external application from the external app server 110 via the messaging client 104 and processing such a document.

[0032] In response to determining that the external resource is a locally installed external application 109, the messaging client 104 instructs the client device 102 to initiate the external application 109 by executing locally stored code corresponding to the external application 109. In response to determining that the external resource is a web-based resource, the messaging client 104 communicates with the external app server 110 to obtain a markup language document corresponding to the selected resource. The messaging client 104 then processes the obtained markup language document to render the web-based external resource within the user interface of the messaging client 104.

[0033] The messaging client 104 can notify a user of the client device 102 or other users related to such a user (e.g., "friends") of activities occurring in one or more external resources. For example, the messaging client 104 can provide notifications to participants in a conversation (e.g., a chat session) in the messaging client 104 regarding the current or recent use of an external resource by one or more members of a user group. One or more users can be invited to join the external resource of the activity or an external resource that was recently used but is currently inactive (in the group of friends) can be launched. The external resource can provide the ability for participants in the conversation (each using the corresponding messaging client messaging client 104) to share items, conditions, statuses, or locations within the external resource with one or more members of the user group entering the chat session. The shared item can be an interactive chat card that the members of the chat can interact with, e.g., to launch the corresponding external resource, view specific information within the external resource, or bring the members of the chat to a specific location or status within the external resource. Within a given external resource, a response message can be sent to a user on the messaging client 104. The external resource can selectively include different media items in the response based on the current context of the external resource.

[0034] The messaging client 104 can present a list of available external resources (e.g., third-party or external applications 109 or applets) to the user to launch or access a given external resource. The list can be presented in a context-sensitive menu. For example, the icons representing different external applications in the external application 109 (or applet) can vary based on how the user launches the menu (e.g., from a conversation interface or from a non-conversation interface).

[0035] In some examples, the messaging client 104 allows a user to automatically tag a given content item (e.g., an image or video) or link the given content item to multiple users or to one or more other users by placing a visual element on the given content item. The visual element can be associated with each of the multiple users and visually depict each of the multiple users, the multiple users including the user who is adding the visual element to the content item. In response to receiving an input from the user to place the visual element on the content item, the messaging client 104 automatically obtains the accounts associated with each user, each user associated with the visual element. The messaging client 104 automatically links the obtained accounts, the obtained accounts can include one or more accounts of each user associated with the visual element. This can cause a notification to be presented on the client devices 102 of the linked user accounts informing the respective users that the visual element has been added to the content item associated with the user.

[0036] Below inFigure 2 Other details of the functions performed by the messaging client 104 are discussed in connection with the content tagging system 224.

[0037] System Architecture

[0038] Figure 2 is a block diagram showing additional details of the messaging system 100 according to some examples. Specifically, the messaging system 100 is shown as including a messaging client 104 and an application server 114. The messaging system 100 includes a number of subsystems that are supported by the messaging client 104 on the client side and by the application server 114 on the server side. These subsystems include, for example, a transient timer system 202, a collection management system 204, an enhancement system 208, a map system 210, a game system 212, an external resource system 220, and a content tagging system 224.

[0039] The transient timer system 202 is responsible for implementing temporary or time-limited access to content by the messaging client 104 and the messaging server 118. The transient timer system 202 includes a number of timers that selectively enable access to messages and associated content (e.g., for presentation and display) via the messaging client 104 based on the duration and display parameters associated with a message or a collection of messages (e.g., a story). Additional details of the operation of the transient timer system 202 are provided below.

[0040] The collection management system 204 is responsible for managing sets or collections of media (e.g., collections of text, image, video, and audio data). Collections of content (e.g., messages, including images, videos, text, and audio) can be organized into "event libraries" or "event stories". Such collections can be made available for a specified period of time (e.g., the duration of the event related to the content). For example, content related to a concert can be made available as a "story" for the duration of that concert. The collection management system 204 can also be responsible for publishing an icon notifying the user interface of the messaging client 104 of the presence of a particular collection.

[0041] The collection management system 204 also includes a curation interface 206 that allows collection managers to manage and curate specific collections of content. For example, the curation interface 206 enables event organizers to curate collections of content related to a particular event (e.g., delete inappropriate content or redundant messages). Additionally, the collection management system 204 employs machine vision (or image recognition technology) and content rules to automatically curate content collections. In certain examples, compensation can be paid to users for including user-generated content in a collection. In such cases, the collection management system 204 operates to automatically pay such users for the use of their content.

[0042] Enhancement system 208 provides various functions that enable a user to enhance (e.g., annotate or otherwise modify or edit) media content associated with a message. For example, enhancement system 208 provides functions related to generating and publishing a media overlay for a message to be processed by messaging system 100. Enhancement system 208 operably provides a media overlay or enhancement (e.g., an image filter) to messaging client 104 based on the geographical location of client device 102. In another example, enhancement system 208 operably provides a media overlay to messaging client 104 based on other information such as the social network information of a user of client device 102. The media overlay may include audio as well as visual content and visual effects.

[0043] Examples of audio and visual content include pictures, text, logos, animations, and sound effects. Examples of visual effects (or visual elements) include color overlays. The audio and visual content or visual effects or visual elements may be applied to a media content item (e.g., a photo) at client device 102. For example, the media overlay may include text, graphic elements, or images that may be superimposed over a photo taken by client device 102. In another example, the media overlay includes a location identifier overlay (e.g., Venice Beach), a name of a live event, or a business name overlay (e.g., Beach Café). In another example, enhancement system 208 uses the geographical location of client device 102 to identify a media overlay that includes the business name at the geographical location of client device 102. The media overlay may include other logos associated with the business. The media overlay may be stored in database 126 and accessed via database server 120.

[0044] In some cases, the visual element may include a graphic that visually depicts and represents multiple users. Such a visual element may include metadata that identifies the user accounts of each of the multiple users. For example, the metadata may include the address, phone number, username, or other unique identifier of each of the multiple users. When a first user of the multiple users adds such a visual element to a content item (e.g., an image or video), the metadata may be accessed to automatically link the content item with the visual element added to the user accounts specified in the metadata of the multiple users. This causes the user accounts to present a notification or alert to the multiple users indicating that the content item has been modified with the visual element associated with the respective user. The user may then select an option to access or view the content item with the visual element and / or establish an augmented reality (AR) or conversation session with each of the user accounts linked to the content item with the visual element.

[0045] In some examples, the augmentation system 208 provides a user-based publishing platform that enables a user to select a geographical location on a map and upload content associated with the selected geographical location. The user can also specify the circumstances under which a particular media overlay should be provided to other users. The augmentation system 208 generates a media overlay that includes the uploaded content and associates the uploaded content with the selected geographical location.

[0046] In other examples, the augmentation system 208 provides a merchant-based publishing platform that enables a merchant to select a particular media overlay associated with a geographical location via a bidding process. For example, the augmentation system 208 associates the media overlay of the highest bidding merchant with the corresponding geographical location for a predefined amount of time. The augmentation system 208 communicates with the image processing server 122 to obtain an AR experience and presents an identifier of such an experience in one or more user interfaces (e.g., as an icon on a live image or video, or as a thumbnail or icon in an interface dedicated to the presented AR experience identifier). Once an AR experience is selected, one or more images, videos, or AR graphic elements are retrieved and presented as an overlay on top of the image or video captured by the client device 102. In some cases, the camera device is switched to a front view (e.g., the front camera device of the client device 102 is activated in response to the activation of a particular AR experience), and the image from the front camera device of the client device 102 instead of the rear camera device of the client device 102 starts to be displayed on the client device 102. One or more images, videos, or AR graphic elements are retrieved and presented as an overlay on top of the image captured and displayed by the front camera device of the client device 102.

[0047] In other examples, the augmentation system 208 is capable of communicating and exchanging data with another augmentation system 208 on another client device 102 and with a server via the network 112. The data exchanged can include: a session identifier that identifies a shared AR session; a transformation between the first client device 102 and the second client device 102 (e.g., a plurality of client devices 102 including the first device and the second device) for aligning the shared AR session to a common origin; a common coordinate system; functions (e.g., commands to activate functions) and other payload data (e.g., text, audio, video, or other multimedia data).

[0048] The enhancement system 208 sends a transformation to the second client device 102 so that the second client device 102 can adjust the AR coordinate system based on the transformation. In this way, the first client device 102 and the second client device 102 synchronize their coordinate systems to display the content in the AR session. Specifically, the enhancement system 208 calculates the origin of the second client device 102 in the coordinate system of the first client device 102. Then, the enhancement system 208 can determine the offset in the coordinate system of the second client device 102 based on the position of the origin in the coordinate system of the second client device 102 from the perspective of the second client device 102. A transformation is generated using the offset so that the second client device 102 generates AR content according to the common coordinate system with the first client device 102.

[0049] The enhancement system 208 can communicate with the client device 102 to establish a separate or shared AR session. The enhancement system 208 can also be coupled to the messaging server 118 to establish an electronic group communication session (e.g., group chat, instant messaging) for the client device 102 in the shared AR session. The electronic group communication session can be associated with a session identifier provided by the client device 102 to gain access to the electronic group communication session and the shared AR / VR session. In one example, the client device 102 first obtains access to the electronic group communication session and then obtains a session identifier in the electronic group communication session that allows the client device 102 to access the shared AR / VR session. In some examples, the client device 102 is able to access the shared AR session without the help of or communication with the enhancement system 208 in the application server 114.

[0050] The map system 210 provides various geographical location functions and supports the presentation of map-based media content and messages by the messaging client 104. For example, the map system 210 enables the display on a map (e.g., stored in Figure 3 the profile data 316 shown) of user icons or avatar representations to indicate the current or past locations of the user's "friends" within the context of the map, as well as the media content (e.g., a collection of messages including photos and videos) generated by such friends. For example, a message posted by the user from a specific geographical location to the messaging system 100 can be displayed to the user's "friends" within the context of that specific location on the map interface of the messaging client 104. The user can also share his or her location and status information with other users of the messaging system 100 (e.g., using an appropriate status avatar) via the messaging client 104, where the location and status information are similarly displayed within the context of the map interface of the messaging client 104 to the selected users.

[0051] The game system 212 provides various game functions within the context of the messaging client 104. The messaging client 104 provides a game interface that presents a list of available games (e.g., web-based games or web-based applications) that can be launched by a user within the context of the messaging client 104 and played with other users of the messaging system 100. The messaging system 100 also enables a particular user to invite such other users to participate in playing a particular game by issuing an invitation from the messaging client 104 to the other users. The messaging client 104 also supports both voice and text messaging (e.g., chat) within the context of playing a game, provides a leaderboard for the game, and supports providing in-game rewards (e.g., game currency and items).

[0052] The external resource system 220 provides an interface for the messaging client 104 to communicate with an external app server 110 to launch or access external resources. Each external resource (app) server 110 hosts a scaled-down version of an application, such as an application based on a markup language (e.g., HTML5) or an external application (e.g., a game, utility, payment, or ride-sharing application external to the messaging client 104). The messaging client 104 can launch the web-based resource (e.g., application) by accessing an HTML5 file from an external resource (app) server 110 associated with the web-based resource. In some examples, the applications hosted by the external resource server 110 are programmed in JavaScript using a software development kit (SDK) provided by the messaging server 118. The SDK includes application programming interfaces (APIs) having functions that can be called or activated by the web-based application. In some examples, the messaging server 118 includes a JavaScript library that provides access to certain user data of the messaging client 104 to a given third-party resource. HTML5 is used as an example technology for programming games, but applications and resources programmed based on other technologies can be used.

[0053] To integrate the functionality of the SDK into the web-based resource, the external resource (app) server 110 downloads the SDK from the messaging server 118 or otherwise receives the SDK. Once downloaded or received, the SDK is included as part of the application code of the web-based external resource. The code of the web-based resource can then call or activate certain functions of the SDK to integrate features of the messaging client 104 into the web-based resource.

[0054] The SDK stored on the messaging server 118 effectively provides a bridge between external resources (e.g., third-party or external applications 109 or applets and the messaging client 104). This provides users with a seamless experience of communicating with other users on the messaging client 104 while also preserving the look and feel of the messaging client 104. To bridge the communication between the external resources and the messaging client 104, in some examples, the SDK facilitates the communication between the external resource server 110 and the messaging client 104. In some examples, the WebView JavaScript Bridge running on the client device 102 establishes two one-way communication channels between the external resources and the messaging client 104. Messages are sent asynchronously between the external resources and the messaging client 104 via these communication channels. Each SDK function call is sent as a message and a callback. Each SDK function is implemented by constructing a unique callback identifier and sending a message with that callback identifier.

[0055] By using the SDK, not all information from the messaging client 104 is shared with the external resource server 110. The SDK restricts which information is shared based on the needs of the external resources. In some examples, each external resource server 110 provides an HTML5 file corresponding to the web-based external resource to the messaging server 118. The messaging server 118 may add a visual representation (e.g., box art or other graphics) of the web-based external resource in the messaging client 104. Once the user selects the visual representation through the graphical user interface (GUI) of the messaging client 104 or indicates that the messaging client 104 accesses the features of the web-based external resource, the messaging client 104 obtains the HTML5 file and instantiates the resources required to access the features of the web-based external resource.

[0056] The messaging client 104 presents a GUI for an external resource (e.g., a login page or a splash screen). During, before, or after presenting the login page or splash screen, the messaging client 104 determines whether the launched external resource has been previously authorized to access the user data of the messaging client 104. In response to determining that the launched external resource has been previously authorized to access the user data of the messaging client 104, the messaging client 104 presents another GUI of the external resource, which includes the functions and features of the external resource. In response to determining that the launched external resource has not been previously authorized to access the user data of the messaging client 104, after a threshold period (e.g., 3 seconds) of displaying the login page or splash screen of the external resource, the messaging client 104 slides out a menu for authorizing the external resource to access the user data (e.g., animates the menu to emerge from the bottom of the screen to the middle or other part of the screen). This menu identifies the types of user data for which the external resource will be authorized to be used. In response to receiving a user selection of the accept option, the messaging client 104 adds the external resource to the list of authorized external resources and enables the external resource to access the user data from the messaging client 104. In some examples, the messaging client 104 authorizes the external resource to access the user data according to the OAuth 2 framework.

[0057] The messaging client 104 controls the types of user data shared with the external resource based on the type of the authorized external resource. For example, access to a first type of user data (e.g., only two-dimensional (2D) avatars of users with or without different avatar characteristics) is provided to an external resource that includes a full-scale external application (e.g., a third-party or external application 109). As another example, access to a second type of user data (e.g., payment information, 2D avatars of the user, 3D avatars of the user, and avatars with various avatar characteristics) is provided to an external resource that includes a scaled-down version of the external application (e.g., a web-based version of a third-party application). Avatar characteristics include different ways of customizing the appearance and feel of the avatar (e.g., different poses, facial features, clothing, etc.).

[0058] The content tagging system 224 generates content items for display via a messaging application (e.g., of a first user). The content tagging system 224 receives, via the messaging application, an input selecting a visual element (e.g., a graphic, a sticker, a graphical element, an AR experience, etc.) (e.g., from the first user), and determines that the visual element visually represents multiple users (e.g., the first user and one or more friends of the first user). The content tagging system 224 adds the visual element to the content item (e.g., overlays the visual element on the content item). In response to adding the visual element to the content item and based on determining that the visual element represents multiple users, the content tagging system 224 links the content item with the visual element to one or more corresponding accounts on the messaging applications of one or more of the multiple users.

[0059] In some examples, in response to receiving the input selecting the visual element, the content tagging system 224 displays, via the messaging application, a confirmation prompt (e.g., to the first user) that identifies one or more of the multiple users (e.g., a second user). In some examples, the content tagging system 224 receives an input selecting a confirmation option from the confirmation prompt. In response to the input selecting the confirmation option, the content item with the visual element is linked to the corresponding accounts of the multiple users.

[0060] In some examples, in response to receiving the input adding the visual element to the content item, the content tagging system 224 automatically links the content item with the visual element to the corresponding accounts of the multiple users. In some examples, the visual element includes a sticker or a graphic, and the content item includes an image or a video. In some examples, the content tagging system 224 receives an input specifying the placement of the visual element on the content item.

[0061] In some examples, the visual element includes multiple avatars representing each of the multiple users. In some examples, the visual element includes captions associated with one or more of the multiple users. In some examples, the content tagging system 224 determines that an input has been received from a first user of the multiple users. A content item can be generated for display on a first client device 102 associated with the first user. In response to linking the content item with the visual element to the corresponding accounts, the content tagging system 224 notifies a second user of the multiple users about the content item with the visual element.

[0062] In some examples, the content tagging system 224 causes a second client device 102 associated with a second user to present an option to view a content item with a visual element. In some examples, the content tagging system 224 identifies a third user among a plurality of users linked to the content item with the visual element. The content tagging system 224 determines that the second user and the third user are not friends on the messaging application. In such a case, in response to determining that the second user and the third user are not friends on the messaging application and in response to determining that the second user and the third user are included among the plurality of users associated with the respective accounts linked to the content item with the visual element, the content tagging system 224 provides the second user with an option to add the third user as a friend on the messaging application.

[0063] In some examples, the content tagging system 224 superimposes a visual element on the content item. In some examples, in response to linking a content item with a visual element to a respective account, the content tagging system 224 automatically launches an AR experience on the respective client devices 102 of the plurality of users. In some examples, in response to linking a content item with a visual element to a respective account, the content tagging system 224 automatically establishes a conversation session among the plurality of users. In some examples, the content tagging system 224 determines that an input has been received from a first user among the plurality of users. A content item can be generated for display on a first client device 102 associated with the first user. The content tagging system 224 shares the content item with the visual element with a plurality of friends of the first user on the messaging application. In some examples, the content tagging system 224 allows each of the plurality of friends to add any one of the plurality of users as a friend on the messaging application.

[0064] Data Architecture

[0065] Figure 3 is a schematic diagram showing a data structure 300 that can be stored in the database 126 of the messaging server system 108 according to certain examples. Although the content of the database 126 is shown as including several tables, it should be understood that the data can be stored in other types of data structures (e.g., stored in an object-oriented database).

[0066] The database 126 includes message data stored within a message table 302. For any particular message, the message data includes at least message sender data, message recipient (or receiver) data, and a payload. Further details regarding information that can be included in a message and included within the message data stored in the message table 302 are described below with reference to Figure 4 describes additional details regarding information that can be included in a message and included within the message data stored in the message table 302.

[0067] The entity table 306 stores entity data and is linked (e.g., by reference) to the entity graph 308 and the profile data 316. Entities for which records are maintained within the entity table 306 can include individuals, corporate entities, organizations, objects, locations, events, etc. Any entity for which the messaging server system 108 stores data can be an identified entity, regardless of the entity type. A unique identifier and an entity type identifier (not shown) are provided for each entity.

[0068] The entity graph 308 stores information about the relationships and associations between entities. By way of example only, such relationships can be social, professional (e.g., working in the same company or organization), interest-based, or activity-based.

[0069] The profile data 316 stores various types of profile data about a specific entity. Based on privacy settings specified by the specific entity, the profile data 316 can be selectively used and presented to other users of the messaging system 100. In the case where the entity is an individual, the profile data 316 includes, for example, a username, a telephone number, an address, settings (e.g., notification and privacy settings), and an avatar representation (or a collection of such avatar representations) selected by the user. Then, a specific user can selectively include one or more of these avatar representations within the content of messages transmitted via the messaging system 100 and on a map interface displayed by the messaging client 104 to other users. The collection of avatar representations can include "status avatars" that present graphical representations of statuses or activities that the user can select to convey at a particular time.

[0070] In the case where the entity is a group, in addition to the group name, members, and various settings (e.g., notifications) for the relevant group, the profile data 316 for the group can similarly include one or more avatar representations associated with the group.

[0071] The database 126 also stores enhancement data, such as overlays or filters, in the enhancement table 310. The enhancement data is associated with videos (the data of which is stored in the video table 304) and images (the data of which is stored in the image table 312) and is applied to the videos and images.

[0072] The database 126 can also store data related to individual and shared AR sessions. This data can include data transmitted between the AR session client controller of the first client device 102 and another AR session client controller of the second client device 102, as well as data transmitted between the AR session client controller and the enhancement system 208. The data can include data for establishing a common coordinate system for a shared AR scene, transformations between devices, session identifiers, images depicting the body, bone joint positioning, wrist joint positioning, feet, etc.

[0073] In one example, a filter is an overlay that is displayed as an overlay on an image or video during presentation to a recipient user. Filters can be of various types, including filters selected by a user from a set of filters presented to the sending user by the messaging client 104 when the sending user is composing a message. Other types of filters include location-based filters (also known as geo-filters), which can be presented to the sending user based on a geographical location. For example, based on geographical location information determined by the global positioning system (GPS) unit of the client device 102, the messaging client 104 can present location-based filters specific to a nearby or specific location within the user interface.

[0074] Another type of filter is a data filter, which can be selectively presented to the sending user by the messaging client 104 based on other input or information collected by the client device 102 during the message creation process. Examples of data filters include the current temperature at a specific location, the current speed at which the sending user is traveling, the battery life of the client device 102, or the current time.

[0075] Other enhanced data that can be stored in the image table 312 includes AR content items (e.g., corresponding to an applied AR experience). An AR content item or AR item can be a real-time special effect and sound that can be added to an image or video.

[0076] As described above, enhanced data includes AR content items, overlays, image transformations, AR images, and similar items involving modifications that can be applied to image data (e.g., video or images). This includes real-time modifications that modify an image when it is captured using the device sensors of the client device 102 (e.g., one or more camera devices) and then display the modified image on the screen of the client device 102. This also includes modifications to stored content (e.g., video clips in a library that can be modified). For example, in a client device 102 capable of accessing multiple AR content items, a user can use a single video clip with multiple AR content items to see how different AR content items will modify the stored clip. For example, by selecting different AR content items for the content, multiple AR content items applying different pseudo-random movement models can be applied to the same content. Similarly, real-time video capture can be used with the shown modifications to show how the video images currently captured by the sensors of the client device 102 will modify the captured data. Such data can be simply displayed on the screen without being stored in memory, or the content captured by the device sensors can be recorded and stored in memory with or without modification (or both). In some systems, a preview feature can show how different AR content items will look within different windows on the display. This can, for example, enable multiple windows with different pseudo-random animations to be viewed simultaneously on the display.

[0077] Accordingly, data using AR content items and various systems or other such transformation systems that use the data to modify content can involve: detection of objects (e.g., faces, hands, bodies, cats, dogs, surfaces, objects, etc.) in video frames; tracking such objects as they leave, enter, and move around the field of view; and modifying or transforming such objects while tracking them. In various examples, different methods can be used to implement such transformations. Some examples can involve: generating a 3D mesh model of one or more objects, and using the transformation and animated textures of the model within the video to implement the transformation. In other examples, tracking of points on an object can be utilized to place an image or texture (which can be 2D or 3D) at the tracking location. In yet another example, neural network analysis of video frames can be used to place an image, model, or texture within the content (e.g., an image or video frame). Accordingly, AR content items refer both to the images, models, and textures used to create transformations in content and to the additional modeling and analysis information required to implement such transformations using object detection, tracking, and placement.

[0078] Real-time video processing can be performed using any kind of video data (e.g., video streams, video files, etc.) stored in the memory of any kind of computerized system. For example, a user can load a video file and save it in the device's memory, or can use the device's sensors to generate a video stream. Additionally, computer animation models can be used to process any object, such as a person's face and various parts of a person's body, an animal, or a non-biological object (e.g., a chair, a car, or other object).

[0079] In some examples, when a specific modification is selected along with the content to be transformed, the computing device identifies the elements to be transformed and then, if the elements are present in the frames of the video, detects and tracks them. The elements of the object are modified according to the request for the modification, thereby transforming the frames of the video stream. For different types of transformations, the frames of the video stream can be transformed by different methods. For example, for frame transformations that mainly involve changing the form of the elements of an object, (e.g., using an Active Shape Model (ASM) or other known methods), characteristic points are calculated for each element of the object. Then, for each element in at least one element of the object, a grid based on the characteristic points is generated. This grid is used in subsequent stages to track the elements of the object in the video stream. During the tracking process, the grids mentioned for each element are aligned with the positioning of each element. Then, additional points are generated on the grid. A first set of first points is generated for each element based on the request for the modification, and a set of second points is generated for each element based on this set of first points and the request for the modification. Then, the frames of the video stream can be transformed by modifying the elements of the object based on this set of first points, this set of second points, and the grid. In such a method, the background of the modified object can also be changed or distorted by tracking and modifying the background of the modified object.

[0080] In some examples, a transformation that uses the elements of an object to change some regions of the object can be performed by calculating characteristic points for each element of the object and generating a grid based on the calculated characteristic points. Points are generated on the grid, and then various regions are generated based on the points. Then, the elements of the object are tracked by aligning the regions for each element with the positions of each element in at least one element, and the nature of the regions can be modified based on the request for the modification, thereby transforming the frames of the video stream. Depending on the specific request for the modification, the attributes of the mentioned regions can be transformed in different ways. Such modifications can involve: changing the color of the region; removing at least some parts of the region from the frames of the video stream; including one or more new objects in the region based on the request for the modification; and modifying or distorting the region or the elements of the object. In various examples, any combination of such modifications or other similar modifications can be used. For some models to be animated, some characteristic points can be selected as control points for determining the entire state space of the options for the model animation.

[0081] In some examples of using a computer animation model to transform image data using face detection, a specific face detection algorithm (e.g., Viola-Jones) is used to detect a face in an image. Then, the ASM algorithm is applied to the face region of the image to detect facial feature reference points.

[0082] Other methods and algorithms suitable for face detection can be used. For example, in some examples, landmarks are used to locate features, where a landmark represents a distinguishable point that exists in most of the images under consideration. For example, for a face landmark, the position of the left eye pupil can be used. If the initial landmark is not identifiable (e.g., in the case of a person wearing an eye patch), then secondary landmarks can be used. Such a landmark identification process can be used for any such object. In some examples, a set of landmarks forms a shape. The coordinates of the points in the shape can be used to represent the shape as a vector. A similarity transformation (allowing translation, scaling, and rotation) that minimizes the average Euclidean distance between the shape points is used to align one shape with another. The average shape is the average of the aligned training shapes.

[0083] In some examples, the search for landmarks starts from an average shape that is aligned with the location and size of the face determined by a full-face detector. Then, this kind of search repeats the following steps until convergence: adjusting the positions of the shape points by template matching of the image texture around each point to propose a tentative shape, and then conforming the tentative shape to a global shape model. In some systems, individual template matching is unreliable, and the shape model pools the results of weak template matches to form a stronger overall classifier. The entire search is repeated at each level of an image pyramid from coarse resolution to fine resolution.

[0084] The transformation system can capture an image or video stream on a client device (e.g., client device 102) and perform complex image manipulations locally on the client device 102 while maintaining an appropriate user experience, computation time, and power consumption. Complex image manipulations can include size and shape changes, emotion transformation (e.g., changing a face from a frown to a smile), state transformation (e.g., making a subject age, reducing apparent age, changing gender), style transformation, application of graphical elements, and any other suitable image or video manipulations implemented by a convolutional neural network that has been configured to execute effectively on the client device 102.

[0085] In some examples, a computer animation model for transforming image data can be used by a system in which a user can use a client device 102 having a neural network to capture an image or video stream of the user (e.g., a selfie), and the neural network operates as part of a messaging client 104 operating on the client device 102. A transformation system operating within the messaging client 104 determines the presence of a face within the image or video stream and provides a modification icon associated with the computer animation model to transform the image data, or the computer animation model can be presented in association with the interfaces described herein. The modification icon includes a change that can be used to modify the basis of the user's face within the image or video stream as part of a modification operation. Once the modification icon is selected, the transformation system initiates a process of converting the user's image to reflect the selected modification icon (e.g., generating a smiling face on the user). Once the image or video stream is captured and a specified modification is selected, the modified image or video stream can be presented in a graphical user interface displayed on the client device 102. The transformation system can implement a complex convolutional neural network on a portion of the image or video stream to generate and apply the selected modification. That is, the user can capture an image or video stream, and once the modification icon has been selected, the modified result can be presented in real-time or near real-time. Additionally, when a video stream is being captured and the selected modification icon remains toggled, the modification can be persistent. A neural network of machine learning can be used to implement such modifications.

[0086] A GUI presenting the modifications performed by the transformation system can provide additional interaction options to the user. Such options can be based on the interface used to initiate content capture and the selection of a particular computer animation model (e.g., initiated from a content creator user interface). In various examples, after an initial selection of a modification icon, the modification can be persistent. The user can toggle the modification on or off by tapping or otherwise selecting the face being modified by the transformation system and store it for later viewing or browsing other areas of the imaging application. In cases where multiple faces are modified by the transformation system, the user can globally toggle the modification on or off by tapping or selecting a single face modified and displayed within the GUI. In some examples, individual faces within a group of multiple faces can be modified separately, or such modifications can be toggled individually by tapping or selecting a single face or a series of faces displayed within the GUI.

[0087] The story table 314 stores data regarding a collection of messages and associated image, video, or audio data, which are assembled into a collection (e.g., a story or a library). The creation of a particular collection can be initiated by a particular user (e.g., each user whose record is maintained in the entity table 306). A user can create a "personal story" in the form of a collection of content that has been created and sent / broadcast by that user. To this end, the user interface of the messaging client 104 can include user-selectable icons to enable the sending user to add specific content to his or her personal story.

[0088] The collection can also constitute a "live story", which is a collection of content from multiple users created manually, automatically, or using a combination of manual and automatic techniques. For example, a "live story" can constitute a curated stream of user-submitted content from various locations and events. Options to contribute content to a particular live story can be presented, for example, via the user interface of the messaging client 104 to users whose client devices have location services enabled and are at a common location event at a particular time. The messaging client 104 can identify live stories to users based on his or her location. The end result is a "live story" told from a community perspective.

[0089] Another type of content collection is referred to as a "location story", which enables users whose client devices 102 are located within a particular geographical location (e.g., on a college or university campus) to contribute to a particular collection. In some examples, contributing to a location story may require secondary authentication to verify that the end user belongs to a particular organization or other entity (e.g., is a student on a university campus).

[0090] As mentioned above, the video table 304 stores video data, which, in one example, is associated with messages for which records are maintained in the message table 302. Similarly, the image table 312 stores image data, which is associated with messages for which the message data is stored in the entity table 306. The entity table 306 can associate various enhancements from the enhancement table 310 with the various images and videos stored in the image table 312 and the video table 304.

[0091] The data structure 300 can also store training data for training one or more machine learning techniques (models) to segment real-world objects or items in a real-world environment depicted in an image corresponding to a location (e.g., a room in a home). The training data can include multiple images and videos and their corresponding ground-truth room segmentations. The images and videos can include a mixture of various real-world objects that can be present in a home or in different rooms of a home. One or more machine learning techniques can be trained to extract features of the received input image or video and establish a relationship between the extracted features and the segmentation. Once trained, the machine learning techniques can receive new images or videos and can compute the segmentation of the items depicted in the newly received images or videos.

[0092] Data communication architecture

[0093] Figure 4 is a schematic diagram showing the structure of a message 400 according to some examples, the message 400 being generated by a messaging client 104 for transmission to another messaging client 104 or a messaging server 118. The content of a particular message 400 is used to populate a message table 302 that is stored within a database 126 and is accessible by the messaging server 118. Similarly, the content of the message 400 is stored in a memory as "in-transit" or "in-flight" data for the client device 102 or the application server 114. The message 400 is shown as including the following example components:

[0094] · Message identifier 402: A unique identifier that identifies the message 400.

[0095] · Message text payload 404: Text to be generated by a user via a user interface of the client device 102 and included in the message 400.

[0096] · Message image payload 406: Image data captured by a camera device component of the client device 102 or retrieved from the memory of the client device 102 and included in the message 400. The image data for the message 400 being sent or received can be stored in an image table 312.

[0097] · Message video payload 408: Video data captured by a camera device component or retrieved from a memory component of the client device 102 and included in the message 400. The video data for the message 400 being sent or received can be stored in a video table 304.

[0098] · Message audio payload 410: Audio data captured by a microphone or retrieved from a memory component of the client device 102 and included in the message 400.

[0099] · Message enhancement data 412: Enhancement data (e.g., filters, stickers, or other annotations or enhancements) representing the enhancement to be applied to the message image payload 406, message video payload 408, or message audio payload 410 of message 400. The enhancement data 412 for a sent or received message 400 can be stored in the enhancement table 310.

[0100] · Message duration parameter 414: A parameter value indicating, in seconds, the amount of time for which the content of the message (e.g., message image payload 406, message video payload 408, message audio payload 410) will be presented to or made accessible to the user via the messaging client 104.

[0101] · Message geographic location parameter 416: Geographic location data (e.g., latitude coordinates and longitude coordinates) associated with the content payload of the message. Multiple message geographic location parameter 416 values can be included in the payload, each of which is associated with a content item included in the content (e.g., a specific image within the message image payload 406, or a specific video within the message video payload 408).

[0102] · Message story identifier 418: An identifier value that identifies one or more content collections (e.g., "stories" identified in the story table 314) associated with a specific content item in the message image payload 406 of message 400. For example, the identifier value can be used to associate each of multiple images within the message image payload 406 with multiple content collections.

[0103] · Message tag 420: Each message 400 can be tagged with multiple tags, each of which indicates a topic of the content included in the message payload. For example, in the case where a specific image included in the message image payload 406 depicts an animal (e.g., a lion), a tag value indicating the relevant animal can be included in the message tag 420. The tag value can be manually generated based on user input or can be automatically generated using, for example, image recognition.

[0104] · Message sender identifier 422: An identifier (e.g., a messaging system identifier, an email address, or a device identifier) indicating the user of the client device 102 on which message 400 was generated and from which it was sent.

[0105] · Message recipient identifier 424: An identifier (e.g., a messaging system identifier, an email address, or a device identifier) indicating the user of the client device 102 to which message 400 is addressed.

[0106] The content (e.g., value) of each component of the message 400 can be a pointer to a location in a table where the content data value is stored. For example, the image value in the message image payload 406 can be a pointer to a location within the image table 312 (or the address of a location within the image table 312). Similarly, the value within the message video payload 408 can point to data stored within the video table 304, the value stored within the message enhancement data 412 can point to data stored within the enhancement table 310, the value stored within the message story identifier 418 can point to data stored within the story table 314, and the values stored within the message sender identifier 422 and the message recipient identifier 424 can point to user records stored within the entity table 306.

[0107] Content tagging system

[0108] As mentioned above, the content tagging system 224 allows users to automatically tag modified content items (content items with visual elements added) or link modified content items to one or more other users. For example, as Figure 5 shown, a GUI 500 can be presented on the client device 102 of a first user. The GUI 500 includes a content item 510, such as an image, video, or animation. The content item 510 can have been previously captured and retrieved from a library of previously captured content. In some cases, the content item 510 can be a live image received from a live video feed from a front camera device or a rear camera device of the client device 102.

[0109] The content tagging system 224 receives an input requesting to add a visual element to the content item 510. In response, the content tagging system 224 presents a menu (not shown) that includes multiple visual elements of different types. For example, the visual elements can include stickers, graphic elements, animations, AR experiences, etc. The menu can include a search input area that allows the user to enter search criteria to find visual elements of interest. The content tagging system 224 can receive an input selecting a specific visual element from the menu. In response, the content tagging system 224 presents the selected specific visual element 520 on the GUI 500. For example, the content tagging system 224 presents the specific visual element 520 at a default location on the content item 510.

[0110] The content tagging system 224 can receive an input of dragging a selected specific visual element 520 to a specified location. The content tagging system 224 can receive an input of modifying the caption of the selected specific visual element 520 and / or inserting a caption into the selected specific visual element 520. After adding the selected specific visual element 520 to the content item 510, the content tagging system 224 can access the metadata associated with the selected specific visual element 520. The metadata can be a data structure listing a username, an account, a phone number, a link, a unique identifier, an IP address, or any other information that can be used to identify the corresponding user on the messaging client 104. The metadata can also indicate which users linked to the specific visual element 520 are friends on the messaging client 104. The content tagging system 224 determines, based on the metadata, that the selected specific visual element 520 is associated with multiple users, including the first user to whom the GUI 500 is presented. For example, the selected specific visual element 520 visually depicts a representation of two users by including the avatars of each user including the messaging client 104. For example, the visual element 520 can be a sticker, a graphic, an AR experience identifier, or an icon that visually identifies each user specified in the metadata associated with the visual element 520. Specifically, the visual element 520 can include an avatar unique to each user specified in the metadata.

[0111] The metadata for the specific visual element 520 indicates / includes the username of each user associated with the visual element 520 on the messaging client 104. The system identifies the users from the metadata by searching the messaging client 104 based on the username and retrieving the respective contact information of the users and the friend list of each user among the users. The users may or may not be friends of the first user to whom the GUI 500 is presented on the messaging client 104. The messaging client 104 can determine which users specified in the metadata are friends with each other by comparing the friend lists retrieved for each user and finding the friends not common in the lists. For example, the messaging client 104 can retrieve a first friend list of a first user associated with the visual element 520 and a second friend list of a second user associated with the visual element 520. The messaging client 104 can compare the first friend list with the second friend list and, based on the comparison, determine that the first user is not on the second friend list. In response, the messaging client 104 determines that the first user and the second user are not friends with each other on the messaging client 104, but both are associated with the same visual element 520.

[0112] In some examples, in response to identifying multiple users in the metadata, the content tagging system 224 automatically retrieves the user account identifier for each of the multiple users. The content tagging system 224 associates the account identifier with the content item and stores the account identifier and the content item 510 that has been enhanced or modified with the selected specific visual element 520. In some cases, the content tagging system 224 stores an indication in each of the account identifiers that links the corresponding account to the modified content item 510. For example, the content tagging system 224 stores an address or link in the account identifier of the first user associated with the visual element 520. When the first user accesses the content tagging system 224, the content tagging system 224 accesses the stored address or link to present a list of content items linked to and associated with the first user.

[0113] In some examples, the content tagging system 224 presents a confirmation prompt (not shown) as an overlay on top of the modified content item 510 to the first user. The content tagging system 224 can cause the confirmation prompt to include a list of the identified users found in the metadata of the selected specific visual element 520. The content tagging system 224 can cause the confirmation prompt to include an option that allows the first user to confirm and trigger the automatic linking of the corresponding account to the modified content item 510.

[0114] In some examples, the content tagging system 224 generates a notification for a second user among the multiple users, informing the second user that the second user's account has been linked to the modified content item 510. The notification can include an option to view or access the content item 510 that includes the selected specific visual element 520. In some examples, the content tagging system 224 can determine that the second user is not a friend of one of the users (e.g., a third user) included in the corresponding account that has been linked to the modified content item 510.

[0115] For example, the first user who added the selected specific visual element 520 can be a friend of each of the multiple users, but some of the multiple users may not be friends with each other. In such a case, the content tagging system 224 presents a GUI to the second user and the third user, such as Figure 6The example GUI 600 shown in [Figure X]. The GUI 600 may include a modified content item 610 (e.g., the content item 510 with the selected specific visual element 520). When the modified content item 610 is presented to a second user, the content tagging system 224 may include an indication 620 of a third user who is also linked to the modified content item 610 but is not a friend of the second user. The indication 620 may include an option for the second user to become friends with the third user on the messaging client 104. In response to receiving an input selecting the add friend option from the indication 620, the content tagging system 224 generates a friend request to the third user that identifies the second user and includes a miniaturized or small view representation of the modified content item 610. Then, the third user may confirm the friendship to establish a friendship with the second user on the messaging client 104. For example, the modified content item 610 with a message indicating that the second user wants to be friends with the third user may be presented on the third user's client device 102. The third user may select the confirm option from the message to become friends with the second user.

[0116] In some examples, the content tagging system 224 automatically establishes a conversation session between each of the multiple users identified in the metadata of the selected specific visual element 520 that has been added to the content item 510. The content tagging system 224 may trigger the presentation of a conversation interface in response to receiving an input to add the selected specific visual element 520 to the content item 510. In such a case, the content tagging system 224 may present a GUI, such as Figure 7 the example GUI 700 shown in [Figure X]. The GUI 700 includes a conversation interface that allows each of the multiple users identified in the metadata of the selected specific visual element 520 to exchange messages with each other in a group conversation setting. The GUI 700 is presented on the respective client devices 102 of each of the multiple users and may include an introductory message or prompt indicating the reason for presenting the conversation interface. The message may identify the first user who added the selected specific visual element 520 to the content item 510 and indicate that the selected specific visual element 520 has been determined to be associated with each of the multiple users. The message may also include a miniaturized or scaled-down version of the modified content item 510 with the selected specific visual element 520 within the conversation interface.

[0117] In some examples, the content tagging system 224 automatically establishes an AR session among each of multiple users identified in the metadata of a selected specific visual element 520 that has been added to a content item 510. The content tagging system 224 can trigger the presentation of the AR session in response to receiving an input to add the selected specific visual element 520 to the content item 510. In such a case, the content tagging system 224 can present a GUI, such as Figure 8 the example GUI 800 shown in. The GUI 800 includes an AR experience interface 810 that allows each of multiple users identified in the metadata of the selected specific visual element 520 to interact with one another in the AR setting. The GUI 800 can be presented on the respective client devices 102 of each of the multiple users and can include an introductory message 820 or prompt indicating the reason for presenting the conversation interface. The message 820 can identify the first user who added the selected specific visual element 520 to the content item 510 and indicate that the selected specific visual element 520 has been determined to be associated with each of the multiple users. The message 820 can also include a miniaturized or scaled-down version of the modified content item 510 with the selected specific visual element 520 within the conversation interface. The GUI 800 can include a list 830 of each of the multiple users associated with the selected specific visual element 520.

[0118] Figure 9 is a flowchart of a process 900 according to some examples. Although the flowchart depicts the operations as a sequential process, many of these operations can be performed in parallel or simultaneously. Additionally, the order of the operations can be rearranged. The process terminates when its operations are complete. The process can correspond to a method, program, etc. The steps of the method can be performed in whole or in part, can be performed in combination with some or all of the steps in other methods, and can be performed by any number of different systems or any part thereof (e.g., a processor included in any system).

[0119] At operation 901, as discussed above, the client device 102 generates a content item for display via a messaging application.

[0120] At operation 902, as discussed above, the client device 102 receives an input to select a visual element via the messaging application.

[0121] At operation 903, as discussed above, the client device 102 determines that the visual element visually represents multiple users.

[0122] At operation 904, as discussed above, the client device 102 adds the visual element to the content item.

[0123] At operation 905, as discussed above, in response to adding a visual element to a content item and based on determining that the visual element represents multiple users, client device 102 links the content item with the visual element to one or more corresponding accounts on the messaging application of one or more of the multiple users.

[0124] Machine architecture

[0125] Figure 10 is a graphical representation of a machine 1000 within which instructions 1008 (e.g., software, program, application, applet, app, or other executable code) can be executed to cause the machine 1000 to perform any one or more of the methods discussed herein. For example, instructions 1008 can cause the machine 1000 to perform any one or more of the methods described herein. The instructions 1008 transform the general unprogrammed machine 1000 into a particular machine 1000 programmed to perform the described and illustrated functions in the described manner. The machine 1000 can operate as a stand-alone device or can be coupled (e.g., networked) to other machines. In a networked deployment, the machine 1000 can operate in a server-client network environment as a server machine or a client machine, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 1000 can include, but is not limited to: server computers, client computers, personal computers (PCs), tablet computers, laptop computers, netbooks, set-top boxes (STBs), personal digital assistants (PDAs), entertainment media systems, cellular telephones, smartphones, mobile devices, wearable devices (e.g., smartwatches), smart home devices (e.g., smart appliances), other smart devices, web appliances, network routers, network switches, network bridges, or any machine capable of sequentially or otherwise executing the instructions 1008 specifying the actions to be taken by the machine 1000. Further, although only a single machine 1000 is shown, the term "machine" shall also be taken to include a collection of machines that individually or jointly execute the instructions 1008 to perform any one or more of the methods discussed herein. For example, the machine 1000 can include the client device 102 or any one of the server devices that form part of the messaging server system 108. In some examples, the machine 1000 can also include both a client system and a server system, where certain operations of a particular method or algorithm are executed on the server side and certain operations of a particular method or algorithm are executed on the client side.

[0126] Machine 1000 may include a processor 1002, a memory 1004, and input / output (I / O) components 1038 that may be configured to communicate with each other via a bus 1040. In an example, the processor 1002 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 1006 and a processor 1010 that execute instructions 1008. The term "processor" is intended to include multi-core processors, which may include two or more independent processors (sometimes referred to as "cores") that may execute instructions simultaneously. Although Figure 10 multiple processors 1002 are shown, machine 1000 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

[0127] Memory 1004 includes a main memory 1012, a static memory 1014, and a storage unit 1016, all of which may be accessed by processor 1002 via bus 1040. Main memory 1004, static memory 1014, and storage unit 1016 store instructions 1008 that embody any one or more of the methods or functions described herein. The instructions 1008 may also reside, completely or partially, within main memory 1012, within static memory 1014, within machine-readable media within storage unit 1016, within at least one of the processors 1002 (e.g., within a cache memory of the processor), or within any suitable combination thereof during execution by machine 1000.

[0128] I / O components 1038 may include various components that receive input, provide output, generate output, transmit information, exchange information, capture measurements, etc. The specific I / O components 1038 included in a particular machine will depend on the type of the machine. For example, a portable machine (e.g., a mobile phone) may include a touch input device or other such input mechanism, while a headless server machine will likely not include such a touch input device. It should be understood that I / O components 1038 may include Figure 10Many other components not shown. In various examples, the I / O component 1038 can include a user output component 1024 and a user input component 1026. The user output component 1024 can include visual components (e.g., a display such as a plasma display panel (PDP), a light-emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), acoustic components (e.g., speakers), haptic components (e.g., a vibration motor, a resistance mechanism), other signal generators, etc. The user input component 1026 can include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, an optical keyboard, or other alphanumeric input components), point-based input components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or other pointing instruments), haptic input components (e.g., physical buttons, a touch screen that provides the position and force of a touch or touch gesture, or other haptic input components), audio input components (e.g., a microphone), etc.

[0129] In other examples, the I / O component 1038 can include a biometric component 1028, a motion component 1030, an environmental component 1032, or a location component 1034 and various other components. For example, the biometric component 1028 includes components for detecting expressions (e.g., hand expressions, facial expressions, voice expressions, body postures, or eye tracking), measuring biometric signals (e.g., blood pressure, heart rate, body temperature, sweating, or brain waves), identifying people (e.g., voice recognition, retina recognition, facial recognition, fingerprint recognition, or electroencephalogram-based recognition), etc. The motion component 1030 includes acceleration sensor components (e.g., accelerometers), gravity sensor components, and rotational sensor components (e.g., gyroscopes).

[0130] The environmental component 1032 includes, for example, one or more camera devices (with still image / photo and video capabilities), lighting sensor components (e.g., photometers), temperature sensor components (e.g., one or more thermometers that detect ambient temperature), humidity sensor components, pressure sensor components (e.g., barometers), acoustic sensor components (e.g., one or more microphones that detect background noise), proximity sensor components (e.g., infrared sensors that detect nearby objects), gas sensors (e.g., gas detection sensors that detect the concentration of hazardous gases for safety or measure pollutants in the atmosphere), or other components that can provide an indication, measurement, or signal corresponding to the surrounding physical environment.

[0131] Regarding the imaging device, the client device 102 may have an imaging device system that includes, for example, a front imaging device on the front surface of the client device 102 and a rear imaging device on the rear surface of the client device 102. The front imaging device may be used, for example, to capture still images and videos of the user of the client device 102 (e.g., "selfies"), and then the still images and videos may be enhanced using the above-described enhancement data (e.g., filters). The rear imaging device may be used, for example, to capture still images and videos in a more traditional imaging device mode, where the images are similarly enhanced using the enhancement data. In addition to the front imaging device and the rear imaging device, the client device 102 may further include a 360° imaging device for capturing 360° photos and videos.

[0132] In addition, the imaging device system of the client device 102 may include a dual rear imaging device (e.g., a main imaging device and a depth sensing imaging device), or even a triple, quadruple, or quintuple rear imaging device configuration on the front and rear sides of the client device 102. For example, these multi-imaging device systems may include a wide-angle imaging device, an ultra-wide-angle imaging device, a telephoto imaging device, a macro imaging device, and a depth sensor.

[0133] The location component 1034 includes a location sensor component (e.g., a GPS receiver component), an altitude sensor component (e.g., an altimeter or barometer that detects air pressure, from which altitude can be obtained), an orientation sensor component (e.g., a magnetometer), etc.

[0134] A variety of techniques can be used to implement communication. The I / O component 1038 also includes a communication component 1036 that is operable to couple the machine 1000 to the network 1020 or the device 1022 via a corresponding coupling or connection. For example, the communication component 1036 may include a network interface component that interfaces with the network 1020 or other suitable devices. In other examples, the communication component 1036 may include a wired communication component, a wireless communication component, a cellular communication component, a near field communication (NFC) component, components (e.g., low power consumption), components, and other communication components for providing communication via other modalities. The device 1022 may be another machine or any peripheral device among various peripheral devices (e.g., a peripheral device coupled via USB).

[0135] In addition, the communication component 1036 can detect an identifier or include components operable to detect an identifier. For example, the communication component 1036 can include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., for detecting one-dimensional barcodes such as Universal Product Code (UPC) barcodes, multi-dimensional barcodes such as Quick Response (QR) codes, Aztec codes, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, UCC RSS-2D barcodes, and other optical codes), or an acoustic detection component (e.g., a microphone for identifying an audio signal of a tag). Additionally, various information can be derived via the communication component 1036, such as a location via Internet Protocol (IP) geolocation, a location via signal triangulation, a location via detecting an NFC beacon signal that can indicate a specific location, etc.

[0136] Various memories (e.g., main memory 1012, static memory 1014, and the memory of the processor 1002) and the storage unit 1016 can store one or more sets of instructions and data structures (e.g., software) implemented or used by any one or more of the methods or functions described herein. When executed by the processor 1002, these instructions (e.g., instructions 1008) cause the various operations to implement the disclosed examples.

[0137] The instructions 1008 can be sent or received over the network 1020 using a transmission medium via a network interface device (e.g., the network interface component included in the communication component 1036) and using any one of several well-known transmission protocols (e.g., HTTP). Similarly, the instructions 1008 can be sent or received using a transmission medium via a coupling (e.g., a peer-to-peer coupling) to the device 1022.

[0138] Software architecture

[0139] Figure 11FIG. 1100 is a block diagram showing a software architecture 1104 that may be installed on any one or more of the devices described herein. The software architecture 1104 is supported by hardware, such as a machine 1102 that includes a processor 1120, a memory 1126, and I / O components 1138. In this example, the software architecture 1104 may be conceptualized as a stack of layers, where each layer provides a specific function. The software architecture 1104 includes the following layers, such as an operating system 1112, libraries 1110, frameworks 1108, and applications 1106. In operation, the application 1106 activates API calls 1150 through the software stack and receives messages 1152 in response to the API calls 1150.

[0140] The operating system 1112 manages hardware resources and provides common services. The operating system 1112 includes, for example: a kernel 1114, services 1116, and drivers 1122. The kernel 1114 serves as an abstraction layer between the hardware and other software layers. For example, the kernel 1114 provides functions such as memory management, processor management (e.g., scheduling), component management, networking, and security settings. The services 1116 may provide other common services for other software layers. The drivers 1122 are responsible for controlling or interfacing with the underlying hardware. For example, the drivers 1122 may include a display driver, a camera driver, a Bluetooth or Bluetooth Low Energy driver, a flash drive, a serial communication driver (e.g., a USB driver), a WI-FI driver, an audio driver, a power management driver, etc.

[0141] The libraries 1110 provide a common low-level infrastructure used by the applications 1106. The libraries 1110 may include system libraries 1118 (e.g., C standard libraries), which provide functions such as memory allocation functions, string manipulation functions, mathematical functions, etc. In addition, the libraries 1110 may include API libraries 1124, such as media libraries (e.g., libraries for supporting the presentation and manipulation of various media formats, such as Moving Picture Experts Group-4 (MPEG4), Advanced Video Coding (H.264 or AVC), Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio codec, Joint Photographic Experts Group (JPEG or JPG), or Portable Network Graphics (PNG)), graphics libraries (e.g., the OpenGL framework for presenting graphics content in 2D and 3D on a display), database libraries (e.g., SQLite that provides various relational database functions), web libraries (e.g., WebKit that provides web browsing functions), etc. The libraries 1110 may also include various other libraries 1128 to provide many other APIs to the applications 1106.

[0142] The framework 1108 provides a common high-level infrastructure used by the applications 1106. For example, the framework 1108 provides various GUI functions, high-level resource management, and high-level location services. The framework 1108 can provide a wide range of other APIs that can be used by the applications 1106, some of which may be specific to a particular operating system or platform.

[0143] In an example, the applications 1106 can include a home application 1136, a contacts application 1130, a browser application 1132, a book reader application 1134, a location application 1142, a media application 1144, a messaging application 1146, a gaming application 1148, and a variety of other applications such as an external application 1140. The applications 1106 are programs that execute functions defined in the program. One or more of the applications 1106 can be created using various programming languages, such as object-oriented programming languages (e.g., Objective-C, Java, or C++) or procedural programming languages (e.g., C or assembly language), which are structured in various ways. In a particular example, the external application 1140 (e.g., an application developed using the ANDROID TM or IOS TM SDK by an entity other than the vendor of a particular platform) can be mobile software running on a mobile operating system such as IOS TM ANDROID TM , Phone, or another mobile operating system. In this example, the external application 1140 can activate an API call 1150 provided by the operating system 1112 to facilitate the functions described herein.

[0144] Glossary

[0145] "Carrier signal" means any non-tangible medium that can store, encode, or carry instructions executed by a machine and includes digital or analog communication signals or other non-tangible media that facilitate the communication of such instructions. Instructions can be sent or received over a network using a transmission medium via a network interface device.

[0146] "Client device" means any machine that interfaces with a communication network to obtain resources from one or more server systems or other client devices. A client device can be, but is not limited to, a mobile phone, a desktop computer, a laptop computer, a PDA, a smartphone, a tablet computer, a Ultrabook, a netbook, a laptop computer, a multiprocessor system, a microprocessor-based or programmable consumer electronics product, a gaming console, a set-top box, or any other communication device that a user can use to access a network.

[0147] "Communication network" refers to one or more portions of a network, which can be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), the Internet, a portion of the Internet, a portion of the public switched telephone network (PSTN), a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, a Wi-Fi network, other types of networks, or a combination of two or more such networks. For example, a network or a portion of a network can include a wireless network or a cellular network, and the coupling can be a code division multiple access (CDMA) connection, a global system for mobile communications (GSM) connection, or other types of cellular or wireless couplings. In this example, the coupling can implement any data transmission technology among various types of data transmission technologies, such as single carrier radio transmission technology (1xRTT), evolved data optimized (EVDO) technology, general packet radio service (GPRS) technology, enhanced data rate GSM evolution (EDGE) technology, the 3rd Generation Partnership Project (3GPP) including 3G, the 4th generation wireless (4G) network, universal mobile telecommunications system (UMTS), high speed packet access (HSPA), worldwide interoperability for microwave access (WiMAX), long term evolution (LTE) standards, other data transmission technologies defined by various standards setting organizations, other long distance protocols, or other data transmission technologies.

[0148] "Component" refers to a logical, device, or physical entity having boundaries that are defined by a function or subroutine call, a branch point, an API, or other techniques that provide partitioning or modularity to a particular processing or control function. Components can be combined with other components via their interfaces to perform machine processing. A component can be an encapsulated functional hardware unit designed to be used with other components and can be part of a program that typically performs a specific function among related functions.

[0149] Components can constitute software components (e.g., code implemented on a machine-readable medium) or hardware components. A "hardware component" is a tangible unit capable of performing certain operations and can be configured or arranged in a physical manner. In various examples, one or more computer systems (e.g., stand-alone computer systems, client computer systems, or server computer systems) or one or more hardware components of a computer system (e.g., a processor or a group of processors) can be configured by software (e.g., an application or a portion of an application) to operate as a hardware component that performs certain operations described herein.

[0150] Hardware components can also be implemented mechanically, electronically, or in any suitable combination thereof. For example, a hardware component can include dedicated circuitry or logic that is permanently configured to perform certain operations. A hardware component can be a dedicated processor, such as a field programmable gate array (FPGA) or an ASIC. A hardware component can also include programmable logic or circuitry that is temporarily configured by software to perform certain operations. For example, a hardware component can include software executed by a general purpose processor or other programmable processor. Once configured by such software, the hardware component becomes a particular machine (or a particular component of a machine) uniquely customized to perform the configured functions, rather than a general purpose processor. It should be appreciated that the decision of whether to implement a hardware component mechanically in dedicated and permanently configured circuitry or in temporarily configured (e.g., software-configured) circuitry can be made based on cost and time considerations. Accordingly, the phrase “hardware component” (or “hardware-implemented component”) should be understood to include a tangible entity, i.e., an entity that is physically constructed, permanently configured (e.g., hardwired) or temporarily configured (e.g., programmed) to operate in some manner or to perform certain operations described herein.

[0151] Consider an example where a hardware component is temporarily configured (e.g., programmed). It is not necessary to configure or instantiate every hardware component in the hardware component at any given time. For example, in a case where a hardware component includes a general purpose processor that is configured by software to be a dedicated processor, the general purpose processor can be configured at different times to be respective different dedicated processors (e.g., including different hardware components). The software accordingly configures one or more particular processors to, for example, constitute a particular hardware component at one time and different hardware components at different times.

[0152] A hardware component can provide information to and receive information from other hardware components. Accordingly, the described hardware components can be considered to be communicatively coupled. In a case where there are multiple hardware components present simultaneously, communication can be achieved through signal transmission between two or more of the hardware components in the hardware component (e.g., via appropriate circuitry and buses). In an example where multiple hardware components are configured or instantiated at different times, communication between such hardware components can be achieved, for example, by storing information in a memory structure accessible by the multiple hardware components and retrieving the information from the memory structure. For example, one hardware component can perform an operation and store the output of the operation in a memory device to which it is communicatively coupled. Then, another hardware component can access the memory device at a subsequent time to retrieve the stored output and process the stored output. A hardware component can also initiate communication with an input device or an output device and can operate on a resource (e.g., a collection of information).

[0153] The various operations of the example methods described herein can be performed, at least in part, by one or more processors temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors can constitute processor-implemented components that operate to perform one or more of the operations or functions described herein. As used herein, a "processor-implemented component" refers to a hardware component implemented using one or more processors. Similarly, the methods described herein can be at least in part processor-implemented, where a particular one or more processors are examples of hardware. For example, at least some of the operations of the method can be performed by one or more processors 1002 or processor-implemented components. Additionally, the one or more processors can also operate to support the execution of relevant operations in a "cloud computing" environment or as "software as a service" (SaaS) operations. For example, at least some of the operations can be performed by a group of computers (as examples of machines including processors), where the operations can be accessed via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., APIs). The execution of certain operations of the method can be distributed among the processors, not residing only within a single machine but being deployed across several machines. In some examples, the processors or processor-implemented components can be located in a single geographical location (e.g., within a home environment, an office environment, or a server farm). In other examples, the processors or processor-implemented components can be distributed across several geographical locations.

[0154] "Computer-readable storage medium" refers to both machine storage media and transmission media. Thus, the term includes both storage devices / media and carrier / modulated data signals. The terms "machine-readable medium", "computer-readable medium", and "device-readable medium" mean the same thing and can be used interchangeably in this disclosure.

[0155] "Ephemeral message" refers to a message that is accessible for a time-limited duration. An ephemeral message can be text, image, video, etc. The access time for an ephemeral message can be set by the message sender. Alternatively, the access time can be a default setting or a setting specified by the recipient. Regardless of the setting technique, the message is transient.

[0156] "Machine storage medium" means a single or multiple storage devices and media (e.g., centralized or distributed databases, and associated caches and servers) that store executable instructions, routines, and data. Thus, the term should be considered to include, without limitation, solid-state memories as well as optical and magnetic media, including memories internal or external to a processor. Specific examples of machine storage media, computer storage media, and device storage media include: non-volatile memories, including, for example, semiconductor memory devices such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), FPGA, and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The terms "machine storage medium", "device storage medium", and "computer storage medium" mean the same thing and may be used interchangeably in this disclosure. The terms "machine storage medium", "computer storage medium", and "device storage medium" expressly exclude carrier waves, modulated data signals, and other such media, at least some of which are covered by the term "signal medium".

[0157] "Non-transitory computer-readable storage medium" means a tangible medium that can store, encode, or carry instructions executable by a machine.

[0158] "Signal medium" means a medium that can store, encode, or carry instructions executable by a machine and includes digital or analog communication signals or other intangible media that facilitate the communication of software or data. The term "signal medium" should be considered to include any form of modulated data signal, carrier wave, etc. The term "modulated data signal" refers to a signal that has one or more of its characteristics set or changed to encode information therein. The terms "transmission medium" and "signal medium" mean the same thing and may be used interchangeably in this disclosure.

[0159] Changes and modifications may be made to the disclosed examples without departing from the scope of the disclosure. These and other changes or modifications are intended to be included within the scope of the disclosure as expressed in the appended claims.

Claims

1. A method, comprising: Generating a content item for display via a messaging application; Receiving an input for selecting a visual element via the messaging application; Determining that the visual element visually represents a plurality of users; Adding the visual element to the content item; And In response to adding the visual element to the content item and based on determining that the visual element represents the plurality of users, linking the content item with the visual element to one or more corresponding accounts on the messaging application of one or more of the plurality of users.

2. The method according to claim 1, further comprising: In response to receiving the input for selecting the visual element, displaying, via the messaging application, a confirmation prompt that identifies the plurality of users.

3. The method according to claim 2, further comprising: Receiving an input for selecting a confirmation option from the confirmation prompt, wherein, in response to the input for selecting the confirmation option, linking the content item with the visual element to the corresponding accounts of the plurality of users.

4. The method according to any one of claims 1 to 3, further comprising: In response to receiving an input for adding the visual element to the content item, automatically linking the content item with the visual element to the corresponding accounts of the plurality of users.

5. The method according to any one of claims 1 to 4, wherein The visual element includes a sticker or a graphic, and wherein the content item includes an image or a video.

6. The method according to any one of claims 1 to 5, further comprising: Receiving an input specifying the placement of the visual element on the content item.

7. The method according to any one of claims 1 to 6, wherein The visual element includes a plurality of avatars representing each of the plurality of users.

8. The method according to claim 7, wherein The visual element includes captions associated with one or more of the plurality of users.

9. The method according to any one of claims 1 to 8, further comprising: Determining that the input has been received from a first user among the plurality of users, and generating the content item for display on a first client device associated with the first user; And In response to linking the content item with the visual element to the corresponding accounts, notifying a second user among the plurality of users about the content item with the visual element.

10. The method according to claim 9, further comprising: Causing a second client device associated with the second user to present an option to view the content item with the visual element.

11. The method according to claim 10, further comprising: Identifying a third user among the plurality of users linked to the content item with the visual element; Determining that the second user and the third user are not friends on the messaging application; And In response to determining that the second user and the third user are not friends on the messaging application and in response to determining that the second user and the third user are included among the plurality of users associated with the corresponding accounts linked to the content item with the visual element, providing the second user with an option to add the third user as a friend on the messaging application.

12. The method according to any one of claims 1 to 11 further comprises: Overlaying the visual element on the content item.

13. The method according to any one of claims 1 to 12 further comprises: Automatically launching an augmented reality (AR) experience on the respective client devices of the plurality of users in response to linking the content item with the visual element to the respective account.

14. The method according to any one of claims 1 to 13 further comprises: Automatically establishing a conversation session among the plurality of users in response to linking the content item with the visual element to the respective account.

15. The method according to any one of claims 1 to 14 further comprises: Determining that the input has been received from a first user among the plurality of users, generating the content item for display on a first client device associated with the first user; And Sharing the content item with the visual element with a plurality of friends of the first user on the messaging application.

16. The method according to claim 15 further comprises allowing each of the plurality of friends to add any one of the plurality of users as a friend on the messaging application.

17. A system comprising: A processor configured to perform operations including: Generating a content item for display via a messaging application; Receiving an input for selecting a visual element via the messaging application; Determining that the visual element visually represents a plurality of users; Adding the visual element to the content item; and Linking the content item with the visual element to one or more corresponding accounts on the messaging application of one or more of the plurality of users in response to adding the visual element to the content item and based on determining that the visual element represents the plurality of users.

18. The operations of the system according to claim 17 further comprise: Displaying, via the messaging application, a confirmation prompt identifying the plurality of users in response to receiving the input for selecting the visual element.

19. The operations of the system according to claim 18 further comprise: Receiving an input for selecting a confirmation option from the confirmation prompt, wherein in response to the input for selecting the confirmation option, the content item with the visual element is linked to the respective accounts of the plurality of users.

20. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations including: Generating a content item for display via a messaging application; Receiving an input for selecting a visual element via the messaging application; Determining that the visual element visually represents a plurality of users; Adding the visual element to the content item; And In response to adding the visual element to the content item and based on determining that the visual element represents the plurality of users, link the content item with the visual element to one or more corresponding accounts on the messaging application of one or more of the plurality of users.

Images