Avatar clothing updates in the messaging system

Abstract

Aspects of the present disclosure comprise a system comprising a computer-readable storage medium storing a method and a program for updating the clothing of a user’s avatar in a messaging system. The program and method provide receiving a request by a messaging application running on a user’s device to identify at least one object depicted in an image captured by a device camera; determining attributes of the object depicted in the image in response to receiving the request; selecting a clothing item associated with the attributes of the object from a plurality of candidate clothing items for the user’s avatar; and displaying the selected clothing item together with the user’s avatar.

Description

Technology Field

[0001] Cross-reference regarding related applications

[0002] This patent application claims the benefit of U.S. Provisional Patent Application No. 63 / 044,062, filed on June 25, 2020, titled "UPDATING AVATAR CLOTHING FOR A USER OF A MESSAGING SYSTEM," the entirety of which is incorporated herein by reference.

[0003] Technology field

[0004] The present disclosure generally relates to a messaging system comprising updating a user's avatar of the messaging system. Background Technology

[0005] Messaging systems provide the exchange of message content between users. For example, a messaging system enables a user to exchange message content (e.g., text, images) with one or more other users. Brief explanation of the drawing

[0006] In drawings that are not necessarily drawn in a fixed proportion, the same reference numbers may describe similar components in different views. To facilitate the identification of a discussion of any specific element or action, the top digit or numbers of a reference number indicate the drawing number where that element is first introduced. Some embodiments are illustrated in the accompanying drawings as examples rather than limitations. FIG. 1 is a schematic representation of a networked environment in which the present disclosure may be placed, according to some exemplary embodiments. FIG. 2 is a schematic representation of a messaging system having both client-side and server-side functionality according to some exemplary embodiments. FIG. 3 is a schematic representation of a data structure as maintained in a database, according to some exemplary embodiments. FIG. 4 is a schematic representation of a message according to some exemplary embodiments. FIG. 5 is an interaction diagram illustrating a process for updating a user's avatar clothing in a messaging system according to some exemplary embodiments. FIG. 6a illustrates an exemplary user interface for detecting clothing in a captured image according to some exemplary embodiments. FIG. 6b illustrates an exemplary user interface for selecting avatar clothing based on detected clothing, according to some exemplary embodiments. FIG. 7 illustrates another exemplary user interface for detecting clothing in a captured image according to some exemplary embodiments. FIG. 8 is a flowchart illustrating a process for updating a user's avatar clothing in a messaging system according to some exemplary embodiments. FIG. 9 is a flowchart of an access-limiting process according to some exemplary embodiments. FIG. 10 is a schematic representation of a machine in the form of a computer system in which a set of instructions can be executed to enable the machine to perform any one or more of the methodologies discussed herein, according to some exemplary embodiments. FIG. 11 is a block diagram illustrating a software architecture in which examples may be implemented according to some exemplary embodiments. Specific details for implementing the invention

[0007] Messaging systems typically enable users to exchange content items (e.g., messages, images, and / or videos) with each other in message threads. Messaging systems may implement an avatar system configured to maintain user avatar representations for use in messaging, or may work with it in other ways.

[0008] The disclosed embodiments provide a messaging client that detects articles of clothing in a captured image (e.g., a live video feed captured by a device camera, or an image selected from a photo library) and presents a user interface having user-selectable avatar outfits that match or otherwise resemble the detected articles of clothing. For example, the messaging client determines attributes of the detected articles of clothing in the captured image (e.g., types of clothing, color, size, and / or brand name). Then, the messaging client searches a database storing available articles of clothing and / or outfits for avatars based on the attributes and selects one or more outfits that most closely match the attributes. Then, the messaging client presents a user interface that displays the selected outfits along with interface elements to update the user avatar based on the selected outfits.

[0009] FIG. 1 is a block diagram illustrating an exemplary messaging system (100) for exchanging data (e.g., messages and associated content) over a network. The messaging system (100) includes multiple instances of client devices (102), each of which hosts multiple applications including messaging clients (104). Each messaging client (104) is coupled to communicate with other instances of messaging clients (104) and messaging server systems (108) over a network (106) (e.g., the Internet).

[0010] A messaging client (104) can communicate and exchange data with another messaging client (104) and a messaging server system (108) through a network (106). The data exchanged between messaging clients (104) and between a messaging client (104) and a messaging server system (108) includes functions (e.g., commands that invoke functions) as well as payload data (e.g., text, audio, video, or other multimedia data).

[0011] The messaging server system (108) provides server-side functionality to a specific messaging client (104) via the network (106). Although specific functions of the messaging system (100) are described herein as being performed by the messaging client (104) or by the messaging server system (108), the location of specific functions within the messaging client (104) or the messaging server system (108) may be a design choice. For example, it may be technically desirable to initially place specific technologies and functions within the messaging server system (108), but later migrate these technologies and functions to the messaging client (104) when the client device (102) has sufficient processing capacity.

[0012] The messaging server system (108) supports various services and operations provided to the messaging client (104). Such operations include transmitting data to the messaging client (104), receiving data from it, and processing data generated by it. This data may include, for example, message content, client device information, geolocation information, media augmentation and overlays, message content persistence conditions, social network information, and live event information. Data exchanges within the messaging system (100) are invoked and controlled through functions available via the user interface (UI) of the messaging client (104).

[0013] Now, referring specifically to the messaging server system (108), an application program interface (API) server (110) is coupled to application servers (114) to provide a programmatic interface. The application servers (114) are communicably coupled to a database server (122), which facilitates access to a database (124) that stores data associated with messages processed by the application servers (114). Similarly, a web server (112) is coupled to the application servers (114) and provides web-based interfaces to the application servers (114). To this end, the web server (112) processes incoming network requests via HTTP (Hypertext Transfer Protocol) and various other related protocols.

[0014] An API (Application Program Interface) server (110) receives and transmits message data (e.g., commands and message payloads) between a client device (102) and application servers (114). Specifically, the application program interface (API) server (110) provides a set of interfaces (e.g., routines and protocols) that can be called or queried by a messaging client (104) to invoke the functionality of the application servers (114). The application program interface (API) server (110) exposes various functions supported by the application servers (114), including account registration, login functionality, transmission of messages through the application servers (114) from a specific messaging client (104) to another messaging client (104), transmission of media files (e.g., images or videos) from the messaging client (104) to the messaging server (116), and settings of a collection of media data (e.g., stories) for possible access by another messaging client (104), searching of a list of friends of a user of the client device (102), searching of such collections, searching of messages and content, adding and deleting entities (e.g., friends) to an entity graph (e.g., social graph), locating friends within the social graph, and opening application events (e.g., related to the messaging client (104)).

[0015] Application servers (114) host a number of server applications and subsystems, including, for example, a messaging server (116), an image processing server (118), and a social network server (120). The messaging server (116) implements a number of message processing techniques and functions, particularly those related to the aggregation and other processing of content (e.g., text and multimedia content) contained in messages received from multiple instances of the messaging client (104). As described in more detail, text and media content from multiple sources may be aggregated into collections of content (e.g., called stories or galleries). These collections are then made available to the messaging client (104). Other processor and memory-intensive data processing may also be performed on the server side by the messaging server (116), taking into account the hardware requirements for such processing.

[0016] The application servers (114) also include an image processing server (118) dedicated to performing various image processing operations with respect to images or videos in the payload of a message typically transmitted from or received from the messaging server (116).

[0017] The social network server (120) supports various social networking functions and services and makes these functions and services available to the messaging server (116). To this end, the social network server (120) maintains and accesses an entity graph (304) (shown in FIG. 3) within a database (124). Examples of functions and services supported by the social network server (120) include the identification of other users of the messaging system (100) with whom a specific user has a relationship or is "following," and also the identification of other entities and interests of the specific user.

[0018] FIG. 2 is a block diagram illustrating additional details regarding a messaging system (100) according to some examples. Specifically, the messaging system (100) is illustrated as including a messaging client (104) and application servers (114). The messaging system (100) implements a number of subsystems supported on the client side by the messaging client (104) and on the server side by the application servers (114). These subsystems include, for example, a short-term timer system (202), a collection management system (204), an augmentation system (208), a map system (210), an object detection system (212), and / or an avatar system (214).

[0019] The short-term timer system (202) is responsible for enforcing temporary or time-limited access to content by the messaging client (104) and the messaging server (116). The short-term timer system (202) includes a plurality of timers that selectively enable access (e.g., for presentation and display) to messages and associated content through the messaging client (104) based on duration and display parameters associated with a message, or a collection of messages (e.g., a story). Further details regarding the operation of the short-term timer system (202) are provided below.

[0020] The collection management system (204) is responsible for managing sets or collections of media (e.g., collections of text, image, video, and audio data). Collections of content (e.g., messages including images, videos, text, and audio) may be organized into "event galleries" or "event stories." These collections may be available for a specific period, such as the duration of the event to which the content is related. For example, content related to a music concert may be available as a "story" for the duration of that music concert. The collection management system (204) may also be responsible for publishing an icon that provides notification of the existence of a specific collection in the user interface of the messaging client (104).

[0021] The collection management system (204) further includes a curation interface (206) that allows a collection manager to manage and curate a collection of specific content. For example, the curation interface (206) enables an event organizer to curate a collection of content related to a specific event (e.g., removing inappropriate content or duplicate messages). Additionally, the collection management system (204) automatically curates content collections using machine vision (or image recognition technology) and content rules. In certain examples, a reward may be paid to the user for including user-generated content in a collection. In these cases, the collection management system (204) operates to automatically pay these users for using their content.

[0022] The augmentation system (208) provides various functions that enable a user to augment media content associated with a message (e.g., annotate or modify or edit in other ways). For example, the augmentation system (208) provides functions related to the creation and publication of media overlays for messages processed by the messaging system (100). The augmentation system (208) operatively supplies media overlays or augmentations (e.g., image filters) to the messaging client (104) based on the geolocation of the client device (102). In another example, the augmentation system (208) operatively supplies media overlays to the messaging client (104) based on other information, such as the social network information of the user of the client device (102). Media overlays may include audio and visual content and visual effects. Examples of audio and visual content include pictures, text, logos, animations, and sound effects. Examples of visual effects include color overlaying. Audio and visual content or visual effects may be applied to a media content item (e.g., a photo) on the client device (102). For example, the media overlay may include text or an image that can be overlaid on top of a photo taken by the client device (102). In another example, the media overlay may include a location identification overlay (e.g., Venice Beach), the name of a live event, or a seller name overlay (e.g., Beach Coffee House). In another example, the augmentation system (208) uses the geolocation of the client device (102) to identify a media overlay containing the seller name at the geolocation of the client device (102). The media overlay may include other indicia associated with the seller.Media overlays are stored in a database (124) and can be accessed through a database server (122).

[0023] In some examples, the augmentation system (208) provides a user-based publication platform that enables users to select a geolocation on a map and upload content associated with the selected geolocation. Users can also specify situations in which a particular media overlay should be provided to other users. The augmentation system (208) creates a media overlay that includes the uploaded content and associates the uploaded content with the selected geolocation.

[0024] In other examples, the augmentation system (208) provides a merchant-based publication platform that enables merchants to select a specific media overlay associated with a geolocation through a bidding process. For example, the augmentation system (208) associates the media overlay of the highest bidder with a corresponding geolocation for a predefined amount of time.

[0025] The map system (210) provides various geographic location functions and supports the presentation of map-based media content and messages by the messaging client (104). For example, the map system (210) enables the display of user icons or avatars (e.g., stored in association with profile data (302) discussed below) on the map to display the current or past locations of the user's "friends" as well as media content created by these friends (e.g., collections of messages including photos and videos) within the context of the map. For example, a message posted by a user to the messaging system (100) from a specific geographic location can be displayed to the specific user's "friends" on the map interface of the messaging client (104) within the context of the map at that specific location. Furthermore, the user can share their location and status information with other users of the messaging system (100) through the messaging client (104) (e.g., using an appropriate status avatar), and this location and status information is similarly displayed to selected users within the context of the map interface of the messaging client (104).

[0026] The object detection system (212) provides various object detection functions within the context of the messaging system (100). The object detection system (212) may use one or more object classifiers to identify objects depicted in a captured image. The image may correspond to a live video feed captured by a camera (e.g., rear or front camera) of the client device (102). Alternatively or additionally, the image may correspond to an image (e.g., a photo) (e.g., a photo library) stored in association with the user of the client device (102).

[0027] In one or more embodiments, the object detection system (212) is configured to implement or otherwise access object recognition algorithms (e.g., machine learning algorithms) configured to scan a captured image and detect / track the movement of objects within the image. By non-limiting example, detectable objects within the image include: human faces, parts of the human body, animals and their parts, landscapes, objects of nature, inanimate objects (e.g., clothing items, chairs, books, cars, buildings, other structures), illustrations of objects (e.g., on posters and / or flyers), text-based objects, equation-based objects, etc.

[0028] Additionally, the object detection system (212) is configured to determine or access the attributes of the objects in other ways. For a specific object, the object detection system (212) can determine or retrieve attributes such as name / type, genre, color, size, shape, texture, environmental factors (e.g., geolocation, time, weather), and / or other supplementary information (e.g., song title / artist for the object corresponding to the media).

[0029] Regarding environmental factors, the object detection system (212) may receive information from the messaging client (104) to identify weather, geographical location, time, etc. around the client device (102) (e.g., via device sensors). The object detection system (212) may rank the detected attributes based on relevance, such as based on association with one or more environmental factors. Other machine learning techniques may be used to select and rank the detected attributes. The object detection system (212) may select an object associated with the highest-ranking attributes from a list of objects detected in a captured image and transmit a display of the selected object to the messaging client (104). Alternatively or additionally, the object detection system (212) may provide to communicate to the messaging client (104) one or more attributes (e.g., name / type) and / or a display of the ranking of attributes for each detected object.

[0030] In one or more embodiments, the object detection system (212) determines that one of the attributes corresponds to a keyword sponsored by a third party. For example, third parties may sponsor or pay to have certain keywords rank higher than others. In response to the determination that a given attribute corresponds to a sponsored keyword, the object detection system (212) may give that attribute a higher rank than other attributes.

[0031] The avatar system (214) provides various avatar-related functions within the context of the messaging system (100). The avatar system (214) is configured to allow a user to create, edit, and / or otherwise maintain a personalized avatar corresponding to the user's graphic representation in the messaging system (100). For example, when creating and / or editing an avatar, the user may select physical features for the avatar, such as face shape, skin tone, hair features (e.g., color, length, type, style), jaw shape, eyebrows, mouth, etc. Furthermore, the user may select clothing for the avatar by selecting clothing items such as shirts, pants / shorts, jackets, shoes, headwear, and / or other traditional clothing items. In one or more embodiments, the selected physical features and clothing for the user's avatar persist across multiple interfaces of the messaging system (100) (e.g., by being stored in association with profile data (302) as discussed below).

[0032] In one or more embodiments, the avatar system (214) provides pre-set templates and / or poses for generating different avatar images (e.g., to convey each user's states, activities and / or moods). For example, different avatar images are user-selectable to include in messages, media content items, stories, etc. To display the location of users within a map view, avatars are also available by the map system (210) as previously described. Additionally, the messaging client (104), together with the messaging server system (108), can display avatar images within various interfaces (e.g., chat interface, reply interface, etc.) to display user presence, state, activity and / or mood.

[0033] FIG. 3 is a schematic diagram illustrating data structures (300) that may be stored in a database (124) of a messaging server system (108) according to specific examples. Although the contents of the database (124) are depicted as including multiple tables, it will be recognized that data may be stored in data structures of other types (e.g., as an object-oriented database).

[0034] The database (124) contains message data stored in the message table (306). For any specific message, this message data includes at least message sender data, message recipient (or receiver) data, and payload. Further details regarding information that may be included in the message and may be included in the message data stored in the message table (306) are described below with reference to FIG. 4.

[0035] The entity table (308) stores entity data and is linked (e.g., for reference) to the entity graph (304) and profile data (302). Entities for which records are maintained within the entity table (308) may include individuals, legal entities, organizations, objects, places, events, etc. Regardless of the entity type, any entity for which the messaging server system (108) stores data may be a recognized entity. Each entity is provided with an entity type identifier (not shown) as well as a unique identifier.

[0036] The entity graph (304) stores information regarding relationships and associations between entities. Such relationships may be based on interests or activities, for example, social, professional (for example, work in a general corporation or organization).

[0037] Profile data (302) stores multiple types of profile data for a specific entity. Profile data (302) may be optionally used and presented to other users of the messaging system (100) based on privacy settings specified by the specific entity. If the entity is an individual, profile data (302) includes, for example, a username, phone number, address, settings (e.g., notification and privacy settings), as well as, if applicable, user-selected avatar representations (or a collection of such avatar representations). Avatar representations (e.g., physical appearance, clothing, status, activities) may be selected by the user along with the avatar attribute table (318) discussed below. The specific user may then optionally include one or more of these avatar representations within the content of messages communicated through the messaging system (100) and on map interfaces displayed to other users by messaging clients (104). A collection of avatar representations may include "state avatars" that present graphic representations of states or activities that a user can select to communicate at a specific time.

[0038] If the entity is a group, the profile data (302) for the group may similarly include one or more avatar representations associated with the group, in addition to the group name, members, and various settings (e.g., notifications) for the relevant group.

[0039] The database (124) also stores augmentation data, such as overlays or filters, in the augmentation table (310). The augmentation data is associated with and applied to videos (data about which is stored in the video table (314)) and images (data about which is stored in the image table (316)).

[0040] In one example, the filters are overlays that are displayed overlaid on an image or video during presentation to the recipient user. The filters may be various types of filters, including user-selected filters from a set of filters presented to the sender user by the messaging client (104) when the sender user is composing a message. Other types of filters include geolocation filters (also known as geo-filters) that may be presented to the sender user based on geolocation. For example, geolocation filters specific to a neighborhood or a particular location may be presented within the user interface by the messaging client (104) based on geolocation information determined by the Global Positioning System (GPS) unit of the client device (102).

[0041] Another type of filter is a data filter that may be optionally presented to the sending user by the messaging client (104) based on other inputs or information collected by the client device (102) during the message generation process. Examples of data filters include the current temperature at a specific location, the current speed at which the sending user is moving, the battery life of the client device (102), or the current time.

[0042] Other augmented data that may be stored in the image table (316) includes augmented reality content items (e.g., corresponding to applying lenses or augmented reality experiences). Augmented reality content items may be real-time special effects and sounds that can be added to an image or video.

[0043] As described above, augmented data includes similar terms referring to augmented reality content items, overlays, image transformations, AR images, and modifications that can be applied to image data (e.g., videos or images). This includes real-time modifications that modify an image, such as when the image is captured using device sensors (e.g., one or more cameras) of the client device (102) and then displayed on the screen of the client device (102) along with modifications. This also includes modifications to stored content, such as video clips within a gallery, that can be modified. For example, on a client device (102) accessing multiple augmented reality content items, the user may use a single video clip along with multiple augmented reality content items to determine how different augmented reality content items modify the stored clip. For example, multiple augmented reality content items applying different pseudorandom movement models may be applied to the same content by selecting different augmented reality content items for the content. Similarly, real-time video capture can be used with the illustrated modifications to show how video images currently being captured by the sensors of the client device (102) will modify the captured data. This data may simply be displayed on the screen and not stored in memory, or the content captured by the device sensors may be written and stored in memory with or without modifications (or both). In some systems, a preview feature may show how different augmented reality content items will appear simultaneously in different windows within the display. This may, for example, allow multiple windows with different pseudo-random animations to be displayed simultaneously on the display.

[0044] Accordingly, data and various systems using augmented reality content items or other such transformation systems to modify content using this data may involve the detection of objects (e.g., faces, hands, bodies, cats, dogs, surfaces, objects, etc.), the tracking of these objects as they move out of, enter, and around video frames, and the modification or transformation of these objects when they are tracked. In various embodiments, different methods may be used to achieve these transformations. Some examples may involve generating a 3D mesh model of an object or objects, and using transformations of the model and animated textures within the video to achieve the transformation. In other examples, tracking points on an object may be used to place an image or texture (which may be 2D or 3D) at the tracked position. In yet other examples, neural network analysis of video frames may be used to place images, models, or textures on the content (e.g., an image or video frame). Therefore, augmented reality content items refer to images, models, and textures used to generate transformations in the content, as well as all additional modeling and analysis information required to achieve these transformations through object detection, tracking, and placement.

[0045] Real-time video processing can be performed with any type of video data (e.g., video streams, video files, etc.) stored in the memory of any type of computerized system. For example, a user can load video files and store them in the device's memory, or generate video streams using the device's sensors. Additionally, any objects, such as human faces and parts of the human body, animals, or inanimate objects like chairs, cars, or other objects, can be processed using computer animation models.

[0046] In some examples, when a specific modification is selected along with the content to be transformed, the elements to be transformed are identified by a computing device and subsequently detected and tracked if they exist in the frames of the video. The elements of the object are modified according to the request for modification, thereby transforming the frames of the video stream. The transformation of the frames of the video stream may be performed by different methods for different types of transformations. For example, for frame transformations that primarily refer to changing the shapes of the elements of an object, characteristic points for each element of the object are calculated (e.g., using Active Shape Model (ASM) or other known methods). Then, a mesh based on the characteristic points is generated for each of at least one element of the object. This mesh is used in the next stage of tracking the elements of the object within the video stream. In the tracking process, the mesh mentioned for each element is aligned with the position of each element. Subsequently, additional points are generated on the mesh. A first set of first points is generated for each element based on the request for modification, and a second set of points is generated for each element based on the first set of points and the request for modification. Subsequently, frames of the video stream can be transformed by modifying the elements of the object based on the mesh and sets of first and second points. In this method, the background of the modified object can also be changed or distorted by tracking and modifying the background.

[0047] In some examples, transformations that modify parts of an object using its elements can be performed by calculating characteristic points for each element of the object and generating a mesh based on the calculated characteristic points. Points are generated on the mesh, and then various regions based on the points are generated. Subsequently, the elements of the object are tracked by aligning the regions for each element with the positions for at least one element, and the characteristics of the regions can be modified based on requests for modification, thereby transforming the frames of the video stream. Depending on a specific request for modification, the characteristics of the mentioned regions can be transformed in different ways. Such modifications may involve changing the color of the regions; removing at least some parts of the regions from the frames of the video stream; including one or more new objects in the regions based on the request for modification; and modifying or distorting the regions or elements of the objects. In various embodiments, any combination of these modifications or other similar modifications may be used. For specific models to be animated, some characteristic points may be selected as control points to be used to determine the entire state-space of options for model animation.

[0048] In some examples of computer animation models for transforming image data using face detection, faces are detected on the image using a specific face detection algorithm (e.g., Viola-Jones). Then, the Active Shape Model (ASM) algorithm is applied to the face region of the image to detect face feature reference points.

[0049] In other examples, other methods and algorithms suitable for face detection may be used. For example, in some embodiments, features are located using landmarks representing distinguishable points present in most of the images under consideration. For face landmarks, for example, the location of the left pupil may be used. If the initial landmark is not identifiable (e.g., a person wearing an eye patch), secondary landmarks may be used. These landmark identification procedures may be used for any such objects. In some examples, a set of landmarks forms a feature. Features may be represented as vectors using the coordinates of points within the feature. One feature is aligned with another using a similarity transformation (allowing for translation, scaling, and rotation) that minimizes the average Euclidean distance between feature points. The average feature is the average of the aligned training features.

[0050] In some examples, a search for landmarks from the average shape aligned with the position and size of the face determined by the global face detector is initiated. Subsequently, this search proposes a tentative shape by adjusting the positions of the shape points through template matching of the image texture around each point, and then repeats the steps of matching the tentative shape to the global shape model until convergence occurs. In some systems, individual template matches are unreliable, and the shape model pools the results of weak template matches to form a stronger global classifier. The global search is repeated at each level of the image pyramid from coarse resolution to fine resolution.

[0051] The transformation system can capture an image or video stream on a client device (102) and perform complex image manipulations locally on the client device (102) while maintaining an appropriate user experience, computation time, and power consumption. Complex image manipulations may include size and shape changes, emotion transfers (e.g., changing a face from a frown to a smile), state transfers (e.g., aging a subject, reducing apparent age, changing gender), style transfers, application of graphic elements, and any other appropriate image or video manipulations implemented by a convolutional neural network configured to be executed efficiently on the client device (102).

[0052] In some examples, a computer animation model for transforming image data may be used by a system capable of capturing a user's image or video stream (e.g., a selfie) using a client device (102) having a neural network that operates as part of a messaging client (104) operating on the client device (102). A transformation system operating within the messaging client (104) determines the presence of a face in the image or video stream and provides modification icons associated with the computer animation model for transforming image data, or the computer animation model may exist as being associated with the interface described herein. The modification icons include changes that may be the basis for modifying the user's face in the image or video stream as part of a modification operation. When a modification icon is selected, the transformation system initiates a process of transforming the user's image to reflect the selected modification icon (e.g., generating a smiling face for the user). The modified image or video stream may be presented in a graphical user interface displayed on the client device (102) as soon as the image or video stream is captured and a specific modification is selected. The transformation system can generate and apply selected modifications by implementing complex convolutional neural networks on a portion of an image or video stream. That is, a user can capture an image or video stream and, if a modification icon is selected, be presented with the modified result in real-time or near real-time. Furthermore, modifications can be sustained as long as the video stream is being captured and the selected modification icon remains toggled. Machine-taught neural networks can be used to enable these modifications.

[0053] A graphical user interface presenting modifications performed by a transformation system may provide the user with additional interaction options. These options may be based on an interface used to initiate the capture and selection of content from a specific computer animation model (e.g., initiation from a content creator user interface). In various embodiments, modifications may be persistent after the initial selection of a modification icon. The user may toggle modifications on or off by tapping or otherwise selecting the face being modified by the transformation system, and may save it for later viewing or browsing to other areas of the imaging application. If multiple faces are modified by the transformation system, the user may toggle modifications on or off globally by tapping or selecting a single face that is modified and displayed within the graphical user interface. In some embodiments, among a group of multiple faces, individual faces may be modified individually, or these modifications may be toggled individually by tapping or selecting an individual face or a series of individual faces displayed within the graphical user interface.

[0054] The story table (312) stores data regarding collections of messages and associated image, video, or audio data that are compiled into collections (e.g., stories or galleries). The creation of a specific collection may be initiated by a specific user (e.g., each user for whom a record is maintained in the entity table (308)). A user may create a “personal story” in the form of a collection of content created and transmitted / broadcasted by that user. To this end, the user interface of the messaging client (104) may include a user-selectable icon to enable the sending user to add specific content to their personal story.

[0055] The collection may also constitute a “live story,” which is a collection of content from multiple users generated manually, automatically, or using a combination of manual and automatic techniques. For example, a “live story” may constitute a curated stream of user-submitted content from various locations and events. Users who have location service-enabled client devices and are at a common location event at a specific time may be presented with an option to contribute content to a specific live story, for example, through the user interface of a messaging client (104). A live story may be identified to the user by the messaging client (104) based on their location. The final result is a “live story” as described in community terms.

[0056] An additional type of content collection is known as a “location story” that enables a user with a client device (102) located within a specific geographic location (e.g., a college or university campus) to contribute to a specific collection. In some examples, contribution to a location story may require a second degree of authentication to verify whether the end user belongs to a specific organization or other entity (e.g., a student on a university campus).

[0057] As mentioned above, the video table (314) stores video data associated with messages for which records are maintained in the message table (306), in one example. Similarly, the image table (316) stores image data associated with messages for which message data is stored in the entity table (308). The entity table (308) can associate various augmentations from the augmentation table (310) with various images and videos stored in the image table (316) and the video table (314).

[0058] The avatar attribute table (318) stores a collection of available attributes to facilitate the creation / editing of avatars. In relation to the avatar system (214), the user can create, edit, and / or otherwise maintain a personalized avatar based on a selection from among the available options in the avatar attribute table (318). As previously discussed, user-selected avatar representation(s) can be stored in profile data (302) in association with user profiles.

[0059] In one or more embodiments, the avatar characteristic table (318) stores a set of available physical characteristics to represent the avatar, such as face shape, skin tone, hair features (e.g., color, length, type, style), jaw shape, eyebrows, mouth, etc. Furthermore, the avatar characteristic table (318) stores a set of available clothing items and / or outfits for the avatar, such as shirts, pants / shorts, jackets, shoes, headwear and / or other traditional clothing items (e.g., including preset combinations of clothing items corresponding to outfits).

[0060] In one or more embodiments, the avatar attribute table (318) stores attribute(s) associated with clothing item(s) and / or garments. For example, a specific clothing item may be associated with one or more predefined words using metadata labels, designations, etc. associated with clothing type (e.g., shirt, pants / shorts, jacket, shoes, or headwear), color, size, style, and / or brand name.

[0061] FIG. 4 is a schematic diagram illustrating the structure of a message (400) according to some examples, generated by a messaging client (104) for communication with an additional messaging client (104) or a messaging server (116). The content of a particular message (400) is used to populate a message table (306) stored in a database (124) accessible by the messaging server (116). Similarly, the content of the message (400) is stored in memory as "in-transit" or "in-flight" data of the client device (102) or application servers (114). The message (400) is illustrated as comprising the following exemplary components:

[0062] ● Message identifier (402): A unique identifier that identifies the message (400).

[0063] ● Message text payload (404): Text generated by the user through the user interface of the client device (102) and included in the message (400).

[0064] ● Message image payload (406): Image data included in the message (400) that is captured by the camera component of the client device (102) or retrieved from the memory component of the client device (102). Image data for the transmitted or received message (400) may be stored in an image table (316).

[0065] ● Message video payload (408): Video data captured by the camera component or retrieved from the memory component of the client device (102) and included in the message (400). Video data for the transmitted or received message (400) may be stored in the video table (314).

[0066] ● Message audio payload (410): Audio data captured by a microphone or retrieved from a memory component of a client device (102) and included in a message (400).

[0067] ● Message augmentation data (412): Augmentation data (e.g., filters, stickers, or other annotations or enhancements) representing augmentations to be applied to the message image payload (406), message video payload (408), or message audio payload (410) of the message (400). Augmentation data for a transmitted or received message (400) may be stored in an augmentation table (310).

[0068] ● Message duration parameter (414): A parameter value indicating the amount of time in seconds during which the content of a message (e.g., message image payload (406), message video payload (408), message audio payload (410)) is presented to or made accessible to the user through the messaging client (104).

[0069] ● Message geolocation parameter (416): Geolocation data (e.g., latitude and longitude coordinates) associated with the content payload of the message. Multiple message geolocation parameter (416) values ​​may be included in the payload, and each of these parameter values ​​is associated with content items included in the content (e.g., a specific image in the message image payload (406), or a specific video in the message video payload (408)).

[0070] ● Message Story Identifier (418): Identifier values ​​that identify one or more content collections (e.g., “stories”) identified in the story table (312) to which a specific content item within the message image payload (406) of the message (400) is associated. For example, multiple images within the message image payload (406) may each be associated with multiple content collections using identifier values.

[0071] ● Message Tag (420): Each message (400) may be tagged with multiple tags, each of which indicates the subject of the content included in the message payload. For example, if a specific image included in the message image payload (406) depicts an animal (e.g., a lion), a tag value indicating the relevant animal may be included in the message tag (420). The tag values ​​may be generated manually based on user input or automatically using, for example, image recognition.

[0072] ● Message sender identifier (422): An identifier representing the user of the client device (102) to which the message (400) was created and the message (400) was sent (e.g., a messaging system identifier, an email address, or a device identifier).

[0073] ● Message recipient identifier (424): An identifier representing the user of the client device (102) to which the message (400) is addressed (e.g., a messaging system identifier, an email address, or a device identifier).

[0074] The contents (e.g., values) of the various components of the message (400) may be pointers to locations within tables where content data values ​​are stored. For example, an image value within the message image payload (406) may be a pointer (or its address) to a location within the image table (316). Similarly, values ​​within the message video payload (408) may point to data stored within the video table (314), values ​​within the message augmentations (412) may point to data stored within the augmentation table (310), values ​​within the message story identifier (418) may point to data stored within the story table (312), and values ​​within the message sender identifier (422) and message receiver identifier (424) may point to user records stored within the entity table (308).

[0075] FIG. 5 is an interaction diagram illustrating a process for updating the avatar clothing of a user of a messaging system according to some exemplary embodiments. For the purposes of explanation, the process (500) is described herein primarily with reference to the messaging client (104) of FIG. 1, and the object detection system (212) and avatar system (214) of FIG. 2. However, one or more blocks (or operations) of the process (500) may be performed by one or more other components and / or other suitable devices. Also, for the purposes of explanation, blocks (or operations) of the process (500) are described herein as occurring serially or linearly. However, multiple blocks (or operations) of the process (500) may occur in parallel or simultaneously. Also, blocks (or operations) of the process (500) do not need to be performed in the illustrated order and / or one or more blocks (or operations) of the process (500) do not need to be performed and / or may be replaced by other operations. The process (500) may be terminated when its operations are completed. Additionally, the process (500) may correspond to a method, procedure, algorithm, etc.

[0076] A messaging client (104) may be associated with a user of the messaging server system (108), and the user may be associated with a user account of the messaging server system (108). As previously discussed, the user may be identified by the messaging server system (108) based on a unique identifier (e.g., a messaging system identifier, an email address, and / or a device identifier) ​​associated with the user account for that user. Additionally, the messaging server system (108) may implement and / or operate with a social network server (120) configured to identify other users (e.g., friends) who have a relationship with a particular user.

[0077] As described in this specification, a messaging client (104) (e.g., together with a messaging server system (108)) is configured to detect clothing items in a captured image (e.g., a live video feed captured by a device camera, or an image selected from a photo library) and to present a user interface having user-selectable avatar outfits that match the detected clothing items. For example, the messaging client (104) determines attributes of the clothing items detected in the captured image (e.g., types of clothing, color, size, brand name). Then, the messaging client (104) searches a database (e.g., a database (124) including an avatar attribute table (318)) that stores available clothing items and / or outfits based on the attributes and provides for selecting one or more outfits that most closely match the attributes. The messaging client (104) presents a user interface that displays the selected outfits along with interface elements to update the user avatar based on the selected outfits.

[0078] In block 502, the messaging client (104) receives user input to identify objects depicted in a captured image. In one or more embodiments, the messaging client (104) activates the camera of the client device (102) (e.g., at startup of the messaging client (104)). The messaging client (104) allows the user to request to scan one or more items in the camera feed captured by the camera. In one or more embodiments, the messaging client (104) detects physical contact between the finger of the user's hand and an area of ​​the touchscreen for a threshold period. For example, the messaging client (104) determines that the user has touched and held their finger on the screen for a threshold time (e.g., 2 seconds).

[0079] Alternatively or additionally, the messaging client (104) receives a user selection of a dedicated scan option (e.g., a button) presented along with the camera feed. In response, the messaging client (104) captures an image being displayed on the screen and processes the image to identify multiple objects based on the image. In an alternative embodiment, the captured image may correspond to an image / video stored in association with the user of the client device (102), for example, within a photo library (e.g., associated with the operating system of the client device (102) and / or associated with a user profile for the messaging system (100)).

[0080] The messaging client (104) sends a request to the object detection system (212) to identify objects in a captured image (operation 504). The request may include the image data itself (e.g., when captured in real time and / or stored locally on the client device (102)). Alternatively, the request may refer to a remote image (e.g., when stored in association with a user profile for the messaging system (100)).

[0081] The object detection system (212) determines the attribute(s) of the object(s) in the captured image (Block 506). As previously discussed, the object detection system (212) may correspond to a subsystem of the messaging system (100) and may be supported on the client side by the messaging client (104) and / or on the server side by the application servers (114). In one or more embodiments, the detection of object(s) within the captured image may be implemented on the client side, on the server side, and / or a combination of the client side and the server side.

[0082] As previously discussed, the object detection system (212) is configured to implement or otherwise access object recognition algorithms (e.g., machine learning algorithms) configured to scan a captured image and detect / track the movement of objects within the image. For example, the object detection system (212) can detect object(s) similar to clothing item(s) in the captured image. In another example, the object detection system (212) can detect objects within the image corresponding to barcodes associated with clothing item(s) (e.g., Universal Product Code (UPC) codes, QR codes, etc.). In yet another example, the object detection system (212) can detect a combination of object(s) and / or barcode(s) depicted in the captured image corresponding to clothing.

[0083] The object detection system (212) is further configured to determine or otherwise access the attributes of the identified objects. For example, with respect to objects corresponding to clothing items, the object detection system (212) may determine attributes such as type (e.g., shirt, pants / shorts, jacket, shoes, or headwear), color, size, style, and / or brand name. As previously discussed, the object detection system (212) may determine attribute(s) corresponding to keywords sponsored by a third party. For example, a third party (e.g., a clothing manufacturer, an advertising agency) may sponsor or pay to have certain keywords (e.g., clothing-related) rank higher than others. In response to the determination that a given attribute corresponds to a sponsored keyword, the object detection system (212) may provide a higher rank to that attribute compared to other attributes.

[0084] The object detection system (212) transmits the attribute(s) of the object(s) to the messaging client (104) (operation 508). By doing so, the object detection system (212) may additionally provide ranking information of the attribute(s). Based on the received attribute(s) and / or ranking information, the messaging client (104) determines (e.g., together with the messaging server system (108)) that at least one of the received attribute(s) is related to clothing for user avatars (block 510).

[0085] The messaging client (104) transmits a request for clothing item(s) to the avatar system (214) based on at least one attribute related to the avatar clothing (e.g., one or more of clothing type, color, size, style and / or brand name) (action 512). For example, if a single clothing item is depicted in the captured image, the request may include attribute(s) related to the single clothing item. In another example, if multiple clothing items are depicted in the captured image, the request may include attribute(s) for each of the multiple clothing items.

[0086] The avatar system (214) searches for and selects clothing item(s) associated with attribute(s) received from the messaging client (104) (Block 514). As previously discussed, the avatar system (214) may correspond to a subsystem of the messaging system (100) and may be supported on the client side by the messaging client (104) and / or on the server side by the application servers (114). In one or more embodiments, the selection of clothing item(s) based on attribute(s) through the avatar system (214) may be implemented on the client side, on the server side, and / or a combination of the client side and the server side.

[0087] The avatar system (214) is configured to search the avatar attribute table (318) for one or more clothing item(s) corresponding to attribute(s) received from the messaging client (104). As previously discussed, the avatar attribute table (318) stores attribute(s) associated with clothing item(s) and / or garments. Thus, the avatar system (214) can perform the search by comparing the received attribute(s) (e.g., clothing type, color, size, style, and / or brand name) with the corresponding attribute fields stored associated with the clothing in the avatar attribute table (318).

[0088] In one or more embodiments, the avatar system (214) may prioritize higher-ranking clothing attribute(s) over lower-ranking clothing attribute(s). Where attribute(s) correspond to multiple clothing items, the avatar system (214) may select the number of each clothing item(s) based on attribute(s). Where attribute(s) correspond to a single clothing item, the avatar system (214) may select the single clothing item corresponding to the attribute(s), or alternatively, select an entire outfit including additional clothing items predefined to be associated with the selected single clothing item.

[0089] The avatar system (214) can calculate respective relevancy scores for clothing item(s) and / or entire outfits based, for example, on the number or frequency of confirmed matches between matching attributes. The relevancy scores can be used to limit the number of selected clothing items and / or outfits (e.g., a predefined number such as three outfits) and / or to display the selected clothing items and / or outfits in a ranked order.

[0090] When the search is complete and / or a predetermined number of matching clothing items are selected, the avatar system (214) transmits a display of the selected clothing item(s) to the messaging client (104), along with relevance scores, where applicable (action 516). The messaging client (104) displays the selected clothing item(s) along with user interface options for updating the user's avatar (block 518). The messaging client (104) receives user input selecting to update the user's avatar (block 520).

[0091] As discussed below in relation to FIGS. 6a and 6b, user interface options may include an option (e.g., a first button) for the user to update their avatar based on a user selection of one or more clothing items provided by the avatar system (214). Alternatively or additionally, user interface options may include an option (e.g., a second button) for the user to redirect to an outfit selection interface. In one or more embodiments, the outfit selection interface is configured to present an extended set of clothing item(s) and / or outfits (e.g., all available clothing item(s) in the avatar attribute table (318) in contrast to the selected clothing items provided by the avatar system (214).

[0092] The messaging client (104) transmits a display of user selection to the avatar system (214) (action 522). The avatar system (214) provides to update the user's avatar based on the display of user selection received from the messaging client (104) (block 524). For example, if the user selects a first button, the avatar system (214) provides to update the user's avatar based on a user selection of one or more clothing items provided by the avatar system (214). In another example, if the user selects a second button, the avatar system (214) provides to update the avatar based on a user selection of clothing options presented within an extended clothing selection interface. The avatar system (214) is configured to update the avatar so that the updated clothing persists across multiple interfaces of the messaging system (100) (e.g., by being stored in association with the user's profile within profile data (302)).

[0093] FIG. 6a illustrates an exemplary user interface (600a) for detecting clothing in a captured image (602) according to some exemplary embodiments. The user interface (600a) includes a camera select button (604), a flash button (606), a carousel interface launch button (614), and a shutter button (616), as well as detected clothing items including a shirt (608), pants (610), and shoes (612).

[0094] In the example of FIG. 6a, a user of the client device (102) captures an image (e.g., a live video feed) from the rear camera of the client device (102). In this example, the captured image (602) corresponds to the user's reflection in a mirror. As illustrated, the captured image (602) depicts the user wearing clothing including a shirt (608), pants (610), and shoes (612).

[0095] The example in FIG. 6a illustrates a captured image (602) as captured by the rear camera of the client device (102), but it is possible for the messaging client (104) to capture an image (e.g., a live video feed) from the front camera instead. For example, the camera select button (604) corresponds to a user-selectable button for switching between the rear camera and the front camera of the client device (102).

[0096] The user interface (600a) further includes a flash button (606) for enabling or disabling the flash for a captured image (602). Additionally, the user interface (600a) includes a carousel interface launch button (614) for launching the carousel interface (626) discussed below in relation to FIG. 6B. Additionally, the user interface (600a) includes a shutter button (616) for creating a media content item including an image (e.g., in response to a press / tap gesture of the shutter button (616)) and / or a video (e.g., in response to a press-and-hold gesture of the shutter button (616)) of screen content to be sent to friends, for example, in a story, etc.

[0097] As discussed above, the user of the client device (102) may provide touch input to the messaging client (104) to perform a scanning operation to identify objects in the captured image (602). For example, the touch input corresponds to a press and hold gesture (e.g., for at least 1 second) on a predefined part of the user interface (600a) (e.g., a part of the device screen where the captured image (602) is displayed).

[0098] In one or more embodiments, during a scan operation, the messaging client (104) is configured to display a scanning graphic (not shown) to indicate that the messaging client (104) is performing a scan operation. For example, the scanning graphic corresponds to an animation displayed during the duration of the scan (e.g., a predetermined duration of 2 seconds).

[0099] In response to receiving touch input, the messaging client (104) (e.g., together with the object detection system (212)) is configured to detect objects in the captured image (602) as well as detect attribute(s) of the detected objects. For example, the object detection system (212) detects attribute(s) associated with a shirt (608), pants (610), and shoes (612). In response to determining that the attribute(s) are associated with clothing, the messaging client (104) (e.g., together with the avatar system (214)) is configured to select avatar clothing items associated with the attributes of the detected shirt (608), pants (610), and shoes (612). Additionally, the messaging client (104) is configured to display the selected clothing items with the user's avatar, as discussed below in relation to FIG. 6b.

[0100] FIG. 6b illustrates an exemplary user interface (600b) for selecting avatar clothing based on detected clothing according to some exemplary embodiments. The user interface (600b) includes a captured image (602), a clothing interface launch button (618), a clothing suggestion list (620), a clothing update button (622), supplementary clothing information (624), a carousel interface (626), and a scan button (630).

[0101] The outfit suggestion list (620) includes user-selectable outfits for updating the user's avatar. As previously discussed, the avatar system (214) may have selected a preset number of clothing items and / or outfits (e.g., three outfits) corresponding to the attributes of the detected shirt (608), pants (610), and shoes (612). The three selected outfits are presented in the outfit suggestion list (620).

[0102] The outfit update button (622) corresponds to a user-selectable button for updating the user's avatar based on an outfit selected within the outfit suggestion list (620). In response to the selection of the outfit update button (622), the avatar system (214) provides for continuously updating the user's avatar with the selected outfit (e.g., in association with the user's profile in the profile data (302)).

[0103] Meanwhile, the outfit interface launch button (618) corresponds to a user-selectable button for redirecting to an outfit selection interface within the messaging client (104). As previously discussed, the outfit selection interface is configured to present an extended set of clothing item(s) and / or outfits for continuously updating the outfit for the user's avatar (e.g., in association with the user's profile in the profile data (302)).

[0104] As illustrated in the example of FIG. 6b, the user interface (600b) further includes supplementary outfit information (624). The supplementary outfit information (624) may correspond to an overlay having a phrase (e.g., "outfit of the day") to supplement outfit selections / suggestions associated with the interface elements (618-622).

[0105] In one or more embodiments, the user interface (600b) includes a carousel interface (626) that allows the user to cycle through and / or select different augmented reality content items to apply / display with respect to the captured image (602). Each available augmented reality content item is represented by a user-selectable icon to switch to its respective augmented reality content item. In one or more embodiments, an icon corresponding to an active augmented reality content item (e.g., a selected AR icon (628)) is displayed in a different way (e.g., larger than) the remaining icons. The user selection of the selected AR icon (628) provides for creating a media content item that includes an image of the screen content (e.g., in response to a press / tap gesture of the selected AR icon (628)) and / or a video (e.g., in response to a press / hold gesture of the selected AR icon (628)) to be sent to friends, for example, in a story, etc.

[0106] Additionally, the user interface (600b) includes a scan button (630). For example, the scan button (630) corresponds to a dedicated button for the user to perform a subsequent scan action (for example, as an alternative to the aforementioned press and hold gesture for scanning).

[0107] FIG. 7 illustrates another exemplary user interface (700) for detecting clothing in a captured image (702) according to some exemplary embodiments. The user interface (700) includes a captured image (702), a flash button (706), a scan prompt (708), a photo selector (710), an avatar scan icon (714), and a scan button (716).

[0108] As previously discussed, the carousel interface (712) allows the user to cycle through and / or select different augmented reality content items to apply / display to the captured image (702). One such augmented reality content item is represented by an avatar scan icon (714). User selection of the avatar scan icon (714) enables the user to initiate a scan operation. For example, in response to user selection of the avatar scan icon (714), the messaging client (104) provides a display of a scan prompt (708) and a photo picker (710). The scan prompt (708) prompts the user to perform a scan operation on a live video feed (e.g., via the aforementioned press and hold gesture). Meanwhile, the photo picker (710) enables the user to perform a scan operation on a selected image / video stored in memory. As previously discussed, the stored image / video may be selected from a photo library stored locally on the device and / or stored from a photo library associated with the user's profile for the messaging system (100).

[0109] In response to user input associated with a scan prompt (708) and / or a photo selector (710), the messaging client (104) determines the attribute(s) of the object(s) (e.g., jacket, scarf, sunglasses) depicted in the captured image (702), selects the clothing item associated with the attribute(s) of the object(s), and provides for displaying the selected clothing item(s) with the user's avatar (e.g., similar to FIG. 6b above). In the example of FIG. 7, the user interface (700) further includes a camera select button (704) for switching between the front camera and the rear camera, a flash button (706) for enabling and / or disabling the flash for the captured image (702), and a scan button (716) for performing a subsequent scan operation.

[0110] FIG. 8 is a flowchart illustrating a process (800) for updating the avatar clothing of a user of a messaging system according to some exemplary embodiments. For the purposes of explanation, the process (800) is described herein primarily with reference to the messaging client (104) of FIG. 1, and the object detection system (212) and avatar system (214) of FIG. 2. However, one or more blocks (or operations) of the process (800) may be performed by one or more other components and / or other suitable devices. Also, for the purposes of explanation, blocks (or operations) of the process (800) are described herein as occurring serially or linearly. However, multiple blocks (or operations) of the process (800) may occur in parallel or simultaneously. Also, blocks (or operations) of the process (800) do not need to be performed in the illustrated order and / or one or more blocks (or operations) of the process (800) do not need to be performed and / or may be replaced by other operations. The process (800) may be terminated when its operations are completed. Additionally, the process (800) may correspond to a method, procedure, algorithm, etc.

[0111] A messaging client (104) receives a request to identify at least one object depicted in an image captured by a device camera (block 802). The image may correspond to a live feed of the device camera. Alternatively or additionally, the image may correspond to a stored image included in a photo library associated with the user.

[0112] The messaging client (104) determines the attributes of the object depicted in the image in response to receiving the request (block 804). The messaging client (104) selects a clothing item associated with the attributes of the object from a plurality of candidate clothing items for the user's avatar (block 806).

[0113] The object depicted in the image may resemble (or correspond to) a clothing item. Alternatively, or additionally, the object depicted in the image may correspond to a barcode associated with a clothing item.

[0114] The messaging client (104) displays a selected clothing item along with the user's avatar (block 808). The messaging client (104) may display a user-selectable interface element for updating the avatar's outfit with the clothing item. The messaging client (104) may receive a user selection of the user-selectable interface element and, in response to receiving the user selection, update the user's profile so that the clothing item is continuously associated with the avatar's outfit. Alternatively or additionally, the messaging client (104) may display a second user-selectable element for redirecting to an outfit selection interface for selecting the avatar's outfit, receive a user selection of the second user-selectable element, and, in response to receiving the user selection, redirect to the outfit selection interface.

[0115] A messaging client (104) can, in response to receiving a request, determine at least one second attribute of at least one second object depicted in an image, select at least one second clothing item associated with at least one second attribute of at least one second object from a plurality of candidate clothing items for a user's avatar, and display the selected clothing item and at least one second clothing item together with the user's avatar.

[0116] FIG. 9 is a schematic diagram illustrating an access-restriction process (900) in which access to content (e.g., short-term message (910), and associated multimedia payload of data) or a collection of content (e.g., group of short-term messages (906)) may be time-restricted (e.g., short-term).

[0117] A short message (910) is illustrated as being associated with a message duration parameter (914), the value of which determines the amount of time the short message (910) is displayed to the recipient user of the short message (910) by the messaging client (104). In one example, depending on the amount of time specified by the sending user using the message duration parameter (914), the recipient user may view the short message (910) for up to 10 seconds.

[0118] The message duration parameter (914) and the message recipient identifier (424) are illustrated as inputs to the message timer (912), and the message timer (912) is responsible for determining the amount of time that a short-term message (910) is shown to a specific recipient user identified by the message recipient identifier (424). In particular, the short-term message (910) will be shown only to the relevant recipient user for a period determined by the value of the message duration parameter (914). The message timer (912) is illustrated as providing an output to a more generalized short-term timer system (202) responsible for the overall timing of the display of content (e.g., the short-term message (910)) to the recipient user.

[0119] A short message (910) is illustrated in FIG. 9 as being included within a short message group (906) (e.g., a collection of messages in a personal story or event story). The short message group (906) has an associated group duration parameter (904), the value of which determines the time duration for which the short message group (906) is presented and accessible to users of the messaging system (100). For example, the group duration parameter (904) may be the duration of a music concert, where the short message group (906) is a collection of content related to that concert. Alternatively, a user (owner user or curator user) may specify a value for the group duration parameter (904) when setting up and creating the short message group (906).

[0120] Additionally, each short message (910) within a short message group (906) has an associated group participation parameter (902), the value of which determines the duration of time the short message (910) is accessible within the context of the short message group (906). Thus, a specific short message group (906) may "expire" and become inaccessible within the context of the short message group (906) before the short message group (906) itself expires with respect to the group duration parameter (904). The group duration parameter (904), the group participation parameter (902), and the message recipient identifier (424) each provide input to a group timer (908), which first determines whether a specific short message (910) of the short message group (906) will be displayed to a specific recipient user, and if so, for how long. Note that the short-term message group (906) also recognizes the identity of a specific recipient user as a result of the message recipient identifier (424).

[0121] Accordingly, the group timer (908) operatively controls the entire lifespan of the associated short-term message group (906) as well as the individual short-term messages (910) included in the short-term message group (906). In one example, each and all short-term messages (910) within the short-term message group (906) remain visible and accessible for a period specified by the group duration parameter (904). In an additional example, a specific short-term message (910) may expire within the context of the short-term message group (906) based on the group participation parameter (902). Note that the message duration parameter (914) can still determine the duration of time during which a specific short-term message (910) is displayed to the receiving user, even within the context of the short-term message group (906). Accordingly, the message duration parameter (914) determines the duration of time that a specific short message (910) is displayed to a receiving user, regardless of whether the receiving user is viewing the short message (910) inside or outside the context of the short message group (906).

[0122] The short-term timer system (202) may also operatively remove a specific short-term message (910) from a short-term message group (906) based on the determination that it has exceeded the associated group participation parameter (902). For example, when the sending user establishes a group participation parameter (902) of 24 hours from the posting, the short-term timer system (202) will remove the associated short-term message (910) from the short-term message group (906) after the specified 24 hours. The short-term timer system (202) also operates to remove the short-term message group (906) when the group participation parameter (902) for each and all short-term messages (910) within the short-term message group (906) has expired, or when the short-term message group (906) itself has expired with respect to the group duration parameter (904).

[0123] In certain usage cases, the creator of a specific short-term message group (906) may specify an indefinite group duration parameter (904). In this case, the expiration of the group participation parameter (902) for the last remaining short-term message (910) within the short-term message group (906) will determine when the short-term message group (906) itself expires. In this case, a new short-term message (910) added to the short-term message group (906) with a new group participation parameter (902) effectively extends the life of the short-term message group (906) to be equal to the value of the group participation parameter (902).

[0124] In response to the short-term timer system (202) determining that a short-term message group (906) has expired (e.g., is no longer accessible), the short-term timer system (202) communicates with the messaging system (100) (and, for example, specifically the messaging client (104)) to prevent the indicator (e.g., an icon) associated with the relevant short-term message group (906) from being displayed within the user interface of the messaging client (104). Similarly, when the short-term timer system (202) determines that the message duration parameter (914) for a specific short-term message (910) has expired, the short-term timer system (202) prevents the messaging client (104) from displaying the indicator (e.g., an icon or text identification) associated with the short-term message (910) from being displayed.

[0125] FIG. 10 is a schematic representation of a machine (1000) on which instructions (1006) (e.g., software, program, application, applet, app, or other executable code) can be executed to cause the machine (1000) to perform any one or more of the methodologies discussed herein. For example, instructions (1006) can cause the machine (1000) to perform any one or more of the methods described herein. Instructions (1006) convert a general non-programmed machine (1000) into a specific machine (1000) programmed to perform the described and illustrated functions in the described manner. The machine (1000) may operate as a standalone device or may be coupled to other machines (e.g., networked). In a networked deployment, the machine (1000) may operate as a server machine or client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine (1000) may include, but is not limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a personal digital assistant (PDA), an entertainment media system, a cellular phone, a smartphone, a mobile device, a wearable device (e.g., a smartwatch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of executing commands (1006) that specify actions to be taken by the machine (1000) sequentially or otherwise.Furthermore, although only a single machine (1000) is exemplified, the term “machine” should also be considered to include a collection of machines that execute instructions (1006) individually or jointly to perform any one or more of the methodologies discussed herein. For example, the machine (1000) may include a client device (102) or any one of a plurality of server devices forming part of a messaging server system (108). In some examples, the machine (1000) may also include both client and server systems, where specific operations of a specific method or algorithm are performed on the server side and specific operations of a specific method or algorithm are performed on the client side.

[0126] The machine (1000) may include processors (1002), memory (1010), and I / O (input / output) components (1022) that can be configured to communicate with each other via a bus (1020). In one example, the processors (1002) (e.g., a CPU (Central Processing Unit), a RISC (Reduced Instruction Set Computing) processor, a CISC (Complex Instruction Set Computing) processor, a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an RFIC (Radio-Frequency Integrated Circuit), other processors, or any suitable combination thereof) may include, for example, a processor (1004) and a processor (1008) that execute instructions (1006). The term “processor” is intended to include multi-core processors that may include two or more independent processors (sometimes referred to as “cores”) capable of executing instructions simultaneously. FIG. 10 illustrates multiple processors (1002), but the machine (1000) may include a single processor having a single core, a single processor having multiple cores (e.g., a multi-core processor), multiple processors having a single core, multiple processors having multiple cores, or any combination thereof.

[0127] The memory (1010) includes a main memory (1012), a static memory (1014), and a storage unit (1016), both of which are accessible to processors (1002) via a bus (1020). The main memory (1010), the static memory (1014), and the storage unit (1016) store instructions (1006) that implement any one or more of the methodologies or functions described herein. The instructions (1006) may also exist, wholly or partially, during the execution by the machine (1000), in the main memory (1012), in the static memory (1014), in a machine-readable medium (1018) within the storage unit (1016), in at least one of the processors (1002) (e.g., in the processor's cache memory), or any suitable combination thereof.

[0128] The I / O components (1022) may include a wide variety of components for receiving inputs, providing outputs, generating outputs, transmitting information, exchanging information, capturing measurements, etc. The specific I / O components (1022) included in a particular machine will depend on the type of machine. For example, portable machines such as mobile phones may include touch input devices or other such input mechanisms, while headless server machines are unlikely to include such touch input devices. It will be acknowledged that the I / O components (1022) may include many other components not shown in FIG. 10. In various examples, the I / O components (1022) may include user output components (1024) and user input components (1026). User output components (1024) may include visual components (e.g., a display such as a plasma display panel (PDP), a light-emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), acoustic components (e.g., a speaker), haptic components (e.g., a vibration motor, a resistance mechanism), other signal generators, etc. User input components (1026) may include alphanumeric input components (e.g., a keyboard, a touch screen configured to receive alphanumeric input, a photo-optical keyboard, or other alphanumeric input components), point-based input components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or other pointing mechanism), tactile input components (e.g., a physical button, a touch screen providing the location and force of a touch or touch gesture, or other tactile input components), audio input components (e.g., a microphone), etc.

[0129] In additional examples, I / O components (1022) may include, among various other components, biometric components (1028), motion components (1030), environment components (1032), or position components (1034). For example, biometric components (1028) include components for detecting expressions (e.g., hand expressions, facial expressions, voice expressions, body gestures, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweat, or brain waves), identifying a person (e.g., voice identification, retinal identification, face identification, fingerprint identification, or EEG-based identification), and doing similar things. Motion components (1030) include acceleration sensor components (e.g., accelerometers), gravity sensor components, and rotation sensor components (e.g., gyroscopes).

[0130] Environmental components (1032) may include, for example, one or more cameras (having still image / photo and video capabilities), light sensor components (e.g., photometers), temperature sensor components (e.g., one or more thermometers for detecting ambient temperature), humidity sensor components, pressure sensor components (e.g., barometers), acoustic sensor components (e.g., one or more microphones for detecting background noise), proximity sensor components (e.g., infrared sensors for detecting nearby objects), gas sensors (e.g., gas detection sensors for detecting concentrations of hazardous gases for safety or measuring contaminants in the atmosphere), or other components capable of providing indications, measurements, or signals corresponding to the surrounding physical environment.

[0131] With respect to the cameras, the client device (102) may have a camera system including, for example, front cameras on the front surface of the client device (102) and rear cameras on the rear surface of the client device (102). The front cameras may be used, for example, to capture still images and videos (e.g., "selfies") of the user of the client device (102), which may then be augmented with the augmentation data (e.g., filters) described above. The rear cameras may be used, for example, to capture still images and videos in a more traditional camera mode, and these images are similarly augmented with augmentation data. In addition to the front and rear cameras, the client device (102) may also include a 360° camera for capturing 360° photos and videos.

[0132] Additionally, the camera system of the client device (102) may include dual rear cameras (e.g., a depth-sensing camera as well as a main camera) or even triple, quadruple, or quintuple rear camera configurations on the front and rear sides of the client device (102). These multiple camera systems may include, for example, a wide camera, an ultra-wide camera, a telephoto camera, a macro camera, and a depth sensor.

[0133] Position components (1034) include position sensor components (e.g., GPS receiver components), altitude sensor components (e.g., altimeters or barometers that detect air pressure from which altitude can be derived), orientation sensor components (e.g., magnetometers), and similar ones.

[0134] Communication can be implemented using a wide variety of technologies. The I / O components (1022) further include communication components (1036) operable to connect the machine (1000) to a network (1040) or devices (1038) through their respective combinations or connections. For example, the communication components (1036) may include a network interface component, or other suitable device for interfacing with the network (1040). In additional examples, the communication components (1036) may include wired communication components, wireless communication components, cellular communication components, near-field communication (NFC) components, Bluetooth ® Components (e.g., Bluetooth) ® Low Energy), Wi-Fi ® It may include other communication components that provide communication through components and other aspects. The devices (1038) may be other machines or any various peripheral devices (e.g., peripheral devices connected via USB).

[0135] Furthermore, the communication components (1036) may include components capable of detecting identifiers or operable to detect identifiers. For example, the communication components (1036) may include Radio Frequency Identification (RFID) tag reader components, NFC smart tag detection components, optical reader components (e.g., optical sensors for detecting one-dimensional barcodes such as Universal Product Code (UPC) barcodes, multi-dimensional barcodes such as Quick Response (QR) codes, Aztec codes, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, UCC RSS-2D barcodes, and other optical codes), or acoustic detection components (e.g., microphones for identifying tagged audio signals). In addition, various information, such as location via Internet Protocol (IP) geolocation, location via Wi-Fi® signal triangulation, and location via detection of NFC beacon signals that can indicate a specific location, can be derived through the communication components (1036).

[0136] Various memories (e.g., main memory (1012), static memory (1014), and memory of the processors (1002)) and storage unit (1016) may store one or more sets of instructions and data structures (e.g., software) that implement any one or more of the methodologies or functions described herein. These instructions (e.g., instructions (1006)) enable various operations to implement the disclosed examples when executed by the processors (1002).

[0137] Commands (1006) may be transmitted or received over a network (1040) using a transmission medium through a network interface device (e.g., a network interface component included in the communication components (1036)) and using any one of several well-known transmission protocols (e.g., HTTP (hypertext transfer protocol)). Similarly, commands (1006) may be transmitted or received using a transmission medium through a connection to the devices (1038) (e.g., peer-to-peer connection).

[0138] FIG. 11 is a block diagram (1100) illustrating a software architecture (1102) that may be installed in any one or more of the devices described herein. The software architecture (1102) is supported by hardware such as a machine (1108) including processors (1148), memory (1150), and I / O components (1152). In this example, the software architecture (1102) may be conceptualized as a stack of layers, each layer providing a specific function. The software architecture (1102) includes layers such as an operating system (1116), libraries (1114), frameworks (1112), and applications (1110). Operationally, applications (1110) invoke API calls (1104) through the software stack and receive messages (1106) in response to the API calls (1104).

[0139] The operating system (1116) manages hardware resources and provides common services. The operating system (1116) includes, for example, a kernel (1142), services (1144), and drivers (1146). The kernel (1142) acts as an abstraction layer between the hardware and other software layers. For example, the kernel (1142) provides memory management, processor management (e.g., scheduling), component management, networking, and security settings, among other functions. The services (1144) may provide other common services for other software layers. The drivers (1146) are responsible for controlling or interfacing with the underlying hardware. For example, drivers (1146) may include display drivers, camera drivers, BLUETOOTH® or BLUETOOTH® Low Energy drivers, flash memory drivers, serial communication drivers (e.g., USB drivers), WI-FI® drivers, audio drivers, power management drivers, etc.

[0140] Libraries (1114) provide common low-level infrastructure used by applications (1110). Libraries (1114) may include system libraries (1136) (e.g., C standard library) that provide functions such as memory allocation functions, string manipulation functions, mathematical functions, etc. Additionally, the libraries (1114) may include API libraries (1138) such as media libraries (e.g., libraries supporting the presentation and manipulation of various media formats such as MPEG4 (Moving Picture Experts Group-4), Advanced Video Coding (H.264 or AVC), MP3 (Moving Picture Experts Group Layer-3), AAC (Advanced Audio Coding), AMR (Adaptive Multi-Rate) audio codecs, Joint Photographic Experts Group (JPEG or JPG), or PNG (Portable Network Graphics), graphics libraries (e.g., OpenGL frameworks used to render graphic content on a display in two dimensions (2D) and three dimensions (3D)), database libraries (e.g., SQLite providing various relational database functions), web libraries (e.g., WebKit providing web browsing functionality). The libraries (1114) may also include a wide variety of other libraries (1140) that provide many different APIs to the applications (1110).

[0141] Frameworks (1112) provide common high-level infrastructure used by applications (1110). For example, frameworks (1112) provide various graphical user interface (GUI) functions, high-level resource management, and high-level location services. Frameworks (1112) may provide a wide range of other APIs that can be used by applications (1110), some of which may be specific to a particular operating system or platform.

[0142] In one example, applications (1110) may include a wide range of other applications such as a home application (1118), a contact application (1124), a browser application (1128), a book reader application (1132), a location application (1120), a media application (1126), a messaging application (1130), a game application (1134), and a third-party application (1122). Applications (1110) are programs that execute functions defined in programs. Various programming languages ​​may be used to create one or more of applications (1110) that are structured in various ways, such as an object-oriented programming language (e.g., Objective-C, Java, or C++) or a procedural programming language (e.g., C or assembly language). In a specific example, a third-party application (1122) (e.g., an application developed by an entity other than a vendor of a specific platform using an ANDROID™ or IOS™ software development kit (SDK)) may be mobile software running on a mobile operating system such as IOS™, ANDROID™, WINDOWS® Phone, or other mobile operating systems. In this example, the third-party application (1122) may invoke API calls (1104) provided by the operating system (1116) to facilitate the functions described herein.

[0143] "Carrier signal" refers to any intangible medium capable of storing, encoding, or carrying instructions for execution by a machine, and includes digital or analog communication signals or other intangible media to facilitate the communication of such instructions. Instructions may be transmitted or received over a network using a transmission medium through a network interface device.

[0144] "Client device" refers to any machine that interfaces with a communication network to obtain resources from one or more server systems or other client devices. A client device may be, but is not limited to, a mobile phone, a desktop computer, a laptop, PDAs (portable digital assistants), smartphones, tablets, ultrabooks, netbooks, laptops, multi-processor systems, microprocessor-based or programmable consumer electronics, game consoles, set-top boxes, or any other communication device that a user can use to access the network.

[0145] "Communication network" refers to one or more parts of a network that may be an ad-hoc network, intranet, extranet, VPN (virtual private network), LAN (local area network), wireless LAN (WLAN), WAN (wide area network), wireless WAN (WWAN), MAN (metropolitan area network), the Internet, part of the Internet, part of the PSTN (Public Switched Telephone Network), POTS (plain old telephone service) network, cellular telephone network, wireless network, Wi-Fi® network, other types of networks, or a combination of two or more of these networks. For example, a network or part of a network may include a wireless or cellular network, and a coupling may be a CDMA (Code Division Multiple Access) connection, a GSM (Global System for Mobile communications) connection, or other types of cellular or wireless coupling.In this example, the combination can implement any of the various types of data transmission technologies, such as 1xRTT (Single Carrier Radio Transmission Technology), EVDO (Evolution-Data Optimized) technology, GPRS (General Packet Radio Service) technology, EDGE (Enhanced Data rates for GSM Evolution) technology, 3GPP (third Generation Partnership Project) including 3G, 4th generation wireless (4G) networks, UMTS (Universal Mobile Telecommunications System), HSPA (High Speed ​​Packet Access), WiMAX (Worldwide Interoperability for Microwave Access), LTE (Long Term Evolution) standards, other things defined by various standard-setting organizations, other long-range protocols, or other data transmission technologies.

[0146] "Component" refers to a device, physical entity, or logic having boundaries defined by function or subroutine calls, branch points, APIs, or other techniques that provide the division or modularization of specific processing or control functions. Components may be combined with other components through their interfaces to execute machine processes. A component may be a packaged functional hardware unit designed to be used with other components and part of a program that performs a specific function among the related functions. Components may constitute either software components (e.g., code implemented on a machine-readable medium) or hardware components. "Hardware component" is a type unit capable of performing specific operations and may be configured or arranged in a specific physical manner. In various exemplary embodiments, one or more computer systems (e.g., standalone computer systems, client computer systems, or server computer systems) or one or more hardware components of a computer system (e.g., a processor or a group of processors) may be configured by software (e.g., an application or a part of an application) as hardware components that operate to perform specific operations as described herein. Hardware components may also be implemented mechanically, electronically, or any suitable combination thereof. For example, a hardware component may include dedicated circuits or logic permanently configured to perform specific operations. A hardware component may be a special-purpose processor, such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). A hardware component may also include programmable logic or circuits that are temporarily configured by software to perform specific operations.For example, a hardware component may include software executed by a general-purpose processor or another programmable processor. Once configured by such software, the hardware components become specific machines (or specific components of a machine) uniquely customized to perform the configured functions and are no longer general-purpose processors. It will be recognized that the decision to implement a hardware component mechanically, in a dedicated, permanently configured circuit, or in a temporarily configured circuit (e.g., configured by software) may be driven by cost and time considerations. Accordingly, the phrase "hardware component" (or "hardware-implemented component") should be understood to encompass type entities, that is, entities that are physically configured, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a particular manner or perform the specific operations described herein. When considering embodiments where hardware components are temporarily configured (e.g., programmed), it is not necessary for each hardware component to be configured or instantiated at any single time instance. For example, in the case where a general-purpose processor is configured by software such that a hardware component becomes a special-purpose processor, the general-purpose processor may be configured as different special-purpose processors at different times (e.g., including different hardware components). Thus, the software configures a specific processor or processors to configure a specific hardware component at one time instance and a different hardware component at a different time instance. Hardware components can provide information to other hardware components and receive information from them.Accordingly, the described hardware components may be considered to be communicably coupled. In cases where multiple hardware components exist simultaneously, communication may be achieved through the transmission of signals between or between two or more of the hardware components (e.g., via appropriate circuits and buses). In embodiments where multiple hardware components are configured or instantiated at different times, communication between such hardware components may be achieved, for example, through the storage and retrieval of information within memory structures accessible to multiple hardware components. For example, one hardware component may perform an operation and store the output of that operation in a memory device communicably coupled thereto. Subsequently, additional hardware components may access the memory device to retrieve and process the stored output. Hardware components may also initiate communication with input or output devices and manipulate resources (e.g., collections of information). Various operations of the exemplary methods described herein may be performed at least partially by one or more processors configured temporarily (e.g., by software) or permanently to perform the relevant operations. Whether configured temporarily or permanently, such processors may constitute processor-implemented components that operate to perform one or more operations or functions described herein. As used herein, "processor-implemented component" refers to a hardware component implemented using one or more processors. Similarly, the methods described herein may be implemented at least partially by a processor, and specific processors or processors are examples of hardware.For example, at least some of the operations of the method may be performed by one or more processors (1004) or components implemented by processors. Furthermore, one or more processors may also operate to support the performance of the relevant operations in a "cloud computing" environment or as "software as a service" (SaaS). For example, at least some of the operations may be performed by a group of computers (as examples of machines including processors), and these operations may be accessible via a network (e.g., the Internet) and through one or more appropriate interfaces (e.g., APIs). The performance of certain operations may not only exist within a single machine but may also be distributed among processors deployed across multiple machines. In some exemplary embodiments, the processors or components implemented by processors may be located in a single geographic location (e.g., a home environment, an office environment, or within a server farm). In other exemplary embodiments, the processors or components implemented by processors may be distributed across multiple geographic locations.

[0147] "Computer-readable storage medium" refers to a machine storage medium. Accordingly, the terms include storage devices / mediums. The terms "machine-readable medium," "computer-readable medium," and "device-readable medium" mean the same thing and may be used interchangeably in this disclosure.

[0148] An "ephemeral message" refers to a message accessible for a time-limited duration. Ephemeral messages can be text, images, videos, etc. The access time for an ephemeral message can be set by the message sender. Alternatively, the access time can be a default setting or a setting specified by the recipient. Regardless of the setting method, the message is transitory.

[0149] "Machine storage medium" refers to a single or multiple storage devices and media (e.g., centralized or distributed databases, and associated caches and servers) that store executable instructions, routines, and data. Accordingly, the term should be considered to include, but not be limited to, solid-state memories including memory inside or outside processors, and optical and magnetic media. Specific examples of machine storage medium, computer storage medium, and device storage medium include, by example, non-volatile memory including semiconductor memory devices, e.g., EPROM (erasable programmable read-only memory), EEPROM (electrically erasable programmable read-only memory), FPGAs, and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The terms "machine storage medium," "device storage medium," and "computer storage medium" mean the same thing and may be used interchangeably in this disclosure. The terms "machine storage medium," "computer storage medium," and "device storage medium" specifically exclude carriers, modulated data signals, and other such media, except that at least some of the carriers, modulated data signals, and other such media are included under the term "signal medium."

[0150] "Non-transient computer-readable storage medium" refers to a type of medium capable of storing, encoding, or carrying instructions for execution by a machine.

[0151] "Signal medium" refers to any intangible medium capable of storing, encoding, or carrying instructions for execution by a machine, and includes digital or analog communication signals or other intangible media for facilitating the communication of software or data. The term "signal medium" should be considered to include any form of modulated data signal, carrier wave, etc. The term "modulated data signal" means a signal in which one or more of its characteristics are set or altered, such as in the matter of encoding information within the signal. The terms "transmission medium" and "signal medium" mean the same thing and may be used interchangeably in this disclosure.

Claims

Claim 1 As a method, a messaging application running on a device associated with a user displays a user interface in which the camera of the device is activated to capture a real-time image, wherein the user interface includes a carousel interface comprising a plurality of icons, wherein the plurality of icons include a first icon user-selectable to update the user's avatar for the messaging application based on performing a scan operation, wherein the plurality of icons further include second icons, each of which is user-selectable to apply to respective augmented reality content in association with the real-time image; wherein the messaging application receives a user input selecting the first icon within the carousel interface; in response to receiving the user input: a step of displaying a scan prompt user-selectable to perform the scan operation with respect to the real-time image; and a step of displaying a photo selector for user selection of a stored image to perform the scan operation; wherein, in response to the user selection of the scan prompt, the attribute of an object depicted in the real-time image is determined; and the step of selecting a clothing item associated with the attribute of the object from a plurality of candidate clothing items (articles of clothing) for the user's avatar. A method comprising the step of displaying the selected clothing item together with the user's avatar. Claim 2 A method according to claim 1, further comprising: displaying a user-selectable outfit update button to continuously update the user's avatar based on the clothing item; receiving a user selection of the outfit update button; and in response to receiving the user selection, updating the user's profile so that the clothing item is continuously associated with the avatar's outfit. Claim 3 A method according to claim 1, further comprising: determining at least one second attribute of at least one second object depicted in the image in response to a user selection of the scan prompt; selecting at least one second clothing item associated with the at least one second attribute of the at least one second object from the plurality of candidate clothing items for the user's avatar; and displaying the selected clothing item and the at least one second clothing item together with the user's avatar. Claim 4 In claim 1, the object depicted in the image is similar to the clothing item. Claim 5 In claim 1, the object depicted in the image corresponds to a barcode associated with the clothing item. Claim 6 A method according to claim 1, wherein the real-time image corresponds to the live feed of the device camera. Claim 7 A method according to claim 1, wherein the stored image is included in a photo library associated with the user. Claim 8 A method according to claim 1, further comprising the step of displaying a user interface including a clothing suggestion list, a clothing update button, and a clothing interface launch button, wherein the clothing suggestion list presents clothing items for updating the user’s avatar, and at least one of the clothing items includes the selected clothing item, the clothing update button is user-selectable to continuously update the avatar based on the user's selection of clothing within the clothing suggestion list, and the clothing interface launch button is user-selectable to launch a clothing selection interface configured to present an expanded set of clothing items for continuously updating the avatar. Claim 9 A method according to claim 1, further comprising: determining the attributes of an object depicted in the stored image in response to a user selection of the stored image through the photo selector; selecting a clothing item associated with the attributes of the object from a plurality of candidate clothing items for the user's avatar; and displaying the selected clothing item together with the user's avatar. Claim 10 As a device, the device comprises: a processor; and a memory storing instructions, wherein the instructions, when executed by the processor, display a user interface in which the camera of the device is activated to capture a real-time image by a messaging application running on the device associated with a user, the user interface comprises a carousel interface including a plurality of icons, wherein the plurality of icons includes a first icon user-selectable to update the user's avatar for the messaging application based on performing a scan operation, the plurality of icons further include second icons, each of which is user-selectable to apply to respective augmented reality content in association with the real-time image -; receive user input by the messaging application selecting the first icon within the carousel interface; and in response to receiving the user input: display a scan prompt user-selectable to perform the scan operation with respect to the real-time image; A device configured to display a photo selector for user selection of a stored image to perform the above scan operation; determine the attributes of an object depicted in the real-time image in response to the user selection of the above scan prompt; select a clothing item associated with the attributes of the object from a plurality of candidate clothing items for the user's avatar; and display the selected clothing item together with the user's avatar. Claim 11 In paragraph 10, the above commands display a user-selectable outfit update button to continuously update the user's avatar based on the outfit item; receive a user selection of the outfit update button; and in response to receiving the user selection, further configure the processor to update the user's profile so that the outfit item is continuously associated with the avatar's outfit. Claim 12 A device according to claim 10, wherein the commands determine at least one second attribute of at least one second object depicted in the image in response to a user selection of the scan prompt; select at least one second clothing item associated with the at least one second attribute of the at least one second object from the plurality of candidate clothing items for the user's avatar; and further configure the processor to display the selected clothing item and the at least one second clothing item together with the user's avatar. Claim 13 In paragraph 10, the object depicted in the image above is a device similar to the clothing item. Claim 14 In paragraph 10, the object depicted in the above image is a device corresponding to a barcode associated with the clothing item. Claim 15 In paragraph 10, the device, wherein the real-time image corresponds to the live feed of the device camera. Claim 16 In paragraph 10, the device, wherein the stored image is included in a photo library associated with the user. Claim 17 As a non-transient computer-readable storage medium, said computer-readable storage medium comprises instructions, said instructions, when executed by a computer, cause said computer to display a user interface in which a camera of said device is activated to capture a real-time image by a messaging application running on a device associated with a user— said user interface comprises a carousel interface comprising a plurality of icons, said plurality of icons comprising a first icon user-selectable to update the user's avatar for said messaging application based on performing a scan operation, said plurality of icons further comprising second icons, each of said second icons user-selectable to apply to said augmented reality content in association with said real-time image—; said messaging application receives user input selecting the first icon within said carousel interface; and in response to receiving said user input: display a user-selectable scan prompt to perform said scan operation with respect to said real-time image; A computer-readable storage medium that displays a photo selector for user selection of a stored image to perform the above-mentioned scan operation; determines the attributes of an object depicted in the real-time image in response to the user selection of the above-mentioned scan prompt; selects a clothing item associated with the attributes of the object from a plurality of candidate clothing items for the user's avatar; and displays the selected clothing item together with the user's avatar. Claim 18 delete Claim 19 delete Claim 20 delete

Images