Automatically generated personalized messages

Abstract

Aspects of the disclosure relate to a system, including a computer-readable storage medium storing at least one program, and a method for performing operations comprising: identifying a current social event; obtaining a list of creative tools associated with the current social event; accessing a plurality of content items generated by a client device; determining that one or more of the creative tools have been applied to a first content item of the plurality of content items; generating a subset of content items including the first content item corresponding to the current social event; automatically populating a social event template graphic based on the subset of content items corresponding to the current social event to generate a social event graphic card; and receiving a request from the client device to send the social event graphic card to one or more other client devices.

Description

[0001] Priority Statement

[0002] This application claims priority to U.S. Patent Application Serial No. 16 / 849,095, filed April 15, 2020, which is incorporated herein by reference in its entirety. Technical Field

[0003] This disclosure generally pertains to the events, and more specifically, to the generation of content related to the events. Background Technology

[0004] Social networking platforms continue to grow globally. Users of these platforms are constantly seeking new ways to connect with their friends around the world. Attached Figure Description

[0005] In drawings that are not necessarily drawn to scale, similar reference numerals may describe similar parts in different views. For ease of identification of any particular element or action being discussed, one or more of the highest-order digits of the reference numerals indicate the drawing number in which the element was first introduced. Some embodiments are shown in the accompanying drawings by way of example rather than limitation, in which:

[0006] Figure 1 It is a diagrammatic representation of a networked environment in which the content of this disclosure can be deployed, based on some examples.

[0007] Figure 2 It is a graphical representation of a messaging system with both client-side and server-side functionality, based on some examples.

[0008] Figure 3 It is a graphical representation of the data structures maintained in the database based on some examples.

[0009] Figure 4 It is a graphical representation based on some example messages.

[0010] Figure 5A This is a block diagram illustrating an example social event card generation platform according to an example implementation.

[0011] Figure 5B This is a flowchart illustrating an example operation of a social event card generation platform according to an example implementation.

[0012] Figures 6 to 8 This is an illustrative user interface for a social event card generation platform based on an example implementation.

[0013] Figure 9It is a graphical representation of a machine in the form of a computer system based on some examples, in which a set of instructions can be executed to cause the machine to perform any or more of the methods discussed herein.

[0014] Figure 10 It is a block diagram showing a software architecture in which examples can be implemented. Detailed Implementation

[0015] The following description includes systems, methods, techniques, instruction sequences, and computer program products embodying illustrative embodiments of this disclosure. In the following description, numerous specific details are set forth for illustrative purposes to provide an understanding of various embodiments. However, it will be apparent to those skilled in the art that embodiments can be practiced without these specific details. Generally, well-known examples of instructions, protocols, structures, and techniques need not be shown in detail.

[0016] One of the challenges users face when trying to connect with each other about shared events is choosing the right content to share. For example, a given user might want to send an e-card related to the Christmas holidays to a friend. To do this, the user must manually search through a vast amount of content and images created by the user to add to the card. After selecting the images to include, the user must also spend time finding the right design and arrangement for the content within the card. This approach is time-consuming, unintuitive, and difficult, ultimately frustrating for users. Consequently, users avoid generating and sharing content related to the shared event, missing specific opportunities to connect with their friends. Some systems automatically and randomly select content for users to view and share, but such systems fail to consider content relevant to the event or location. This approach diminishes the importance of the selected content and ultimately provides a collection of content that is not unique or does not have distinct meaning among users. This results in such features being underutilized and can degrade the overall user experience, ultimately wasting system resources such as storage, processing, and display resources.

[0017] The disclosed implementation improves the efficiency of using electronic devices by automatically generating holiday cards, such as greeting cards, which are unique and meaningful to the user and can be easily shared with other users. Specifically, the disclosed implementation identifies current social events and obtains a list of creative tools associated with the current social events. Multiple content items generated by client devices are identified, and it is determined whether one or more creative tools have been applied to a first content item among the multiple content items. A subset of content items corresponding to the current social event is generated, including the first content item, and a social event template graphic is automatically populated based on the subset of content items corresponding to the current social event. A request is received from the client device to send a social event graphic card based on the populated template to one or more other client devices. In this way, meaningful and unique holiday greeting cards are automatically generated with minimal user interaction, allowing users to easily contact their friends about events such as: globally celebrated holidays, globally celebrated events, events celebrated in a specific region, religious holidays, national states of emergency, periods of quarantine in a specific region, or secular holidays.

[0018] This significantly improves how users interact, connect, and exchange messages on social media platforms. In particular, it dramatically improves the user experience, reducing the number of steps users must take to generate content related to shared events, and making interacting with friends and using social media platforms more enjoyable. This is because users can receive automatically generated holiday greeting cards without actually searching for content, manually selecting content, manually organizing content, and opening different interfaces to generate the cards.

[0019] Networked computing environment

[0020] Figure 1 This is a block diagram illustrating an example messaging system 100 for exchanging data (e.g., messages and associated content) over a network. Messaging system 100 includes multiple instances of client devices 102, each hosting several applications including message client 104. Each message client 104 is communicatively coupled to other instances of message server system 108 via network 106 (e.g., the Internet).

[0021] The messaging client 104 is able to communicate and exchange data with another messaging client 104 and the messaging server system 108 via the network 106. The data exchanged between messaging clients 104 and between messaging client 104 and messaging server system 108 includes functions (e.g., commands to activate functions) and payload data (e.g., text, audio, video, or other multimedia data).

[0022] Each messaging client 104 can also communicate with the social event greeting card generation platform 126. The social event card generation platform 126 identifies the current social event and obtains a list of creation tools associated with it. The platform identifies multiple content items generated by the client devices and determines that one or more creation tools have been applied to a first content item among these. The platform generates a subset of content items corresponding to the current social event, including the first content item, and automatically populates a social event template graphic based on this subset to generate a holiday card. The platform presents the holiday card's identifier on the messaging client 104. In response to receiving a user's selection of the identifier, the platform sends the holiday card to one or more other client devices (e.g., the client devices of a friend of the user of messaging client 104).

[0023] The message server system 108 is capable of providing server-side functionality to a specific message client 104 via network 106. While some functions of the message system 100 are described herein as being performed by either the message client 104 or the message server system 108, the location of certain functions within the message client 104 or the message server system 108 can be a design choice. For example, it is technically preferred that certain technologies and functions be initially deployed within the message server system 108, but subsequently migrated to the message client 104, which has sufficient processing power on the client device 102.

[0024] The message server system 108 supports various services and operations provided to the message client 104. Such operations include sending data to and receiving data from the message client 104, and processing data generated by the message client 104. As an example, this data may include message content, client device information, geolocation information, media enhancements and overlays, message content persistence conditions, social network information, and live event information. Data exchange within the message system 100 is activated and controlled via functions available through the user interface (UI) of the message client 104.

[0025] Specifically, turning to message server system 108, application programming interface (API) server 110 is coupled to application server 112, which includes social event card generation platform 126, and provides a programming interface to application server 112. Application server 112 is communicatively coupled to database server 118, which provides easy access to database 120, which stores data associated with messages processed by application server 112. Similarly, web server 124 is coupled to application server 112 and provides a web-based interface to application server 112. To this end, web server 124 processes incoming network requests via Hypertext Transfer Protocol (HTTP) and several other related protocols.

[0026] Application Programming Interface (API) server 110 receives and sends message data (e.g., commands and message payloads) between client device 102 and application server 112. Specifically, API server 110 provides a set of interfaces (e.g., routines and protocols) that can be invoked or queried by messaging client 104 to activate the functionality of application server 112. API server 110 exposes various functionalities supported by application server 112, including: account registration; login functionality; sending messages from one messaging client 104 to another messaging client 104 via application server 112; sending media files (e.g., images or videos) from messaging client 104 to messaging server 114 and enabling possible access by another messaging client 104; setting up media data sets (e.g., stories); retrieving a user's friend list for client device 102; retrieving such a set; retrieving messages and content; adding and deleting entities (e.g., friends) in an entity graph (e.g., a social graph); locating friends in a social graph; and opening application events (e.g., related to messaging client 104).

[0027] Application server 112 hosts several server applications and subsystems, including, for example, message server 114, image processing server 116, social event card generation platform 126, and social network server 122. Message server 114 implements several message processing technologies and functions, particularly those related to the aggregation and other processing of content (e.g., text and multimedia content) included in messages received from multiple instances of message client 104. As will be described in further detail, text and media content from multiple sources can be aggregated into content collections (e.g., referred to as stories or libraries). These collections are then made available to message client 104. Given the hardware requirements for such processing, additional processor- and memory-intensive data processing can also be performed on the server side by message server 114.

[0028] Application server 112 also includes image processing server 116, which is dedicated to performing various image processing operations, typically relative to the image or video within the payload of a message sent from or received from message server 114.

[0029] Social networking server 122 supports various social networking functions and services and makes these functions and services available to message server 114. To this end, social networking server 122 maintains and accesses entity graph 306 (such as...) within database 120. Figure 3 (As shown). Examples of functions and services supported by the social network server 122 include identifying other users of the messaging system 100 with whom a particular user has a relationship or who is “following” them, as well as identifying the interests and other entities of a particular user.

[0030] System Architecture

[0031] Figure 2 This is a block diagram illustrating further details of a messaging system 100 according to some examples. Specifically, messaging system 100 is shown as including a messaging client 104 and an application server 112. Messaging system 100 contains multiple subsystems supported on the client side by messaging client 104 and on the server side by application server 112. These subsystems include, for example, a short-run timer system 202, a collection management system 204, an enhancement system 206, a map system 208, and a game system 210. In some implementations, enhancement system 206 implements some or all of the functionality of social event card generation platform 126.

[0032] The short-lived timer system 202 is responsible for forcing temporary or time-limited access to content by the message client 104 and the message server 114. The short-lived timer system 202 includes several timers that selectively enable access (e.g., for rendering and displaying) of messages and associated content via the message client 104 based on duration and display parameters associated with a message or set of messages (e.g., a story). Further details regarding the operation of the short-lived timer system 202 are provided below.

[0033] The collection management system 204 is responsible for managing groups or collections of media (e.g., collections of text, images, video, and audio data). Collections of content (e.g., messages, including images, videos, text, and audio) can be organized into "event libraries" or "event stories." Such collections can be made available for a specified time period (e.g., the duration of the event the content relates to). For example, content related to a concert can be made available as a "story" for the duration of the concert. The collection management system 204 can also be responsible for publishing icons that notify the user interface of the messaging client 104 of the existence of a specific collection.

[0034] Furthermore, the collection management system 204 includes a curation interface 212, which enables collection managers to manage and curate specific content collections. For example, the curation interface 212 allows event organizers to curate content collections related to a specific event (e.g., removing inappropriate content or redundant messages). Additionally, the collection management system 204 employs machine vision (or image recognition technology) and content rules to automatically curate content collections. In some embodiments, users may be compensated for including user-generated content in the collection. In such cases, the collection management system 204 operates to automatically pay such users for using their content.

[0035] Enhancement system 206 provides various functionalities that enable users to enhance (e.g., annotate or otherwise modify or edit) media content associated with a message. For example, enhancement system 206 provides functionalities related to generating and publishing media overlays for messages processed by messaging system 100. Enhancement system 206 can operablely provide media overlays or enhancements (e.g., image filters) to messaging client 104 based on the geographic location of client device 102. In another example, enhancement system 206 can operablely provide media overlays to messaging client 104 based on other information such as the social network information of the user of client device 102. Media overlays can include audio and visual content as well as visual effects. Examples of audio and visual content include pictures, text, logos, animations, and sound effects. Examples of visual effects include color overlays. Audio and visual content or visual effects can be applied to media content items (e.g., photos) at client device 102. For example, media overlays can include text or images that can be overlaid on top of a photo taken by client device 102. In another example, media overlays include location identifier overlays (e.g., Venice Beach), live event name or business name overlays (e.g., beach cafe). In another example, enhancement system 206 uses the geographic location of client device 102 to identify a media overlay that includes the name of a merchant at the geographic location of client device 102. The media overlay may include other tags associated with the merchant. The media overlay may be stored in database 120 and accessed through database server 118.

[0036] In some examples, enhancement system 206 provides a user-based publishing platform that allows users to select geographic locations on a map and upload content associated with those locations. Users can also specify situations where certain media overlays should be provided to other users. Enhancement system 206 generates a media overlay that includes the uploaded content and associates it with the selected geographic location.

[0037] In other examples, Enhancement System 206 provides a merchant-based publishing platform that enables merchants to select specific media overlays associated with geographic locations via a bidding process. For example, Enhancement System 206 associates the media overlay of the highest bidder with a corresponding geographic location for a predefined amount of time.

[0038] Map system 208 provides various geolocation functions and supports the presentation of map-based media content and messages by messaging client 104. For example, map system 208 can display (e.g., stored in profile data 308) user icons or avatars on the map to indicate the current or past locations of the user's "friends," as well as media content (e.g., a collection of messages including photos and videos) generated by these friends within the map's context. For example, on the map interface of messaging client 104, messages posted by a user from a specific geographic location to messaging system 100 can be displayed to the specific user's "friends" within the context of that specific location on the map. Users can also share their location and status information with other users of messaging system 100 via messaging client 104 (e.g., using appropriate status avatars), which is displayed to selected users within the context of the map interface of messaging client 104.

[0039] The game system 210 provides various game functions within the context of the messaging client 104. The messaging client 104 provides a game interface that displays a list of available games, which can be started by a user within the context of the messaging client 104 and played with other users of the messaging system 100. The messaging system 100 also enables specific users to invite other users to play specific games by sending invitations from the messaging client 104 to such other users. The messaging client 104 also supports both voice and text communication within the game context (e.g., chat), provides leaderboards for the game, and also supports providing in-game rewards (e.g., coins and items).

[0040] Data Architecture

[0041] Figure 3 This is a schematic diagram illustrating a data structure 300 that can be stored in a database 120 of a message server system 108, according to certain examples. Although the contents of the database 120 are shown to include several tables, it should be understood that the data can be stored in other types of data structures (e.g., as an object-oriented database).

[0042] Database 120 includes message data stored in message table 302. For any given message, this message data includes at least message sender data, message receiver (or recipient) data, and a payload. See below for reference. Figure 4Further details are provided regarding information that can be included in the message and in the message data stored in message table 302.

[0043] Entity table 304 stores entity data and (for example, links to entity diagram 306 and profile data 308). Entities that maintain records within entity table 304 can include individuals, company entities, organizations, objects, locations, events, etc. Regardless of the entity type, any entity whose data is stored in message server system 108 can be an identifiable entity. Each entity is provided with a unique identifier and an entity type identifier (not shown).

[0044] Entity graph 306 stores information about the relationships and associations between entities. For example, such relationships can be social, professional (e.g., working in the same company or organization), interest-based, or activity-based.

[0045] Profile data 308 stores various types of profile data about a specific entity. Based on privacy settings specified by the specific entity, profile data 308 can be selectively used and presented to other users of messaging system 100. In the case of an individual, profile data 308 includes, for example, a username, phone number, address, settings (e.g., notification and privacy settings), and an avatar representation (or a set of such avatar representations) selected by the user. A specific user can then selectively include one or more of these avatar representations in the content of messages transmitted via messaging system 100 and in a map interface displayed to other users by messaging client 104. The set of avatar representations may include “status avatars,” which present a graphical representation of a user’s chosen state or activity at a specific time.

[0046] In the case that the entity is a group, in addition to the group name, members and various settings of the associated group (e.g., notifications), the group profile data 308 may similarly include one or more avatars associated with the group.

[0047] Database 120 also stores enhancement data, such as overlays or filters, in enhancement table 310. The enhancement data is associated with and applied to videos (video data is stored in video table 314) and images (image data is stored in image table 316).

[0048] In one example, the filter is an overlay displayed as an image or video during presentation to the receiving user. Filters can be of various types, including filters selected by the user from a set of filters presented to the sending user by the message client 104 when the sending user is composing a message. Other types of filters include geolocation filters (also known as geographic filters), which can be presented to the sending user based on geolocation. For example, a geolocation filter specific to a nearby or specific location can be presented by the message client 104 within the user interface based on geolocation information determined by the Global Positioning System (GPS) unit of the client device 102.

[0049] Another type of filter is a data filter, which can be selectively presented to the sending user by the messaging client 104 based on other inputs or information collected by the client device 102 during the message creation process. Examples of data filters include the current temperature at a specific location, the current speed of the sending user, the battery life of the client device 102, or the current time.

[0050] Other augmented data that can be stored in image table 316 includes augmented reality content items (e.g., corresponding to an applied lens or augmented reality experience). Augmented reality content items can be real-time effects and sounds that can be added to images or videos.

[0051] Event list 307 stores data representing various events of the social event card generation platform 126. Event list 307 provides start time and date data for each event, which the social event card generation platform 126 uses to automatically generate a social event template graphic (e.g., a holiday greeting card) corresponding to a given event in the events preceding or following the start of the given event. Each event in event list 307 may include a threshold time period, which controls how long after the start or end of the event the social event template graphic remains available for selection and sharing with other users. For example, event list 307 stores a first event (e.g., New Year's Day) with a corresponding start time (e.g., midnight) and date (e.g., January 1st of each given year) and a three-day threshold. In this case, any automatically generated social event template graphic corresponding to the first event can be made available on messaging client 104 for sharing with other users for a three-day period after the start or end of the first event. After the three-day threshold period, the identifier of the social event template graphic corresponding to the first event is removed to prevent the social event template graphic from being selected for sharing with other users.

[0052] As another example, event list 307 stores a second event (e.g., Christmas) with a corresponding start time (e.g., midnight) and date (e.g., December 25th of a given year). As another example, event list 307 stores a third event (e.g., Hanukkah) with a corresponding start time (e.g., sunset time of the day before the start date of the event) and start date (e.g., a specified lunar date of a given year that varies annually in the Gregorian calendar). As another example, event list 307 stores a fourth event (e.g., the start or end of a national emergency) with a corresponding start date (e.g., the day a national emergency is declared to have begun or ended). As another example, event list 307 stores a fifth event (e.g., the start of a quarantine period) with a corresponding start date (e.g., the day a quarantine period begins) and a specified area associated with the quarantine period (e.g., a city, zip code, state, or location specified during the quarantine period). The fifth event may be associated with a predetermined time interval specifying how long after the start of the quarantine period the automatic generation of a social event template graph will be initiated. For example, if the scheduled time interval is 30 days, in this case, when the current date is 30 days or more after the start of the quarantine period, the social event template graphic will automatically populate and can be used to share with other users during another scheduled time period.

[0053] Event list 307 stores any number of events with their corresponding start times and dates, including any globally celebrated holiday, globally celebrated events, events celebrated in a specific region, Easter, Good Friday, Hanukkah, religious holidays, national states of emergency, regional quarantine periods, or secular holidays. In some implementations, users can manually input events into event list 307 by specifying the event's name, visual attributes (for generating social event template graphics), and start time or start date. In some implementations, the start times of events stored in event list 307 are relative to Coordinated Universal Time (UTC) (e.g., the start time indicates the start time of the event in UTC time with zero offset).

[0054] Event list 307 stores a set of creation tools associated with each event. For example, a first set of creation tools, which can be used to modify one or more content items generated by messaging client 104, is associated with a first event stored in event list 307. Such creation tools include any image or video modification features that can be applied to enhance or modify an image or video before it is stored or shared. Such creation tools include augmented reality graphics or items, annotations, titles, filters, etc. The first set of creation tools includes features specific to the first event, such as a title, graphics, or words describing the first event. As another example, a second set of creation tools, which can be used to modify one or more content items generated by messaging client 104, is associated with a second event stored in event list 307. The second set of creation tools includes features specific to the second event, such as a title, graphics, or words describing the second event (e.g., a title with the name of the event or a symbol of the event (e.g., a Thanksgiving turkey)).

[0055] Event list 307 stores the minimum number of content items modified using the creative tools associated with each event. For example, the first event in event list 307 is associated with the minimum number of three content items, and the second event is associated with the minimum number of six content items. The minimum number of content items is used to conditionally control whether a social event template graph is automatically generated for a given client device. For example, if a given client device has previously generated a set of content items associated with the first event where only two creative tools are applied (e.g., fewer content items than the minimum number associated with the first event), then the social event template graph associated with the first event is not automatically generated. If a given client device has previously generated a set of content items associated with the second event where only 10 creative tools are applied (e.g., more content items than the minimum number associated with the second event), then the social event template graph associated with the second event is automatically generated.

[0056] The minimum number of content items associated with each event is dynamically adjusted based on factors such as event popularity, users' previous engagement patterns during the event's duration, user engagement levels using messaging client 104, and event importance. In some cases, when the first event is more popular than the second event, the minimum number associated with the first event is greater than the minimum number associated with the second event. In some cases, when the percentage of users creating content during the first event is greater than that of the second event, the minimum number associated with the first event is less than that of the second event. In some cases, when the percentage of users creating content during the first event is greater than that of the second event, the minimum number associated with the first event is less than that of the second event. In some cases, when the user engagement level using messaging client 104 exceeds a threshold, the minimum number associated with the first event is greater than the minimum number associated with the second event. In some cases, when the user engagement level using messaging client 104 exceeds a threshold, the minimum number associated with the first event is less than that of the second event.

[0057] As described above, augmented data includes augmented reality content items, overlays, image transformations, AR images, and similar items referring to modifications that can be applied to image data (e.g., video or images). This includes real-time modifications, which modify an image as it is captured using the device sensors (e.g., one or more cameras) of client device 102, and then display the modified image on the screen of client device 102. This also includes modifications to stored content (e.g., video clips in a modifiable library). For example, in client device 102 where multiple augmented reality content items are accessible, a user can use a single video clip with multiple augmented reality content items to see how different augmented reality content items will modify the stored clip. For example, by selecting different augmented reality content items for the same content, multiple augmented reality content items with different pseudo-random motion models can be applied to that same content. Similarly, real-time video capture can be used in conjunction with the modifications shown to demonstrate how the video image currently captured by the sensors of client device 102 will modify the captured data. Such data can be simply displayed on the screen without being stored in memory, or the content captured by the device sensors can be recorded and stored in memory with or without modification (or both). In some systems, the preview function can show how different augmented reality content items will be displayed simultaneously in different windows on the monitor. For example, this allows viewing multiple windows with different pseudo-random animations on the monitor at the same time.

[0058] Therefore, using augmented reality content items and various systems, or other such transformation systems that modify content using that data, can involve detecting objects (e.g., faces, hands, bodies, cats, dogs, surfaces, objects, etc.) in video frames, tracking these objects as they leave, enter, and move around within the field of view, and modifying or transforming them while tracking them. In various implementations, different methods can be used to achieve such transformations. Some examples may involve generating 3D mesh models of one or more objects, and using transformations and animated textures of the models within the video to achieve the transformations. In other examples, tracking points on the objects can be used to place images or textures (which can be two-dimensional or three-dimensional) at the tracked locations. In further examples, neural network analysis of video frames can be used to place images, models, or textures within content (e.g., images or video frames). Thus, augmented reality content items refer both to the images, models, and textures used to create transformations within the content, and to the additional modeling and analysis information required to achieve such transformations through object detection, tracking, and placement.

[0059] Real-time video processing can be performed using any type of video data (e.g., video streams, video files, etc.) stored in the memory of any type of computerized system. For example, a user can load a video file and store it in the device's memory, or a video stream can be generated using the device's sensors. Furthermore, computer-animated models can be used to process any object, such as human faces and parts of the human body, animals, or inanimate objects (e.g., chairs, cars, or other objects).

[0060] In some examples, when a specific modification is selected along with the content to be transformed, the elements to be transformed are identified by a computing device and then detected and tracked if they exist in the frames of the video. The elements of the object are modified according to the modification request, thereby transforming the frames of the video stream. For different kinds of transformations, the transformation of the video stream frames can be performed using different methods. For example, for frame transformations that primarily refer to changes in the form of object elements, feature points of each element of the object are calculated (e.g., using an Active Shape Model (ASM) or other known methods). Then, a feature point-based mesh is generated for each of at least one element of the object. This mesh is used for subsequent stages of tracking the elements of the object in the video stream. During tracking, the mentioned mesh for each element is aligned with the position of each element. Then, additional points are generated on the mesh. A first set of points is generated for each element based on the modification request, and a second set of points is generated for each element based on this first set of points and the modification request. The elements of the object can then be modified based on this first set of points and this second set of points, as well as the mesh, to transform the frames of the video stream. In this method, the background of the modified object can also be altered or distorted by tracking and modifying the background.

[0061] In some examples, transformations of certain regions of an object using its elements can be performed by calculating feature points for each element of the object and generating a mesh based on those calculated feature points. Points are generated on the mesh, and various regions are then generated based on these points. The elements of the object are then tracked by aligning the regions of each element with the positions of at least one of the elements, and the properties of the regions can be modified based on modification requests, thereby transforming frames of the video stream. Depending on the specific modification request, the properties of the mentioned regions can be transformed in different ways. Such modifications can involve: changing the color of the region; removing at least some portions of the region from the frames of the video stream; including one or more new objects in the region based on the modification request; and modifying or distorting the elements of the region or object. In various implementations, any combination of such modifications or other similar modifications can be used. For some models to be animated, some feature points can be selected as control points to determine the entire state space for options used in model animation.

[0062] In some examples of computer animation models that use face detection to transform image data, a specific face detection algorithm (e.g., Viola-Jones) is used to detect faces in the image. The Active Shape Model (ASM) algorithm is then applied to the facial regions of the image to detect facial feature reference points.

[0063] In other examples, other methods and algorithms suitable for face detection can be used. For example, in some implementations, landmarks are used to locate features that represent distinguishable points present in most of the images considered. For example, for facial landmarks, the location of the left pupil could be used. If the initial landmark is unrecognizable (e.g., if the person is wearing an eye patch), secondary landmarks can be used. Such a landmark recognition process can be used for any such object. In some examples, a set of landmarks forms a shape. The shape can be represented as a vector using the coordinates of points in the shape. One shape is aligned with another shape through a similarity transformation (allowing translation, scaling, and rotation) that minimizes the average Euclidean distance between the points of the shapes. The average shape is the average of the aligned training shapes.

[0064] In some examples, a landmark search begins with an average shape aligned with the position and size of a face determined by a global face detector. This search then repeats the following steps: proposing provisional shapes by adjusting the localization of shape points through template matching of the image texture around each point, and then conforming the provisional shapes to a global shape model until convergence occurs. In some systems, individual template matching is unreliable, and the shape model pools the results of weak template matching to form a stronger overall classifier. The entire search is repeated at each level of the image pyramid, from coarse to fine resolution.

[0065] The transformation system can capture image or video streams on a client device (e.g., client device 102) and perform complex image manipulations locally on client device 102 while maintaining an appropriate user experience, computation time, and power consumption. Complex image manipulations can include size and shape changes, emotion shifts (e.g., changing a face from frowning to smiling), state shifts (e.g., aging an object, reducing its apparent age, or changing its gender), style shifts, application of graphical elements, and any other suitable image or video manipulations implemented by a convolutional neural network that has been configured to execute efficiently on client device 102.

[0066] In some examples, a computer animation model for transforming image data can be used by a system in which a user can use a client device 102 with a neural network to capture an image or video stream of the user (e.g., a selfie), the neural network operation being part of a messaging client 104 operating on the client device 102. A transformation system operating within the messaging client 104 determines the presence of a face in the image or video stream and provides a modification icon associated with the computer animation model to transform the image data, or the computer animation model can be presented as associated with the interface described herein. The modification icon includes changes that can be the basis for modifying the user's face in the image or video stream as part of a modification operation. Once a modification icon is selected, the transformation system initiates a process of transforming the user's image to reflect the selected modification icon (e.g., generating a smiley face on the user). Once the image or video stream is captured and the specified modification is selected, the modified image or video stream can be presented in a graphical user interface displayed on the client device 102. The transformation system can implement a complex convolutional neural network on a portion of the image or video stream to generate and apply the selected modification. In other words, once an edit icon is selected, the user can capture an image or video stream and see the changes in real-time or near real-time. Furthermore, while a video stream is being captured, the changes can be persistent, and the selected edit icon continues to be toggled. Machine learning neural networks can be used to achieve this type of modification.

[0067] A graphical user interface (GUI) presenting modifications performed by the transformation system can provide users with additional interactive options. Such options can be based on an interface used to initiate content capture and selection for a specific computer animation model (e.g., initiated from a content creator user interface). In various implementations, modifications can be persisted after the initial selection of the modification icon. Users can turn modifications on or off by tapping or otherwise selecting a face modified by the transformation system and save it for later viewing or browsing to other areas of the imaging application. In cases where the transformation system modifies multiple faces, users can globally turn modifications on or off by tapping or selecting a single face modified and displayed within the GUI. In some implementations, individual faces within a set of multiple faces can be modified separately, or such modifications can be toggled individually by tapping or selecting a single face or a series of individual faces displayed within the GUI.

[0068] Story table 312 stores data related to messages and collections of associated image, video, or audio data, which are compiled into collections (e.g., stories or libraries). The creation of a specific collection can be initiated by a specific user (e.g., each user who maintains their records in entity table 304). A user can create a "personal story" in the form of a collection of content that has been created and sent / broadcast by that user. To this end, the user interface of messaging client 104 may include user-selectable icons that allow the sending user to add specific content to his or her personal story.

[0069] Collections can also constitute "live stories," which are collections of content from multiple users, created manually, automatically, or using a combination of manual and automatic technologies. For example, a "live story" can constitute a curated flow of user-submitted content from various locations and events. Users whose client devices have location services enabled and are at a co-located event at a specific time can be presented with options, for example, via the user interface of messaging client 104, to contribute content to a specific live story. The live story can be identified to the user by messaging client 104 based on their location. The end result is a "live story" told from a community perspective.

[0070] Another type of content collection is called a "location story," which allows users whose client devices 102 are located in a specific geographic location (e.g., on a college or university campus) to contribute to a specific collection. In some examples, contributing to a location story may require secondary authentication to verify that the end user belongs to a specific organization or other entity (e.g., a student on a university campus).

[0071] As mentioned above, video table 314 stores video data, which in one example is associated with a message whose record is stored in message table 302. Similarly, image table 316 stores image data associated with messages whose message data is stored in entity table 304. Entity table 304 can associate various enhancements from enhancement table 310 with various images and videos stored in image table 316 and video table 314.

[0072] Data communication architecture

[0073] Figure 4 This is a schematic diagram illustrating the structure of message 400 according to some examples, generated by message client 104 for transmission to another message client 104 or message server 114. The content of a particular message 400 is used to populate message table 302 stored in database 120, which is accessible by message server 114. Similarly, the content of message 400 is stored in memory as "in transit" or "in flight" data for client device 102 or application server 112. Message 400 is shown to include the following example components.

[0074] • Message Identifier 402: A unique identifier that identifies message 400.

[0075] • Message text payload 404: The text to be generated by the user via the user interface of the client device 102 and included in message 400.

[0076] • Message image payload 406: Image data captured by the camera component of the client device 102 or retrieved from the memory of the client device 102 and included in the message 400. The image data of the message 400 used for sending or receiving may be stored in the image table 316.

[0077] • Message video payload 408: Video data captured by the camera device component or retrieved from the memory component of the client device 102 and included in the message 400. The video data used to send or receive the message 400 can be stored in the video table 314.

[0078] • Message audio payload 410: Audio data captured by the microphone or retrieved from the memory component of the client device 102 and included in message 400.

[0079] • Message enhancement data 412: Enhancement data (e.g., filters, stickers, or other enhancements) representing enhancements to be applied to the message image payload 406, message video payload 408, or message audio payload 410 of message 400. Enhancement data for the message 400 to be sent or received can be stored in enhancement table 310.

[0080] • Message duration parameter 414: A parameter value that indicates the amount of time, in seconds, during which the content of the message (e.g., message image payload 406, message video payload 408, message audio payload 410) will be presented to the user or made accessible to the user via the message client 104.

[0081] • Message geolocation parameter 416: Geolocation data (e.g., latitude and longitude coordinates) associated with the content payload of the message. The payload may include multiple message geolocation parameter 416 values, each of which is associated with a content item included in the content (e.g., a specific image within the message image payload 406, or a specific video within the message video payload 408).

[0082] • Message Story Identifier 418: An identifier value that identifies one or more content sets (e.g., "Stories" identified in Story Table 312) by associating a specific content item in the message image payload 406 of message 400 with said one or more content sets. For example, the identifier value can be used to associate multiple images within the message image payload 406 with multiple content sets, respectively.

[0083] • Message Tag 420: Each message 400 can be labeled with multiple tags, each of which indicates the subject of the content included in the message payload. For example, in the case where a specific image in the message image payload 406 depicts an animal (e.g., a lion), a tag value indicating the relevant animal can be included in the message tag 420. The tag values ​​can be generated manually based on user input, or automatically using, for example, image recognition.

[0084] • Message sender identifier 422: An identifier (e.g., a messaging system identifier, email address, or device identifier) ​​indicating the user of the client device 102 on which message 400 is generated and from which message 400 is sent.

[0085] • Message recipient identifier 424: An identifier (e.g., message system identifier, email address, or device identifier) ​​indicating the user of the client device 102 to which message 400 is addressed.

[0086] The content (e.g., values) of various components of message 400 can be pointers to locations in tables storing content data values. For example, image values ​​in message image payload 406 can be pointers to locations (or addresses) within image table 316. Similarly, values ​​in message video payload 408 can point to data stored in video table 314, values ​​stored in message enhancement data 412 can point to data stored in annotation table 310, values ​​stored in message story identifier 418 can point to data stored in story table 312, and values ​​stored in message sender identifier 422 and message receiver identifier 424 can point to user records stored in entity table 304.

[0087] Figure 5A This is a block diagram illustrating an example social event card generation platform 126 according to an exemplary implementation. The social event card generation platform 126 includes a standard module 211, an event recognition module 220, a content item selection module 250, a social event template graphic generation module 240, and a social event card presentation module 230.

[0088] The event identification module 220 compares the start time or date of events stored in the event list 307 with the current time and current geographic location. When the event identification module 220 determines that the current time or date in a given area matches the start time or date of a given event in the list 307, the event identification module 220 initiates the process of generating a social event template graph by transmitting the identifier of the given event to the standards module 211, the content item selection module 250, and the social event template graph generation module 240. The event identification module 220 can perform this comparison for each geographic area and each event in the list 307 at specified time intervals (e.g., every 5 minutes) or in real time (e.g., continuously).

[0089] In some implementations, the event recognition module 220 retrieves a first event from the event list 307. The event recognition module 220 determines the geographic region associated with the first event. The event recognition module 220 compares the current geographic region of the client device 102 with the geographic region associated with the first event. When the event recognition module 220 determines that the current geographic region of the client device 102 is within a predetermined range of the geographic region associated with the first event, the event recognition module 220 determines whether the current date or time matches or is within a threshold time interval of the start date or time associated with the first event. For example, the event recognition module 220 determines whether the current date is the same as the start date of the first event, or whether the current date is within 3 days after the start date of the first event. In such cases, the event recognition module 220 provides an identifier for the first event to the standards module 211, the content item selection module 250, and the social event template graphics generation module 240.

[0090] As another example, event identification module 220 retrieves a second event from event list 307. The second event may correspond to an isolation period published in a specific geographic region. Event identification module 220 determines that the current location of the client device matches or is within a threshold distance of the geographic location associated with the second event. In response, event identification module 220 determines the start date of the second event. Event identification module 220 compares the current date with the start date and determines that the current date is more than a predetermined time interval (e.g., 30 days) older than the start date of the second event. In response, event identification module 220 provides the identifier of the second event to standard module 211, content item selection module 250, and social event template graphics generation module 240.

[0091] In some implementations, the event recognition module 220 compares the current date with the start date and determines that the current date is less than a predetermined time interval (e.g., 3 days) earlier than the start date of the second event. In response, the event recognition module 220 provides the social event card presentation module 230 with an identifier for the second event and an indication that the second event is approaching within 3 days. The social event card presentation module 230 presents the identifier in the user interface of the messaging client 104, informing the user about the automatic generation of a holiday card for the upcoming second event. The user can select this identifier to receive information about the automatic generation of the event. For example, as... Figure 7As shown, a user interface 701 of messaging client 104 is presented. This user interface 701 includes an identifier 710 for an upcoming holiday or event (e.g., Thanksgiving) that is less than three days away. In response to receiving a user selection of identifier 710, a user interface 702 including a prompt is presented. The prompt includes a message informing the user that any content generated by the user during the upcoming holiday can be used to automatically generate holiday cards (e.g., included in a specific holiday recap).

[0092] Return to reference Figure 5A The criteria module 211 obtains one or more creative tools associated with the event identified by the event recognition module 220. For example, the criteria module 211 accesses the event list 307 to retrieve one or more criteria for selecting content items related to the first event. As an example, the criteria module 211 retrieves the date of the first event, the time range of the current social event, the location associated with the first event, the country associated with the first event, or the language associated with the first event. As another example, the criteria module 211 retrieves at least one of the following: a title including one or more words associated with the first event, a graphic element representing the first event, or an augmented reality element associated with the first event. The criteria obtained by the criteria module 211 are provided to the content item selection module 250 to select content items for generating social event template graphics.

[0093] Content item selection module 250 accesses a content collection previously generated and stored by client device 102. Content item selection module 250 obtains metadata associated with the content collection and uses criteria provided by standards module 211 to filter the content collection into a filtered set of content items. For example, content item selection module 250 removes content items from the content collection associated with location data that does not match the location data associated with the first event. As another example, content item selection module 250 removes content items from the content collection associated with a country (e.g., content items captured in a specific country) that does not match the country associated with the first event. As another example, content item selection module 250 removes content items from the content collection associated with a language (e.g., including spoken or written forms of the first language) that does not match the second language associated with the first event. As another example, content item selection module 250 removes content items from the content collection associated with a date or timestamp that exceeds a threshold time period (e.g., more than 3 days) before or after the start or end of the first event. Specifically, the content item selection module 250 ensures that only content items generated during the first event (or the identified event) are included in a filtered set of content items.

[0094] Content item selection module 250 analyzes the filtered set of content items to identify those enhanced or modified using one or more creative tools and generates a subset of the filtered content items. For example, content item selection module 250 accesses a first content item in the filtered set and determines, based on the first content item's metadata, whether one or more creative tools are applied to it. Specifically, content item selection module 250 determines whether the first content item is enhanced, modified, altered, or strengthened using a title, graphic element, or augmented reality element including one or more words after or during its capture. Content item selection module 250 retrieves identifiers describing the one or more creative tools applied to the first content item. Content item selection module 250 obtains a list of creative tools associated with the first event and determines whether the retrieved identifier matches one or more creative tools in the list associated with the first event. In this example, each creative tool is associated with a unique identifier stored in the metadata of the captured and modified content item. The unique identifier is compared with unique identifiers in a list of creation tools to determine whether the creation tool used to modify the content item matches one or more creation tools associated with the first event. For example, if a user captures an image during the Thanksgiving holiday and enhances the image with the word "Thanksgiving" or a picture of a turkey, the content item selection module 250 determines that the image was enhanced or modified using one or more creation tools associated with the first event (e.g., the Thanksgiving holiday).

[0095] Content item selection module 250 generates a count value indicating how many of a filtered set of content items have been enhanced or modified using one or more creation tools associated with the first event. For example, if the user generates five content items (e.g., images and videos) enhanced with creation tools specific to the first event, the count value could be five. Content item selection module 250 provides a subset of content items and the count value to social event template graphics generation module 240. In some implementations, content item selection module 250 receives a minimum number of content items associated with the first event from standard module 211. Content item selection module 250 compares the count value with the received minimum number. In response to determining that the count value is less than the minimum number, content item selection module 250 terminates the process of generating the holiday card. In response to determining that the count value is greater than or equal to the minimum number, content item selection module 250 instructs social event template graphics generation module 240 to automatically populate the template graphics based on the subset of content items.

[0096] The social event template graphics generation module 240 stores multiple social event template graphics associated with each event in the event list 307. For example, the social event template graphics generation module 240 stores a first plurality of social event template graphics associated with a first event and a second plurality of social event template graphics associated with a second event. Figure 6 An illustrative social event template graphic 600 stored by the social event template graphic generation module 240 is shown. A first social event template graphic 610 includes a first set of slots 612 for inserting content items received from the content item selection module 250. The slots can be arranged in different ways and can have different sizes. Some slots can be arranged in a portrait image, and some slots can be arranged in a landscape image. The first social event template graphic 610 includes one or more graphic elements that thematically represent an event associated with the first social event template graphic 610. A second social event template graphic 620 includes a second set of slots 622 for inserting content items received from the content item selection module 250. The second set of slots 622 can be larger in number than the first set of slots 612, allowing more content items to be filled into the second social event template graphic 610 compared to the first social event template graphic 610. The slots can be arranged in different ways and can have different sizes. Some slots can be arranged in a portrait image, and some slots can be arranged in a landscape image. The second social event template graphic 620 includes one or more graphic elements that thematically represent an event associated with the second social event template graphic 620. The graphic elements of social event template graphics 610 and 620 may be the same or different between the first social event template graphic 610 and the second social event template graphic 620.

[0097] Return to reference Figure 5AThe social event template graphic generation module 240 selects one of the social event template graphics associated with the first event based on a count value provided by the content item selection module 250. For example, the social event template graphic generation module 240 compares the number of slots in each social event template graphic with the count value. The social event template graphic generation module 240 selects a social event template graphic that includes a number of slots equal to or fewer than the count value. Specifically, the social event template graphic generation module 240 determines that the second social event template graphic 620 includes more slots than the count value. In this case, the social event template graphic generation module 240 determines that the first social event template graphic 610 includes a number of slots less than or equal to the count value. For example, if the count value is 5 and there are 4 slots in the first social event template graphic 610, the social event template graphic generation module 240 selects the first social event template graphic 610. In some cases, the social event template graphic generation module 240 selects a social event template graphic that includes a number of slots greater than the count value but less than a threshold amount (e.g., less than two slots). Specifically, the social event template graphic generation module 240 selects a social event template graphic that includes five slots, with the five slots being greater than a count value of three (within two slots).

[0098] The social event template graphic generation module 240 randomly or pseudo-randomly selects content items from a subset of content items sufficient to fill the slots in the selected social event template graphic. The module randomly or pseudo-randomly places the selected content items into the corresponding slots and formats, scales, and resizes them to fit the slots. The module randomly or pseudo-randomly selects annotations to apply to one or more content items and adds the selected annotations as overlays on top of the filled template graphic. The module randomly or pseudo-randomly selects one or more theme-related graphic elements (e.g., theme-related colors or images) to place on top of the social event template graphic. For example, the module adds turkeys and fall colors to a social event template graphic associated with Thanksgiving. The module outputs a social event card (e.g., a holiday card) to the social event card rendering module 230, which includes the filled and annotated social event template graphics.

[0099] The social event card presentation module 230 presents the identifier of the holiday card (social event card) generated by the social event template graphics generation module 240 on the graphical user interface of the messaging client 104 of the client device 102. The social event card presentation module 230 retrieves a threshold time period from the event list 307 associated with the first event. The social event card presentation module 230 compares the current time or date with the start time or date of the first event. The social event card presentation module 230 determines that the current time or date exceeds the start time or date of the first event by a first amount. The social event card presentation module 230 compares this first amount with the threshold time period. In response to determining that the first amount is less than the threshold time period, the social event card presentation module 230 maintains the identifier in the display of the messaging client 104. In response to determining that the first amount is greater than the threshold time period, the social event card presentation module 230 removes the identifier from the display of the messaging client 104 to prevent the user from sharing or accessing the social event card.

[0100] The social event card presentation module 230 receives the user's selection of the identifier to be displayed from the messaging client 104. In response, the social event card presentation module 230 presents a holiday card (e.g., a social event card) to the user. The social event card presentation module 230 provides options for specifying a list of recipients and for modifying the social event card. For example, the user can rearrange, remove, or add graphic elements and content items to the social event card. The user can also add new annotations to the social event card. The user can choose the option to specify a list of recipients or the option to send it to all of the user's friends, and in response, the social event card presentation module 230 sends the holiday card to the specified list of recipients for display on their respective client devices 102.

[0101] Figure 5B This is a flowchart illustrating example operations of the social event card generation platform 126 in execution process 500 according to an exemplary embodiment. Process 500 can be implemented with computer-readable instructions executable by one or more processors, such that the operations of process 500 can be performed partially or entirely by functional components of message server system 108; therefore, process 500 is described below by way of example with reference to it. However, in other embodiments, at least some operations of process 500 can be deployed on various other hardware configurations. Therefore, process 500 is not intended to be limited to message server system 108 and can be implemented wholly or partially by any other component. Operations in process 500 can be performed in any order, in parallel, or can be completely skipped and omitted.

[0102] At operation 501, the social event card generation platform 126 identifies the current social event. For example, when the event identification module 220 determines that the current geographic area of ​​the client device 102 is within a predetermined range of the geographic area associated with the first event, and the current date or time matches or is within a threshold time interval of the start date or time associated with the first event, the event identification module 220 identifies the first event as a current social event (e.g., an event that is currently happening or has recently happened).

[0103] At operation 502, the social event card generation platform 126 obtains a list of creative tools associated with the current social event. For example, standard module 211 retrieves at least one of the following: a title containing one or more words associated with the first event, a graphical element representing the first event, or an augmented reality element associated with the first event.

[0104] At operation 503, the social event card generation platform 126 accesses multiple content items generated by the client device. For example, the content item selection module 250 accesses a set of content previously generated and stored by the client device 102.

[0105] At operation 504, the social event card generation platform 126 determines that one or more creation tools in the list have been applied to a first content item among multiple content items. For example, the content item selection module 250 analyzes the set of content items to identify content items that have been enhanced or modified using one or more creation tools and generates a subset of content items. Specifically, the content item selection module 250 determines whether a title, graphic element, or augmented reality element including one or more words was used to enhance, modify, change, or strengthen the first content item after or during its capture.

[0106] At operation 505, the social event card generation platform 126 generates a subset of content items corresponding to the current social event, including the first content item. For example, the content item selection module 250 includes content items that have been enhanced or modified using one or more creation tools in the subset of content items corresponding to the first event.

[0107] At operation 506, the social event card generation platform 126 automatically fills in the social event template graphic based on the subset of content items corresponding to the current social event to generate a holiday card. For example, the social event template graphic generation module 240 selects a social event template graphic from multiple social event template graphics associated with the first event, and randomly or pseudo-randomly selects content items from the subset of content items sufficient to fill the slots in the selected social event template graphic.

[0108] At operation 507, the social event card generation platform 126 receives a request from a client device to send a holiday card to one or more other client devices. For example, the social event card presentation module 230 presents the identifiers of the holiday cards generated by the social event template graphics generation module 240 on the graphical user interface of the messaging client 104 of client device 102, and provides the user with the option to specify a list of recipients to whom the holiday cards are sent.

[0109] Figure 8 This is an illustrative user interface for a holiday card generation platform based on an example implementation. Specifically, Figure 8 The user interface 800 includes an identifier 810 for a holiday card associated with a first event (e.g., Thanksgiving). The identifier 810 remains displayed in the user interface 800 for a specified threshold time period following the start time or date of the first event. For example, the identifier 810 continues to be displayed in the user interface 800 for three days after the start of the first event. In some cases, the identifier 810 is for a second event and remains displayed for a different amount of time (e.g., seven days after the start of the second event). The amount of time the identifier 810 is displayed is based on the type of event or the popularity or importance of the event.

[0110] In response to receiving a user's selection of identifier 810, the social event card generation platform 126 presents an automatically generated holiday card 801. The holiday card 801 includes content items 820 captured, stored, and enhanced using creation tools associated with the first event. The holiday card 801 also includes one or more graphic elements 822 related to the theme of the first event. An option (not shown) is presented allowing the user to share the holiday card with one or more other designated recipients. Another option (not shown) is presented to allow the user to modify items or add items to the holiday card 801.

[0111] Machine architecture

[0112] Figure 9This is a schematic representation of machine 900, in which instructions 908 (e.g., software, programs, applications, applets, or other executable code) can be executed to cause machine 900 to perform any or more of the methods discussed herein. For example, instructions 908 can cause machine 900 to perform any or more of the methods described herein. Instructions 908 transform the general, non-programmed machine 900 into a specific machine 900 programmed to perform the described and illustrated functions in the described manner. Machine 900 can operate as a standalone device or can be coupled (e.g., networked) to other machines. In a networked deployment, machine 900 can operate as a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. Machine 900 may include, but is not limited to: server computers, client computers, personal computers (PCs), tablet computers, laptop computers, netbooks, set-top boxes (STBs), personal digital assistants (PDAs), entertainment media systems, cellular phones, smartphones, mobile devices, wearable devices (e.g., smartwatches), smart home devices (e.g., smart appliances), other smart devices, web devices, network routers, network switches, network bridges, or any machine capable of sequentially or otherwise executing instructions 908 specifying actions to be taken by machine 900. Furthermore, although only a single machine 900 is shown, the term "machine" should also be considered to include a collection of machines that individually or jointly execute instructions 900 to perform any one or more methods discussed herein. For example, machine 900 may include client device 102 or any of several server devices forming part of message server system 108. In some examples, machine 900 may also include both client and server systems, wherein certain operations of a particular method or algorithm are performed on the server side and certain operations of a particular method or algorithm are performed on the client side.

[0113] Machine 900 may include processor 902, memory 904, and input / output (I / O) components 938, which may be configured to communicate with each other via bus 940. In the example, processor 902 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, processor 906 and processor 910 capable of executing instructions 908. The term "processor" is intended to include multi-core processors, which may include two or more independent processors (sometimes referred to as "cores") capable of executing instructions simultaneously. Although Figure 9Multiprocessor 902 is shown, but machine 900 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

[0114] Memory 904 includes main memory 912, static memory 914, and memory cells 916, all of which are accessible by processor 902 via bus 940. Main memory 904, static memory 914, and memory cells 916 store instructions 908 embodying any one or more of the methods or functions described herein. Instructions 908 may also reside wholly or partially in main memory 912, in static memory 914, in a machine-readable medium within memory cells 916, in at least one processor 902 (e.g., in the processor's cache memory), or in any suitable combination thereof during execution by machine 900.

[0115] I / O component 938 may include various components for receiving input, providing output, generating output, sending information, exchanging information, capturing measurement results, etc. The specific I / O component 938 included in a particular machine will depend on the type of machine. For example, a portable machine such as a mobile phone may include a touch input device or other such input mechanism, while a headless server machine is unlikely to include such a touch input device. It will be understood that I / O component 938 may include... Figure 9 Many other components are not shown. In various examples, I / O component 938 may include user output component 924 and user input component 926. User output component 924 may include visual components (e.g., displays such as plasma display panels (PDPs), light-emitting diode (LED) displays, liquid crystal displays (LCDs), projectors, or cathode ray tube (CRT) displays), auditory components (e.g., speakers), tactile components (e.g., vibration motors, resistance mechanisms), other signal generators, etc. User input component 926 may include alphanumeric input components (e.g., keyboards, touchscreens configured to receive alphanumeric input, photoelectric keyboards, or other alphanumeric input components), point-based input components (e.g., mice, touchpads, trackballs, joysticks, motion sensors, or other pointing instruments), tactile input components (e.g., physical buttons, touchscreens that provide the position and force of a touch or touch gesture, or other tactile input components), audio input components (e.g., microphones), etc.

[0116] In another example, I / O component 938 may include biometric component 928, motion component 930, environmental component 932, or position component 934, as well as a wide range of other components. For example, biometric component 928 includes components for detecting expressions (e.g., hand expressions, facial expressions, vocal expressions, body posture, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweating, or brain waves), and recognizing a person (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or EEG-based recognition). Motion component 930 may include: accelerometer components (e.g., accelerometers), gravity sensor components, and rotation sensor components (e.g., gyroscopes).

[0117] The environmental component 932 may include, for example, one or more camera devices (with still image / photograph and video capabilities), lighting sensor components (e.g., photometers), temperature sensor components (e.g., one or more thermometers that detect ambient temperature), humidity sensor components, pressure sensor components (e.g., barometers), acoustic sensor components (e.g., one or more microphones that detect background noise), proximity sensor components (e.g., infrared sensors that detect nearby objects), gas sensors (e.g., gas detection sensors that detect the concentration of hazardous gases for safety purposes or measure pollutants in the atmosphere), or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment.

[0118] Regarding the camera device, client device 102 may have a camera device system including, for example, a front-facing camera on the front surface of client device 102 and a rear-facing camera on the rear surface of client device 102. The front-facing camera may be used, for example, to capture still images and videos (e.g., "selfies") of the user of client device 102, which can then be enhanced using the aforementioned enhancement data (e.g., filters). For example, the rear-facing camera may be used to capture still images and videos in a more conventional camera device mode, which are similarly enhanced using the enhancement data. In addition to the front and rear cameras, client device 102 may also include a 360° camera for capturing 360° photos and videos.

[0119] Furthermore, the camera system of client device 102 may include dual rear cameras (e.g., a main camera and a depth-sensing camera), or even include triple, quadruple, or quintuple rear camera configurations on the front and rear sides of client device 102. For example, these multi-camera systems may include wide-angle cameras, ultra-wide-angle cameras, telephoto cameras, macro cameras, and depth sensors.

[0120] The position component 934 may include a positioning sensor component (e.g., a GPS receiver component), an altitude sensor component (e.g., an altimeter or barometer that detects air pressure and from which altitude can be obtained), an orientation sensor component (e.g., a magnetometer), etc.

[0121] A wide variety of technologies can be used to implement communication. I / O component 938 also includes communication component 936, which is operable to couple machine 900 to network 920 or device 922 via appropriate coupling or connection. For example, communication component 936 may include a network interface component or another suitable device to interface with network 920. In other examples, communication component 936 may include wired communication component, wireless communication component, cellular communication component, near field communication (NFC) component, etc. Components (e.g.) (low power consumption) Components and other communication components that provide communication via other modes. Device 922 can be any of other machines or various peripheral devices (e.g., a peripheral device coupled via USB).

[0122] Furthermore, the communication component 936 can detect identifiers or include components operable to detect identifiers. For example, the communication component 936 may include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting one-dimensional barcodes such as Universal Product Code (UPC) barcodes, multi-dimensional barcodes such as Quick Response (QR) codes, Aztec codes, data matrices, data symbols, MaxiCode, PDF417, Ultra Code, UCC RSS-2D barcodes, and other optical codes) or an acoustic detection component (e.g., a microphone for identifying audio signals from tags). Additionally, various information can be obtained via the communication component 936, such as location via Internet Protocol (IP) geolocation, etc. Location methods include signal triangulation and NFC beacon signals that can indicate a specific location.

[0123] Various memories (e.g., main memory 912, static memory 914, and the memory of processor 902) and storage units 916 may store one or more sets of instructions and data structures (e.g., software) implemented or used by any one or more of the methods or functions described herein. When executed by processor 902, these instructions (e.g., instruction 908) enable various operations to implement the disclosed examples.

[0124] Instructions 908 can be sent or received over network 920 via a network interface device (e.g., a network interface component included in communication component 936), using a transmission medium and any of a plurality of known transmission protocols (e.g., Hypertext Transfer Protocol (HTTP)). Similarly, instructions 908 can be sent or received via a transmission medium through a coupling to device 922 (e.g., peer-to-peer coupling).

[0125] Software Architecture

[0126] Figure 10 This is a block diagram 1000 illustrating a software architecture 1004 that can be installed on any one or more of the devices described herein. The software architecture 1004 is supported by hardware such as a machine 1002 including a processor 1020, memory 1026, and I / O components 1038. In this example, the software architecture 1004 can be conceptualized as a stack of layers, where each layer provides specific functionality. The software architecture 1004 includes layers such as an operating system 1012, libraries 1010, a framework 1008, and an application 1006. Operationally, the application 1006 activates an API call 1050 via the software stack and receives a message 1052 in response to the API call 1050.

[0127] Operating system 1012 manages hardware resources and provides public services. Operating system 1012 includes, for example, kernel 1014, services 1016, and drivers 1022. Kernel 1014 acts as an abstraction layer between the hardware layer and other software layers. For example, kernel 1014 provides functions such as memory management, processor management (e.g., scheduling), component management, networking, and security settings. Services 1016 can provide other public services to other software layers. Driver 1022 is responsible for controlling or interfacing with the underlying hardware. For example, driver 1022 may include a display driver, a camera driver, etc. or Low-power drivers, flash drives, serial communication drivers (e.g., USB drives), Drivers, audio drivers, power management drivers, etc.

[0128] Library 1010 provides common low-level infrastructure used by application 1006. Library 1010 may include system library 1018 (e.g., the C standard library), which provides functions such as memory allocation, string manipulation, and mathematical functions. Additionally, library 1010 may include API library 1024, such as media libraries (e.g., libraries for supporting the rendering and manipulation of various media formats, such as Moving Picture Experts Group 4 (MPEG4), Advanced Video Coding (H.264 or AVC), Moving Picture Experts Group Layer 3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio codec, Joint Picture Experts Group (JPEG or JPG), or Portable Web Graphics (PNG)), graphics libraries (e.g., the OpenGL framework for rendering graphic content on a display in two-dimensional (2D) and three-dimensional (3D) formats), database libraries (e.g., SQLite, which provides various relational database functions), web libraries (e.g., WebKit, which provides web browsing functionality), etc. Library 1010 may also include various other libraries 1028 to provide many other APIs to application 1006.

[0129] Framework 1008 provides common high-level infrastructure for use by application 1006. For example, framework 1008 provides various graphical user interface (GUI) functions, advanced resource management, and advanced location services. Framework 1008 can provide a wide range of other APIs that can be used by application 1006, some of which may be specific to a particular operating system or platform.

[0130] In the example, application 1006 may include home application 1036, contact application 1030, browser application 1032, book reader application 1034, location application 1042, media application 1044, messaging application 1046, game application 1048, and a variety of other applications such as third-party application 1040. Application 1006 is a program that performs the functions defined in the program. One or more applications 1006 can be created using various programming languages, such as object-oriented programming languages ​​(e.g., Objective-C, Java, or C++) or procedural programming languages ​​(e.g., C or assembly language). In a particular example, third-party application 1040 (e.g., used by an entity other than the vendor of a particular platform using Android) TM or iOS TM Applications developed using a Software Development Kit (SDK) can be used on platforms such as iOS. TM ANDROID TM , Mobile software running on the phone's mobile operating system or another mobile operating system. In this example, a third-party application 1040 can activate API calls 1050 provided by the operating system 1012 to facilitate the functions described herein.

[0131] Glossary

[0132] "Carrier signal" refers to any intangible medium capable of storing, encoding, or carrying instructions to be executed by a machine, and includes digital or analog communication signals or other intangible media to facilitate the communication of such instructions. Instructions can be sent or received over a network using a transmission medium via a network interface device.

[0133] "Client device" refers to any machine that interfaces with a communication network to obtain resources from one or more server systems or other client devices. Client devices can be, but are not limited to, mobile phones, desktop computers, laptop computers, portable digital assistants (PDAs), smartphones, tablet computers, ultrabooks, netbooks, laptop computers, multiprocessor systems, microprocessor-based or programmable consumer electronics, game consoles, set-top boxes, or any other communication device that a user can use to access the network.

[0134] "Communication network" refers to one or more parts of a network, which can be an ad hoc network, intranet, extranet, virtual private network (VPN), local area network (LAN), wireless LAN (WLAN), wide area network (WAN), wireless WAN (WWAN), metropolitan area network (MAN), the Internet, a part of the Internet, a part of the Public Switched Telephone Network (PSTN), a POTS (Plain Old-Style Telephone Service) network, a cellular telephone network, a wireless network, etc. A network, other types of networks, or a combination of two or more such networks. For example, a network or part of a network may include a wireless network or a cellular network, and the coupling may be a Code Division Multiple Access (CDMA) connection, a Global System for Mobile Communications (GSM) connection, or other types of cellular or wireless coupling. In this example, the coupling can implement any of a variety of data transmission technologies, such as Single Carrier Radio Transmission (1xRTT), Evolved Data Optimization (EVDO), General Packet Radio Service (GPRS), Enhanced Data Rate Evolution of GSM (EDGE), 3rd Generation Partnership Project (3GPP) (including 3G, 4G, Universal Mobile Telecommunications System (UMTS), High Speed ​​Packet Access (HSPA), Global Microwave Access Interoperability (WiMAX), Long Term Evolution (LTE) standards), other standards defined by various standards-setting organizations, other long-distance protocols, or other data transmission technologies.

[0135] A "component" is a device, physical entity, or logic having boundaries defined by functional or subroutine calls, branch points, APIs, or other technologies that partition or modularize a particular processing or control function. A component can be combined with other components via its interface to perform machine processing. A component can be an encapsulated functional hardware unit designed for use with other components and can be part of a program that typically performs a specific function within a related function.

[0136] Components can constitute software components (e.g., code implemented on a machine-readable medium) or hardware components. A “hardware component” is a tangible unit capable of performing certain operations and can be configured or arranged in some physical manner. In various example implementations, one or more computer systems (e.g., standalone computer systems, client computer systems, or server computer systems) or one or more hardware components (e.g., processors or processor groups) of a computer system can be configured by software (e.g., an application or application portion) to perform certain operations described herein.

[0137] Hardware components can also be implemented mechanically, electronically, or in any suitable combination thereof. For example, a hardware component may include dedicated circuitry or logic permanently configured to perform certain operations. A hardware component may be a dedicated processor, such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). A hardware component may also include programmable logic or circuitry temporarily configured by software to perform certain operations. For example, a hardware component may include software executed by a general-purpose processor or other programmable processor. Once configured by such software, the hardware component becomes a specific machine (or a specific part of a machine) uniquely tailored to perform the configured function, and no longer a general-purpose processor. It will be understood that the decision to implement a hardware component mechanically in dedicated and permanently configured circuitry or in temporarily configured (e.g., software-configured) circuitry may be driven by cost and time considerations. Accordingly, the phrase “hardware component” (or “hardware-implemented component”) should be understood to include tangible entities, i.e., entities physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate or perform certain operations described herein.

[0138] Consider implementations where hardware components are temporarily configured (e.g., programmed), eliminating the need to configure or instantiate each hardware component at any given time. For example, where the hardware components include a general-purpose processor configured by software as a dedicated processor, this general-purpose processor can be configured as different dedicated processors (e.g., including different hardware components) at different times. The software accordingly configures one or more specific processors to constitute a specific hardware component, for example, at one time and as different hardware components at different times.

[0139] Hardware components can provide information to and receive information from other hardware components. Therefore, the described hardware components can be considered communicatively coupled. In the presence of multiple hardware components, communication can be achieved through signal transmission (e.g., via appropriate circuitry and buses) between or among two or more hardware components. In embodiments where multiple hardware components are configured or instantiated at different times, such communication between hardware components can be achieved, for example, by storing information in a memory structure accessed by the multiple hardware components and retrieving information from that memory structure. For example, a hardware component can perform an operation and store the output of that operation in a memory device to which it is communicatively coupled. Other hardware components can then access the memory device at a subsequent time to retrieve and process the stored output. Hardware components can also initiate communication with input or output devices and can operate on resources (e.g., information collection).

[0140] Various operations of the example methods described herein can be performed, at least partially, by one or more processors configured, either temporarily (e.g., by software) or permanently, to perform the relevant operations. Whether temporarily or permanently configured, such processors can constitute components of a processor implementation that operate to perform one or more operations or functions described herein. As used herein, "processor-implemented component" refers to a hardware component implemented using one or more processors. Similarly, the methods described herein can be implemented, at least partially, by processors, where a particular one or more processors are examples of hardware. For example, at least some operations of the methods can be performed by one or more processors 902 or processor-implemented components. Furthermore, one or more processors can also operate to support the execution of relevant operations in a "cloud computing" environment or as "Software as a Service" (SaaS). For example, at least some operations can be performed by a group of computers (as an example of machines including processors), where these operations are accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., APIs). The execution of certain operations can be distributed among processors, not residing solely within a single machine, but deployed across multiple machines. In some example implementations, the processor or processor-implemented components may be located in a single geographic location (e.g., within a home environment, an office environment, or a server cluster). In other example implementations, the processor or processor-implemented components may be distributed across several geographic locations.

[0141] "Computer-readable storage medium" refers to both machine-readable storage media and transmission media. Therefore, the term includes both storage devices / media and carrier / modulated data signals. The terms "machine-readable medium," "computer-readable medium," and "device-readable medium" refer to the same thing and can be used interchangeably in this disclosure.

[0142] A "brief message" is a message that is accessible for a limited time. Email messages can be text, images, videos, etc. The access time for a brief message can be set by the sender. Alternatively, the access time can be a default setting or a setting specified by the recipient. Regardless of the setting technique, the message is temporary.

[0143] "Machine storage medium" refers to one or more storage devices and media (e.g., centralized or distributed databases, and associated caches and servers) that store executable instructions, routines, and data. Therefore, the term should be considered to include, but is not limited to, solid-state memory and optical and magnetic media, including memory internal or external to the processor. Specific examples of machine storage media, computer storage media, and device storage media include: non-volatile memory, including, for example, semiconductor memory devices (e.g., erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), FPGAs, and flash memory devices); disks (such as internal hard drives and removable disks); magneto-optical disks; and CD-ROM and DVD-ROM disks. The terms "machine storage medium," "device storage medium," and "computer storage medium" refer to the same thing and are used interchangeably in this disclosure. The terms "machine storage medium," "computer storage medium," and "device storage medium" expressly exclude carrier waves, modulated data signals, and other such media, at least some of which are covered by the term "signal medium."

[0144] "Non-transitory computer-readable storage medium" refers to a tangible medium capable of storing, encoding, or carrying instructions that can be executed by a machine.

[0145] "Signal medium" means any intangible medium capable of storing, encoding, or carrying instructions executable by a machine, and includes digital or analog communication signals or other intangible media that facilitate the communication of software or data. The term "signal medium" should be considered to include any form of modulated data signal, carrier wave, etc. The term "modulated data signal" means a signal in which one or more characteristics of a signal are set or altered to encode information. The terms "transmission medium" and "signal medium" mean the same thing and may be used interchangeably in this disclosure.

[0146] Changes and modifications may be made to the disclosed embodiments without departing from the scope of this disclosure. Such changes and other modifications are intended to be included within the scope of this disclosure as set forth in the appended claims.

Claims

1. A method for generating event-related content, comprising: The current social event is automatically identified by one or more processors based on matching the current time with the start time associated with a predetermined social event or within a threshold time interval of the start time. Obtain a list of creative tools associated with the current social event; Access multiple content items generated by the client device; Determine that one or more of the creation tools in the list have been applied to a first content item among the plurality of content items; In response to determining that one or more of the creation tools in the list have been applied to the first content item, a subset of content items including the first content item corresponding to the current social event is generated; Based on at least a subset of the content items corresponding to the current social event, a social event template graphic is automatically populated to generate a social event graphic card. as well as Receive a request from the client device to send the social event graphics card to one or more other client devices.

2. The method according to claim 1, wherein, The current social events referred to are New Year's Eve, Christmas Eve, Christmas Day, holidays celebrated worldwide, events celebrated worldwide, events celebrated in a specific region, Easter, Good Friday, Hanukkah, religious holidays, national states of emergency, periods of quarantine in a specific region, or secular holidays.

3. The method according to any one of claims 1 to 2, wherein, The current social event either occurred within a threshold time period or is currently occurring.

4. The method according to claim 3, wherein, The threshold time period includes a time period that depends on the attributes associated with the event.

5. The method according to any one of claims 1 to 4, further comprising: Before generating a subset of the content items, the multiple content items are filtered based on one or more criteria associated with the current social event.

6. The method according to claim 5, wherein, The one or more criteria include at least one of the following: the date of the current social event, the time range of the current social event, the location associated with the current social event, the country associated with the current social event, or the language associated with the current social event.

7. The method according to any one of claims 1 to 6, wherein, The list of creative tools includes at least one of the following: a title containing one or more words associated with the current social event, a graphic element representing the current social event, or an augmented reality element associated with the current social event.

8. The method according to any one of claims 1 to 7, further comprising: Determine the number of content items included in the subset of the content items; as well as The social event template graphic is selected from multiple social event template graphics based on the number of content items included in the subset of the content items.

9. The method according to claim 8, further comprising: A subset of the content items is determined to be sufficient to fill a first quantity of the first social event template graphic among the plurality of social event template graphics; It is determined that the first quantity is less than the threshold; A subset of the content items is determined to be sufficient to fill a second quantity of the second social event template graphic among the plurality of social event template graphics; as well as In response to determining that the second quantity is greater than or equal to the threshold, the second social event template graphic is selected as the social event template graphic.

10. The method according to any one of claims 1 to 9, further comprising: The social event template graphic is determined to include a greater number of content item slots than the number of content items included in the subset of the content items; The first group of content item slots is filled with a subset of the content items; as well as The second group of content item slots is filled with relevant content on topics associated with the current social event.

11. The method according to any one of claims 1 to 10, further comprising: Content items are randomly or pseudo-randomly selected from the plurality of content items to be included in a subset of the content items.

12. The method according to any one of claims 1 to 11, further comprising: This causes the identifier of the social event template graphic to be displayed on the client device; as well as In response to receiving the user's selection of the identifier, the social event template graphic is sent to the one or more other client devices.

13. The method according to any one of claims 1 to 12, further comprising: The identifier is removed after a threshold period has elapsed since the start of the current social event.

14. The method according to any one of claims 1 to 13, wherein, The current social events mentioned are identified in response to the termination of the quarantine period or when the state of emergency is declared over.

15. The method according to any one of claims 1 to 14, further comprising: Determine that one or more creation tools in the list, less than a threshold number, have been applied to one or more of the plurality of content items; as well as In response to determining that one or more creation tools in the list, less than the threshold number, have been applied to the one or more content items, the population of the social event template graphic is prevented.

16. The method according to any one of claims 1 to 15, wherein, The threshold number is adjusted based on the type of the current social event or based on historical information indicating how many users applied one or more creation tools to the content item when the previous current social event occurred.

17. The method according to any one of claims 1 to 16, wherein, The threshold number is adjusted based on a measure of how active the user associated with the client device is on the messaging application platform.

18. The method according to any one of claims 1 to 17, further comprising: Before the current social event begins, the user of the client device is notified about the generation of the social event template graphic.

19. A system comprising: The processor is configured to perform operations, said operations including: The current social event is automatically identified based on whether it matches the current time with the start time associated with the predetermined social event or is within a threshold time interval of the start time. Obtain a list of creative tools associated with the current social event; Access multiple content items generated by the client device; Determine that one or more of the creation tools in the list have been applied to a first content item among the plurality of content items; In response to determining that one or more of the creation tools in the list have been applied to the first content item, a subset of content items including the first content item corresponding to the current social event is generated; Based on at least a subset of the content items corresponding to the current social event, a social event template graphic is automatically populated to generate a social event graphic card; and Receive a request from the client device to send the social event graphics card to one or more other client devices.

20. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations including: The current social event is automatically identified based on whether it matches the current time with the start time associated with the predetermined social event or is within a threshold time interval of the start time. Obtain a list of creative tools associated with the current social event; Access multiple content items generated by the client device; Determine that one or more of the creation tools in the list have been applied to a first content item among the plurality of content items; In response to determining that one or more of the creation tools in the list have been applied to the first content item, a subset of content items including the first content item corresponding to the current social event is generated; Based on at least a subset of the content items corresponding to the current social event, a social event template graphic is automatically populated to generate a social event graphic card. as well as Receive a request from the client device to send the social event graphics card to one or more other client devices.

Images