Hybrid search system for customizable media
Through a hybrid search system combining rough search and thorough search, character counting conditions and text logic are used to evaluate glyph characteristics, the problem of large processing volume when searching for customizable avatars suitable for text input in social network systems is solved, and efficient personalized avatars are achieved deal with.
Patent Information
- Application Number
- CN202280046841.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2021-09-08
- Filing Date
- 2022-06-29
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-06-29
AI Technical Summary
In social networking systems, it is difficult for the prior art to effectively reduce processing while maintaining good recall and accuracy when searching for customizable avatars suitable for combining user-provided text input.
Using a hybrid search system, using a combination of rough search and thorough search, identify suitable split options using character counting conditions (minimum threshold, uncertain threshold and maximum threshold) and evaluate the glyph characteristics of individual characters through text-based logic , to determine whether text input can be incorporated into the avatar.
It effectively reduces the search processing volume while maintaining high recall and accuracy, ensuring the accuracy and efficiency of personalized processing in avatars.
Smart Images

Figure CN117597690B_ABST
Abstract
Description
[0001] Priority declaration
[0002] This application claims the benefit of priority to U.S. Provisional Application Serial No. 63 / 217,030 filed on June 30, 2021 and U.S. Patent Application Serial No. 17 / 447,121 filed on September 8, 2021, each of which is incorporated herein by reference in its entirety. Technical Field
[0003] The present disclosure relates generally to creating and manipulating digital content. Background Art
[0004] As the popularity of mobile-based social networking systems continues to grow, users increasingly share media content items such as electronic images, animations or videos with each other. These media content items are often uniquely personalized and therefore reflect the need to encourage electronic visual communication on a global scale.
[0005] Social networking systems include millions of users. Each user in the social networking system can send media content items between members of his or her personal social networking profile or to individuals outside the social networking profile. BRIEF DESCRIPTION OF THE DRAWINGS
[0006] In the accompanying drawings, similar reference numerals may describe similar components in different views, and the drawings are not necessarily drawn to scale. To easily identify the discussion of any particular element or action, the highest digit or digits in the reference numeral refer to the figure number in which the element is first introduced. Some non-limiting examples are shown in the accompanying drawings, in which:
[0007] Figure 1 is a diagrammatic representation of a networked environment in which the present disclosure may be deployed, according to some examples.
[0008] Figure 2 is a diagrammatic representation of a messaging system having both client-side and server-side functionality according to some examples.
[0009] Figure 3 is a diagrammatic representation of data structures as maintained in a database according to some examples.
[0010] Figure 4 is a graphical representation of messages according to some examples.
[0011] Figure 5 is a flow chart for access restriction processing according to some examples.
[0012] Figure 6A flow diagram of a hybrid search process according to some examples is shown.
[0013] Figure 7 is a diagrammatic representation of an item selection user interface according to some examples.
[0014] Figure 8 is a pictorial representation of a contextual user interface including a personalized media content item, according to some examples.
[0015] Fig. 9 is a pictorial representation of an additional contextual user interface including a personalized media content item according to some examples.
[0016] Fig.10 is a diagrammatic representation of a machine in the form of a computer system according to some examples within which a set of instructions may be executed for causing the machine to perform any one or more of the methodologies discussed herein.
[0017] Fig.11 is a block diagram illustrating a software architecture in which examples may be implemented. DETAILED DESCRIPTION
[0018] The following description includes systems, methods, techniques, instruction sequences and computing machine program products that embody the illustrative embodiments of the present disclosure. In the following description, for the purpose of explanation, many specific details are set forth to provide an understanding of the various embodiments of the subject of the present invention. However, it will be apparent to those skilled in the art that the embodiments of the subject of the present invention can be implemented without these specific details. Typically, known instruction examples, protocols, structures and techniques are not necessarily shown in detail.
[0019] In a social networking system such as a messaging system, digital representations of individual users, called avatars, can be presented in two or three dimensions and can be integrated into a virtual game, messaging system, or interactive software application. The avatars are designed on the back end of the messaging system and stored in an image database. The avatars can be accessed by the messaging system for individual user configuration, e.g., for adding features, text, etc.
[0020] In one example, the messaging system provides an avatar that is configured to incorporate text phrases and allows a user to personalize the avatar by specifying text to be incorporated into the avatar's presentation. For the purposes of this description, an avatar that can be personalized by specifying text to be incorporated into the avatar is referred to as a customizable avatar. An avatar that has been configured to incorporate text provided by a user is referred to as a personalized avatar. A personalized avatar that integrates user-provided input (e.g., letters, words, phrases, and emoticons) can be overlaid on top of a media content item (e.g., a photo or video), can be included in a chat message, can be stored for future use, etc.
[0021] Depending on the text characteristics of a given avatar (e.g., the size of a text placeholder associated with the avatar), not every avatar will be appropriate for the input text provided by the user. For example, if the text characteristics of the avatar indicate that the avatar can contain no more than one line of text or no longer than 5 characters, then the text input spelling "FAREWELL!" will not be considered a match to that avatar. On the other hand, if the avatar is configured so that it can contain three lines of text, with a maximum of 5 characters per line, then the text input spelling "FARE THEE WELL!" will be considered a match to the avatar's text characteristics because the text input can be incorporated into the three lines of text in the avatar as "FARE" and "THEE" as well as "WELL!". The messaging system can be configured to detect the text input provided by the user and identify a customizable avatar that is suitable for incorporating the text input.
[0022] The following technical problems are solved by providing a hybrid search system as described herein: reducing the amount of processing involved in searching for customizable avatars suitable for incorporating text input provided by a user, while maintaining good recall and precision.
[0023] In some examples, the hybrid search system first performs a rough search to determine, with respect to each customizable avatar, whether it is suitable for incorporating input text entered by the user, whether it is not suitable for incorporating text entered by the user, or whether it is not suitable or not suitable for incorporating text entered by the user. The rough search is based on comparing the character count of the input text to a threshold stored in association with the individual avatars. After the rough search, a more thorough search is performed only with respect to customizable avatars for which the rough search did not produce a clear indication of whether the user-entered text can be incorporated into the avatar.
[0024] In order to implement the hybrid search method, the hybrid search system associates each customizable avatar with a set of parameters, which are referred to as character count conditions (CCC) for the purpose of this description. The CCC includes a minimum threshold, an uncertain threshold, and a maximum threshold. The minimum threshold indicates the minimum number of characters that can be incorporated into a line of text in the customizable avatar. For example, the customizable avatar can be configured to require that the line of text cannot be empty, or that the line should include at least two characters. The uncertain threshold indicates the maximum number of characters that can be presented as a line of text when the line is completely filled with the maximum character (e.g., "W"). The maximum threshold indicates the maximum number of characters that can be included in a line of text when the line is completely filled with the minimum character (e.g., "i"). If the character count in the text entered by the user is less than or equal to the maximum number of the maximum characters (uncertain threshold), it is certain that the text can be incorporated into the associated line in the customizable avatar. If the character count in the text entered by the user is greater than the maximum number of the minimum characters (maximum threshold), it is certain that the text cannot be incorporated into the associated line in the customizable avatar. The customizable avatar is stored in association with a data structure that includes an identifier for the avatar, a value indicating the number of lines of text that may be incorporated into the avatar, and a corresponding CCC associated with each line.
[0025] The processing performed by the hybrid search system can be described as follows. The system detects user input in the form of text. The detected input text is then used to generate split options that include various combinations of parts of the text input that are segmented into one or more lines. For example, the input text "Be My Valentine" can be split into the following four split options (as shown in Example 1 below).
[0026] Example 1
[0027] Option (1) “Be My Valentine”
[0028] Option (2) "Be My" and "Valentine"
[0029] Option (3) "Be" and "My Valentine"
[0030] Options (4) "Be", "My", and "Valentine"
[0031] With respect to each split option, the hybrid search system performs a rough search to identify each customizable avatar as suitable, unsuitable, or uncertain based on the CCC of each customizable avatar. Using the split options shown in Example 1, the rough search may identify an avatar that is configured to combine three lines of text - the first line CCC is (1, 2, 4), the second line CCC is (1, 4, 6), and the third line CCC is (1, 10, 12) - as suitable with respect to split option (4). Such an avatar can be personalized to include "Be" in the first line, "My" in the second line, and "Valentine" in the third line. The rough search may identify the following avatars as unsuitable with split option (4): an avatar with less than three lines; or an avatar with three lines, but the maximum threshold for each line is 7, which is less than the number of characters in the word "Valentine". A cursory search may identify the following avatar as uncertain about split option (4): the avatar has three lines, the first line CCC is (1, 2, 3), the second line CCC is (1, 4, 5), and the third line CCC is (1, 8, 9). In this case, the uncertainty threshold and maximum threshold for the third line are 8 and 9, respectively, and the third word in split option (4) has 9 characters, which may or may not fit in line 3 of the text placeholder, depending on the specific combination of characters in the word, as some characters may be wider than others and some characters may be narrower than others.
[0032] The results of the rough search can be characterized by whether the corresponding split options are suitable, unsuitable, or uncertain relative to a given customizable avatar. If, based on the rough search, at least one of the split options generated for the input text has been identified as suitable relative to a given avatar, the avatar is personalized by combining the input text in the form of the split option and presented on the display device as a selection option for the user. If, based on the rough search, all split options generated for the input text have been identified as unsuitable relative to a given avatar, the avatar is no longer considered a candidate for personalization by the input text. If, based on the rough search, none of the split options generated for the input text have been identified as suitable relative to a given avatar, and at least one of the split options generated for the input text has been identified as uncertain relative to a given avatar, the hybrid search system can continue with a thorough search, which involves computationally more expensive text-based logic to determine whether the split options can be re-characterized as suitable for incorporation into a given avatar. In some examples, the text-based logic may need to evaluate the corresponding glyph features of individual characters in one or more rows of the uncertain split options. Glyph features include one or more of font type, size, dimension, kerning, and shape of each individual character in a line of text.
[0033] If, based on the exhaustive search, a split option has been recharacterized as suitable for incorporation into a given avatar, the avatar is personalized by incorporating the input text in the form of the split option and presented on a display device as a selection option for the user.
[0034] When a user selects one of the personalized avatars presented on the display device, the messaging system detects the selection and, in response, includes the selected personalized avatar into a contextual user interface, such as a chat messaging user interface (allowing a user to create, send, and receive messages over a network), an image editing user interface (allowing a user to edit previously captured images), etc.
[0035] It will be noted that while the methods are described herein with respect to customizable avatars, the methods may be beneficially used with respect to any media content item that may be associated with a character count condition and configured to incorporate text data provided by a user.
[0036] Networked computing environment
[0037] Figure 1 1 is a block diagram illustrating an example messaging system 100 for exchanging data (e.g., messages and associated content) over a network. The messaging system 100 includes multiple instances of client devices 102, each of the multiple instances hosting a number of applications including a messaging client 104. Each messaging client 104 is communicatively coupled to other instances of the messaging client 104 and a messaging server system 108 via a network 106 (e.g., the Internet).
[0038] The messaging client 104 is able to communicate and exchange data with another messaging client 104 and a messaging server system 108 via the network 106. The data exchanged between the messaging clients 104 and between the messaging clients 104 and the messaging server system 108 includes functions (e.g., commands to activate functions) and payload data (e.g., text, audio, video, or other multimedia data).
[0039] The messaging server system 108 provides server-side functionality to specific messaging clients 104 via the network 106. Although certain functions of the messaging system 100 are described herein as being performed by the messaging client 104 or by the messaging server system 108, it may be a design choice whether certain functions are located within the messaging client 104 or the messaging server system 108. For example, it may be technically preferred to initially deploy certain technologies and functions within the messaging server system 108, but later migrate the technologies and functions to the messaging client 104 where the client device 102 has sufficient processing power.
[0040] The messaging server system 108 supports various services and operations provided to the messaging clients 104. Such operations include sending data to the messaging clients 104, receiving data from the messaging clients 104, and processing data generated by the messaging clients 104. As examples, the data may include message content, client device information, geolocation information, media enhancements and overlays, message content persistence conditions, social network information, live event information, and images and videos captured using a viewfinder ring flash using a front-facing camera of an associated client device. The exchange of data within the messaging system 100 is activated and controlled by functions available via a user interface (UI) of the messaging client 104. For example, the messaging client 104 may present a camera view UI that displays the output of a digital image sensor of a camera provided with the client device 102, a chat message UI that allows a user to create, send, and receive messages over a network, an image editing user interface that allows a user to edit an image (e.g., an image previously captured using a camera view UI), and the like.
[0041] Turning now specifically to the messaging server system 108, an application program interface (API) server 110 is coupled to and provides a programming interface to an application server 112. The application server 112 is communicatively coupled to a database server 118, which facilitates access to a database 120 that stores data associated with messages processed by the application server 112. Similarly, a web server 124 is coupled to the application server 112 and provides a web-based interface to the application server 112. To this end, the web server 124 processes incoming network requests via the Hypertext Transfer Protocol (HTTP) and several other related protocols.
[0042] The application program interface (API) server 110 receives and sends message data (e.g., commands and message payloads) between the client device 102 and the application server 112. Specifically, the application program interface (API) server 110 provides a set of interfaces (e.g., routines and protocols) that the messaging client 104 can call or query to activate the functions of the application server 112. The application program interface (API) server 110 exposes various functions supported by the application server 112, including: account registration; login functionality; sending messages from a particular messaging client 104 to another messaging client 104 via the application server 112; sending media files (e.g., images or videos) from a messaging client 104 to the messaging server 114 and for possible access by another messaging client 104; setting of media data collections (e.g., stories); retrieving a friend list of a user of the client device 102; retrieving such collections; retrieving messages and content; adding and removing entities (e.g., friends) from an entity graph (e.g., a social graph); locating friends in a social graph; and opening application events (e.g., related to a messaging client 104).
[0043] The application server 112 hosts several server applications and subsystems, including, for example, a messaging server 114, an image processing server 116, and a social network server 122. The messaging server 114 implements several message processing technologies and functions, particularly those related to the aggregation and other processing of content (e.g., text and multimedia content) included in messages received from multiple instances of the messaging client 104. As will be described in further detail, text and media content from multiple sources can be aggregated into collections of content (e.g., referred to as stories or galleries). These collections are then made available to the messaging client 104. In some examples, a collection may include items with overlaid personalized media content items, such as personalized avatars of text corresponding to text input provided by a user. In view of the hardware requirements for such processing, other processor and memory intensive data processing may also be performed on the server side by the messaging server 114.
[0044] The application server 112 also includes an image processing server 116 that is dedicated to performing various image processing operations, typically on images or videos within the payload of messages sent from the messaging server 114 or received at the messaging server 118. Some of the various image processing operations may be performed by various AR components, which may be hosted or supported by the image processing server 116. In some examples, the image processing server 116 is configured to provide the functionality of the hybrid search system described herein, which helps reduce the amount of processing involved when searching for customizable media content items suitable for incorporating text input provided by a user, while maintaining good recall and precision.
[0045] The social network server 122 supports various social networking functions and services and makes them available to the messaging server 114. To this end, the social network server 122 maintains and accesses the entity graph 306 (e.g., Figure 3 Examples of functions and services supported by social network server 122 include identifying other users in messaging system 100 that have a "friend" relationship with a particular user or that the particular user is "following," and identifying interests and other entities of a particular user.
[0046] System Architecture
[0047] Figure 2 1 is a block diagram showing additional details about the messaging system 100 according to some examples. Specifically, the messaging system 100 is shown as including a messaging client 104 and an application server 112. The messaging system 100 includes several subsystems that are supported on the client side by the messaging client 104 and on the server side by the application server 112. These subsystems include, for example, a transient timer system 202, a collection management system 204, an enhancement system 206, and a hybrid search system 208.
[0048] The hybrid search system 208 is configured to identify customizable media content items that can be personalized by incorporating text input provided by a user (as described herein).
[0049] The transient timer system 202 is responsible for enforcing temporary or time-limited access to content by the messaging client 104 and the messaging server 114. The transient timer system 202 includes several timers that selectively enable access (e.g., for presentation and display) of messages and associated content via the messaging client 104 based on duration and display parameters associated with a message or a collection of messages (e.g., a story). Additional details regarding the operation of the transient timer system 202 are provided below.
[0050] The collection management system 204 is responsible for managing collections or collections of media (e.g., collections of text, images, video, and audio data). Collections of content (e.g., messages, including images, videos, text, and audio) can be organized into "event libraries" or "event stories." Such collections can be made available for a specified time period, such as the duration of an event related to the content. For example, content related to a concert can be made available as a "story" for the duration of the concert. In another example, a collection may include content generated using one or more AR components. In some examples, the media content items in the collection include overlay augmented data in the form of personalized media content items, such as personalized avatars. The collection management system 204 may also be responsible for publishing an icon that provides notification of the existence of a particular collection to the user interface of the messaging client 104.
[0051] In addition, the collection management system 204 also includes a curation interface 212 that allows a collection manager to manage and curate specific collections of content. For example, the curation interface 212 enables an event organizer to curate a collection of content related to a specific event (e.g., to remove inappropriate content or redundant messages). In addition, the collection management system 204 uses machine vision (or image recognition technology) and content rules to automatically curate content collections. In some examples, compensation may be paid to users for including user-generated content in a collection. In such cases, the collection management system 204 operates to automatically pay such users for using their content.
[0052] The enhancement system 206 provides various functions that enable a user to enhance (e.g., annotate or otherwise modify or edit) media content that may be associated with a message. For example, the enhancement system 206 provides functions related to generating and publishing media overlays for messages processed by the messaging system 100. The media overlays may be stored in the database 120 and accessed through the database server 118.
[0053] The enhancement system 206 provides various functions that enable a user to enhance (e.g., annotate or otherwise modify or edit) media content associated with a message. For example, the enhancement system 206 provides functions related to generating and publishing media overlays for messages processed by the messaging system 100. The enhancement system 206 is operable to provide media overlays or enhancements (e.g., image filters) to the messaging client 104 based on the geolocation of the client device 102. In another example, the enhancement system 206 is operable to provide media overlays to the messaging client 104 based on other information such as social network information of the user of the client device 102. Media overlays can include audio and visual content and visual effects. Examples of audio and visual content include pictures, text, logos, animations, and sound effects. Examples of visual effects include color overlays.
[0054] Audio and visual content or visual effects may be applied to a media content item (e.g., a photo) at the client device 102. For example, a media overlay may include text or an image that may be overlaid on top of a photo taken by the client device 102, or included in a message sent from a messaging client executing on one client device to a messaging client executing on another client device. The media overlay may be configured to incorporate custom text provided by the user, and may be identified as being suitable or unsuitable for incorporation into input text using the methods described herein with respect to a hybrid search system.
[0055] In another example, the media overlay includes a location identification overlay (e.g., Venice Beach), the name of a live event, or a business name overlay (e.g., Beach Cafe). In another example, the enhancement system 206 uses the geolocation of the client device 102 to identify a media overlay that includes the name of the business at the geolocation of the client device 102. The media overlay may include other tags associated with the business. The media overlay may be stored in the database 126 and accessed through the database server 118.
[0056] In some examples, the augmentation system 206 is configured to provide access to an AR component that can be implemented using a programming language suitable for application development, such as JavaScript or Java, and identified in the messaging server system by a corresponding AR component identifier. The AR component may include or reference various image processing operations corresponding to image modification, filtering, media overlay, transformation, etc. These image processing operations can provide an interactive experience of a real-world environment, in which objects, surfaces, backgrounds, lighting, etc. captured by a digital image sensor or camera are enhanced by computer-generated perceptual information. In this case, the AR component includes a collection of data, parameters, and other assets required to apply a selected augmented reality experience to an image or video feed.
[0057] In some embodiments, the AR component includes a module configured to modify or transform image data presented within a graphical user interface (GUI) of a client device in some manner. For example, AR component data can be used to perform complex additions or transformations to content images, such as adding bunny ears to a person's head in a video clip, adding a floating heart with a background color to a video clip, changing the scale of a person's features within a video clip, or many other such transformations. This includes both real-time modifications that modify the image using the AR component modification when the image is captured using a camera associated with the client device and then displayed on the screen of the client device, as well as modifications to stored content, such as video clips in a library that can be modified using the AR component.
[0058] Various augmented reality functions that can be provided by the AR component include detecting objects (e.g., faces, hands, bodies, cats, dogs, surfaces, objects, etc.), tracking these objects as they leave, enter, and move around the field of view in a video frame, and modifying or transforming them as they are tracked. In various embodiments, different methods for implementing such transformations can be used. For example, some embodiments may involve generating a 3D mesh model of one or more objects, and using transformations and animated textures of the model within the video to implement the transformations. In other embodiments, tracking of points on an object can be used to place an image or texture at the tracked location, which image or texture can be two-dimensional or three-dimensional. In other embodiments, neural network analysis of video frames can be used to place images, models, or textures in content (e.g., frames of an image or video). Therefore, AR component data refers to both images, models, and textures used to create transformations in content, as well as additional modeling and analysis information required to implement such transformations using object detection, tracking, and placement.
[0059] Data Architecture
[0060] Figure 3 is a diagram illustrating a data structure 300 that may be stored in a database 120 of a messaging server system 108 according to some examples. Although the contents of the database 120 are illustrated as including several tables, it should be understood that data may be stored in other types of data structures (e.g., as an object-oriented database).
[0061] The database 120 includes message data stored in the message table 302. For any particular message, the message data includes at least message sender data, message recipient (or receiver) data, and a payload. The payload of the message may include content generated using the viewfinder ring flash. Figure 4Additional details regarding information that may be included in a message and included within the message data stored in message table 302 are described.
[0062] The entity table 304 stores entity data and is linked (e.g., referenced) to the entity graph 306 and profile data 308. The entities for which records are maintained within the entity table 304 may include individuals, corporate entities, organizations, objects, places, events, etc. Regardless of the entity type, any entity for which the messaging server system 108 stores data may be an identified entity. Each entity is provided with a unique identifier as well as an entity type identifier (not shown).
[0063] The entity graph 306 stores information about relationships and associations between entities. By way of example only, such relationships may be social, professional (e.g., working in a common company or organization), interest-based, or activity-based. With reference to functionality provided by the AR component, the entity graph 306 stores information that may be used to determine additional profiles connected to a profile representing a user controlling an associated client device when the AR component is configured to allow modification of a target media content object using a portrait image of a user other than the portrait image of the user controlling the associated client device. As mentioned above, a portrait image of a user may be stored in a user profile representing the user in a messaging system.
[0064] Profile data 308 stores multiple types of profile data about a particular entity. Based on the privacy settings specified by the particular entity, profile data 308 can be selectively used and presented to other users of the messaging system 100. In the case where the entity is a person, profile data 308 includes, for example, a user name, phone number, address, settings (e.g., notification and privacy settings), and a user-selected visualization (or a collection of such visualizations). A particular user can then selectively include one or more of these visualizations within the content of messages transmitted via the messaging system 100 and on a map interface displayed to other users by the messaging client 104. A collection of visualizations can include a "status visualization" that presents a graphical representation of a status or activity that a user may choose to convey at a particular time.
[0065] The database 120 also stores enhancement data in an enhancement table 310. The enhancement data is associated with and applied to videos (whose data is stored in a video table 314) and images (whose data is stored in an image table 316). In some examples, the enhancement data is used by various AR components including an AR component. An example of enhancement data is an augmented reality (AR) tool that can be used in an AR component to implement image transformation. Image transformation includes real-time modification of an image (e.g., a video frame) while the image is captured using a digital image sensor of the client device 102. The modified image is displayed on the screen of the client device 102 together with the modification.
[0066] The story table 312 stores data about a collection of messages and associated image, video, or audio data that is compiled into a collection (e.g., a story or library). The creation of a particular collection can be initiated by a particular user (e.g., each user whose record is maintained in the entity table 304). A user can create a "personal story" in the form of a collection of content that has been created and sent / broadcasted by the user. To this end, the user interface of the messaging client 104 may include a user-selectable icon to enable the sending user to add specific content to his or her personal story. In some examples, the story table 312 stores one or more images or videos created using a viewfinder ring flash.
[0067] As mentioned above, the video table 314 stores video data, which in one example is associated with a message whose record is maintained within the message table 302. Similarly, the image table 316 stores image data that may be associated with a message whose message data is stored in the entity table 304. In some examples, the image table 316 stores customizable overlays such as customizable avatars. The entity table 304 can associate various enhancements from the enhancement table 310 with various images and videos stored in the image table 316 and the video table 314.
[0068] Data communication architecture
[0069] Figure 4 is a schematic diagram illustrating the structure of a message 400 according to some examples, the message 400 being generated by the messaging client 104 for transmission to another messaging client 104 or the messaging server 114. The content of a particular message 400 is used to populate a message table 302 stored in the database 120, which is accessible by the messaging server 114. Similarly, the content of the message 400 is stored in memory as "in-flight" or "in-flight" data of the client device 102 or the application server 112. In some examples, the content of the message 400 includes an image or video created using an AR component. The message 400 is shown as including the following example components:
[0070] Message identifier 402 : a unique identifier that identifies the message 400 .
[0071] Message text payload 404 : Text to be generated by a user via the user interface of the client device 102 and included in the message 400 .
[0072] Message image payload 406: Image data captured by a camera component of the client device 102 or retrieved from a memory component of the client device 102 and included in the message 400. Image data for a sent or received message 400 may be stored in the image table 316. In some examples, the image data may include a personalized image, such as a personalized avatar that includes text corresponding to the text entered by the user.
[0073] Message video payload 408: Video data captured by the camera component or retrieved from the memory component of the client device 102 and included in the message 400. Video data for a sent or received message 400 may be stored in the video table 314. The video data may include content generated using a viewfinder ring flash.
[0074] Message audio payload 410 : audio data captured by a microphone or retrieved from a memory component of the client device 102 and included in the message 400 .
[0075] Message enhancement data 412: Enhancement data (e.g., filters, stickers, or other annotations or enhancements) representing enhancements to be applied to the message image payload 406, message video payload 408, message audio payload 410 of the message 400. The enhancement data for a sent or received message 400 may be stored in the enhancement table 310.
[0076] Message duration parameter 414: A parameter value indicating the amount of time in seconds that the content of a message (eg, message image payload 406, message video payload 408, message audio payload 410) will be presented to or made accessible to a user via messaging client 104.
[0077] Message geolocation parameters 416: Geolocation data (e.g., latitude and longitude coordinates) associated with the content payload of the message. Multiple message geolocation parameter 416 values may be included in the payload, each of which is associated with information about a content item included in the content (e.g., a specific image within a message image payload 406, or a specific video within a message video payload 408).
[0078] Message story identifier 418: An identifier value that identifies one or more content collections (e.g., a "story" identified in stories table 312) associated with a particular content item in message image payload 406 of message 400. For example, an identifier value may be used to associate multiple images within message image payload 406 with each of multiple content collections.
[0079] Message tags 420: Each message 400 may be tagged with a plurality of tags, each of which indicates the subject of the content included in the message payload. For example, where a particular image included in the message image payload 406 depicts an animal (e.g., a lion), a tag value indicating the relevant animal may be included within the message tags 420. The tag values may be manually generated based on user input, or may be automatically generated using, for example, image recognition.
[0080] • Message sender identifier 422: An identifier (eg, a messaging system identifier, an email address, or a device identifier) that indicates the user of the client device 102 on which the message 400 was generated and from which the message 400 was sent.
[0081] • Message recipient identifier 424: An identifier (eg, a messaging system identifier, an email address, or a device identifier) that indicates the user of the client device 102 to which the message 400 is addressed.
[0082] The content (e.g., values) of the various components of message 400 may be pointers to locations in a table where content data values are stored. For example, an image value in message image payload 406 may be a pointer to a location (or address of a location) within image table 316. Similarly, a value within message video payload 408 may point to data stored within video table 314, a value stored within message enhancement 412 may point to data stored in enhancement table 310, a value stored within message story identifier 418 may point to data stored in story table 312, and values stored within message sender identifier 422 and message recipient identifier 424 may point to user records stored in entity table 304.
[0083] Time-based access restriction architecture
[0084] Figure 5 is a schematic diagram illustrating an access restriction process 500, according to which access to content (e.g., a multimedia payload of ephemeral message 502 and associated data) or a collection of content (e.g., ephemeral message group 504) may be time-limited (e.g., ephemeral). In some examples, the content of ephemeral message 502 includes an image or video created using a viewfinder ring flash.
[0085] The ephemeral message 502 is shown to be associated with a message duration parameter 506, the value of which determines the amount of time that the ephemeral message 502 will be displayed by the messaging client 104 to the receiving user of the ephemeral message 502. In one example, the ephemeral message 502 may be viewed by the receiving user for up to 10 seconds, depending on the amount of time specified by the sending user using the message duration parameter 506. In some examples, the ephemeral message 502 may include a personalized image, such as a personalized avatar that includes text corresponding to the text entered by the user.
[0086] The message duration parameter 506 and the message recipient identifier 424 are shown as inputs to a message timer 512, which is responsible for determining the amount of time that the transient message 502 is shown to the particular receiving user identified by the message recipient identifier 424. In particular, the transient message 502 is only shown to the relevant receiving user for the period of time determined by the value of the message duration parameter 506. The message timer 512 is shown as providing an output to a more general transient timer system 202, which is responsible for the overall timing of displaying content (e.g., the transient message 502) to the receiving user.
[0087] exist Figure 5 1. Ephemeral message 502 is shown as being included within an ephemeral message group 504 (e.g., a collection of messages in a personal story or event story). Ephemeral message group 504 has an associated group duration parameter 508, the value of which determines the duration for which ephemeral message group 504 is presented and accessible to users of messaging system 100. For example, group duration parameter 508 may be the duration of a concert, wherein ephemeral message group 504 is a collection of content related to the concert. Alternatively, a user (an owning user or a curator user) may specify the value of group duration parameter 508 when performing setup and creation of ephemeral message group 504.
[0088] In addition, each ephemeral message 502 within the ephemeral message group 504 has an associated group participation parameter 510, the value of which determines the duration for which the ephemeral message 502 will be accessible within the context of the ephemeral message group 504. Thus, a particular ephemeral message group 504 may "expire" and become inaccessible within the context of the ephemeral message group 504 before the ephemeral message group 504 itself expires according to the group duration parameter 508. The group duration parameter 508, the group participation parameter 510, and the message recipient identifier 424 each provide inputs to a group timer 514, which first operatively determines whether a particular ephemeral message 502 of the ephemeral message group 504 will be displayed to a particular receiving user, and if so, how long to display it. Note that the identity of the particular receiving user is also known to the ephemeral message group 504 due to the message recipient identifier 424.
[0089] Thus, the group timer 514 operatively controls the total lifespan of the associated ephemeral message group 504 and the individual ephemeral messages 502 included in the ephemeral message group 504. In one example, each ephemeral message 502 within the ephemeral message group 504 remains viewable and accessible for a period of time specified by the group duration parameter 508. In another example, within the context of the ephemeral message group 504, a particular ephemeral message 502 may expire based on the group participation parameter 510. Note that even within the context of the ephemeral message group 504, the message duration parameter 506 may still determine the duration for which a particular ephemeral message 502 is displayed to a receiving user. Thus, the message duration parameter 506 determines the duration for which a particular ephemeral message 502 is displayed to a receiving user, regardless of whether the receiving user is viewing the ephemeral message 502 within or outside the context of the ephemeral message group 504.
[0090] In addition, the transient timer system 202 may be operative to remove a particular transient message 502 from the transient message group 504 based on a determination that the particular transient message 502 has exceeded the associated group participation parameter 510. When the sending user has established a group participation parameter 510 of 24 hours from posting, the transient timer system 202 will remove the associated transient message 502 from the transient message group 504 after the specified 24 hours. The transient timer system 202 is also operative to remove a transient message group 504 when the group participation parameter 510 for each transient message 502 within the transient message group 504 has expired, or when the transient message group 504 itself has expired according to the group duration parameter 508.
[0091] In some use cases, the creator of a particular ephemeral message group 504 may specify an indefinite group duration parameter 508. In this case, the expiration of the group participation parameter 510 of the last remaining ephemeral message 502 within the ephemeral message group 504 will determine when the ephemeral message group 504 itself expires. In this case, a new ephemeral message 502 with a new group participation parameter 510 added to the ephemeral message group 504 effectively extends the life of the ephemeral message group 504 to a value equal to the group participation parameter 510.
[0092] In response to the transient timer system 202 determining that the transient message group 504 has expired (e.g., is no longer accessible), the transient timer system 202 communicates with the messaging system 100 (and, for example, in particular, the messaging client 104) to cause the indicia (e.g., icon) associated with the relevant transient message group 504 to no longer be displayed within the user interface of the messaging client 104. Similarly, when the transient timer system 202 determines that the message duration parameter 506 of a particular transient message 502 has expired, the transient timer system 202 causes the messaging client 104 to no longer display the indicia (e.g., icon or text identification) associated with the transient message 502.
[0093] Process flow and user interface
[0094] Figure 6 A flow chart of a hybrid search process 600 according to some examples is shown. Although certain operations of the process 600 may be described as being performed by certain devices, in different examples, different devices or combinations of devices may perform these operations. For example, the operations described below may be performed by a client device 102, or in conjunction with a server-side computing device (e.g., a messaging server system 108) or a third-party server computing device.
[0095] At operation 610, the method begins with the following operations: accessing a text input that may be provided by a user. The text input is associated with a character count. Operation 620 includes identifying each item from a media content item stored in a database as being suitable, unsuitable, or uncertain relative to the text input. Identifying a media content item as uncertain indicates that the item is suitable for matching a character count of a first size (e.g., a minimum size) and is not suitable for matching a character count of a second size (a larger size).
[0096] As described above, each customizable media content item is associated with a CCC that includes a first condition, a second condition, and a third condition. For a given media content item, the value of the first condition is a minimum threshold, the value of the second condition is an indeterminate threshold (indicating the maximum number of characters that can be presented as a line of text if the line is completely filled with the maximum characters), and the value of the third condition is a maximum threshold (indicating the maximum number of characters that can be included in a line of text if the line is completely filled with the minimum characters).
[0097] For example, assume that the text input is "HELLO" with a character count of 5. Also assume that the customizable media content item is associated with CCC (1, 4, 7), which indicates that it can incorporate up to 7 small characters or text (e.g., "iiiiiii"), but no more than 4 large characters, such that the text cannot be larger than "WWWW". Although the character count of "HELLO" is less than 7, without evaluating the glyph features of the individual characters in "HELLO", it cannot be determined with certainty that such input text can be incorporated into the customizable media content item.
[0098] As described above, identifying the media content item as suitable relative to the input text by performing a coarse search includes: determining that the character count of the input text is greater than a value associated with the first condition and less than a value associated with the third condition. Identifying the media content item as unsuitable includes: determining that the character count of the input text is greater than a value associated with the third condition. Identifying the media content item as uncertain includes: determining that the character count of the input text is greater than a value associated with the second condition and less than a value associated with the third condition.
[0099] Operation 630 includes applying text matching logic to the text input with respect to each media content item identified as ambiguous so as to clarify the ambiguity and identify the respective item as suitable or unsuitable. Applying the text matching logic includes evaluating glyph features of individual characters relative to the customizable media content items.
[0100] As mentioned above, customizable media content items can be suitable for combining one or more lines of text, and can be stored in association with a data structure, and this data structure comprises the value (line count) of the number of text lines that indication can be incorporated into the avatar and the corresponding CCC associated with each line, i.e. a group of line count text presentation conditions.Each member in this group of line count text presentation conditions comprises the first condition, the second condition and the third condition.In some examples, input text is used to generate split options, and wherein each option comprises the text input that is split into one or more lines.If it is determined that customizable media content items can be combined with at least one split option, then customizable media content items are identified as being suitable for text input.If based on the CCC associated with customizable media content items, it is determined that customizable media content items can not be combined with any one split option, then customizable media content items are identified as being not suitable for text input.
[0101] If it is determined based on the CCC associated with the customizable media content item that (1) for none of the split options, the number of characters in each line is less than or equal to the value of the associated second condition from the CCC (the maximum value of the largest characters), and (2) for at least one of the split options, the number of characters in each line is less than or equal to the value of the associated third condition (the maximum value of the smallest characters), then the customizable media content item is identified as uncertain with respect to the text input. Determining that the number of characters in a line of text is less than or equal to the value of the associated second condition from the CCC can be referred to as matching the second condition. Determining that the number of characters in a line of text is less than or equal to the value of the associated third condition from the CCC can be referred to as matching the third condition.
[0102] The split options identified as uncertain with respect to the input text may be further analyzed by evaluating the respective glyph features of the individual characters in the uncertain options with respect to the customizable media content item. Based on the results of the evaluation, the customizable media content item may be recharacterized as suitable or unsuitable with respect to the uncertain options.
[0103] return Figure 6 , operation 640 comprises generating personalized media content items by incorporating text input into each of the media content items that have been identified as being suitable. In one example, the personalized media content items include two image layers. The first image layer may include text generated based on the input text, and the text is overlaid on a predetermined position in the second image layer, and the second image layer may include an image stored in the associated customizable media content items. In some examples, when generating the personalized media content items, the system manipulates text (e.g., letter spacing, letter size, letter font, etc.). In another example, the system manipulates the features of the customizable media content items (e.g., avatar orientation and alignment, rotation, media object placement, etc.) so that text is integrated into the media content items.
[0104] Operation 650 includes generating an item selection user interface including one or more of the personalized media content items. Figure 7 An example of an item selection user interface included in schematic 700 is shown. Figure 7 The text entry "HELLO" in area 710 and a presentation 720 of a personalized media content item (in this example, an avatar) are shown.
[0105] After the item selection user interface is presented on the display device, and in response to receiving an indication of an item selected from the personalized media content items in the item selection user interface, the selected item is included in the contextual user interface. Figure 8 An example of a contextual user interface included in diagrammatic representation 800 is shown. Figure 8 A chat messaging user interface is shown in which a personalized avatar is included in area 810 in conjunction with the text input “HELLO”. Fig. 9 Another example of a contextual user interface is shown in FIG, as diagrammatic representation 900, where a personalized avatar in conjunction with a text input “HELLO” is included in an image editing user interface in area 910. Shown in area 910 is an image of clouds (e.g., representing a previously captured photo) with the personalized avatar in conjunction with the text input “HELLO” overlaid on the image of the clouds.
[0106] Machine Architecture
[0107] Fig.101000 within which instructions 1008 (e.g., software, programs, applications, applet, app, or other executable code) may be executed to cause the machine 1000 to perform any one or more of the methods discussed herein. For example, the instructions 1008 may cause the machine 1000 to perform any one or more of the methods described herein. The instructions 1008 convert a general-purpose, non-programmed machine 1000 into a specific machine 1000 that is programmed to perform the functions described and illustrated in the manner described. The machine 1000 may operate as a standalone device or may be coupled (e.g., networked) to other machines. In a networked deployment, the machine 1000 may operate as a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 1000 may include, but is not limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a personal digital assistant (PDA), an entertainment media system, a cellular phone, a smart phone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of sequentially or otherwise executing instructions 1008 specifying actions to be taken by the machine 1000. In addition, although only a single machine 1000 is shown, the term "machine" should also be considered to include a collection of machines that individually or jointly execute instructions 1008 to perform any one or more of the methods discussed herein. For example, the machine 1000 may include any of the client device 102 or a plurality of server devices that form part of the messaging server system 108. In some examples, the machine 1000 may also include both a client system and a server system, wherein certain operations of a particular method or algorithm are performed on the server side and certain operations of a particular method or algorithm are performed on the client side.
[0108] The machine 1000 may include a processor 1002, a memory 1004, and an input / output I / O component 1038, which may be configured to communicate with each other via a bus 1040. In an example, the processor 1002 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 1006 that executes instructions 1008 and a processor 1010. The term "processor" is intended to include multi-core processors, which may include two or more independent processors (sometimes referred to as "cores") that can execute instructions simultaneously. Although Fig.10 Multiple processors 1002 are shown, but the machine 1000 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.
[0109] The memory 1004 includes a main memory 1012, a static memory 1014, and a storage unit 1016, which are all accessible by the processor 1002 via the bus 1040. The main memory 1004, the static memory 1014, and the storage unit 1016 store instructions 1008 that embody any one or more of the methodologies or functions described herein. The instructions 1008 may also reside, completely or partially, within the main memory 1012, within the static memory 1014, within the machine-readable medium 1018 within the storage unit 1016, within at least one of the processors 1002 (e.g., within a cache memory of a processor), or within any suitable combination thereof during execution thereof by the machine 1000.
[0110] I / O components 1038 may include various components for receiving input, providing output, generating output, sending information, exchanging information, capturing measurements, etc. The specific I / O components 1038 included in a particular machine will depend on the type of machine. For example, a portable machine such as a mobile phone may include a touch input device or other such input mechanism, while a headless server machine may not include such a touch input device. It should be understood that I / O components 1038 may include Fig.101024 and 1026. In various examples, the I / O components 1038 may include a user output component 1024 and a user input component 1026. The user output component 1024 may include a visual component (e.g., a display such as a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), an auditory component (e.g., a speaker), a tactile component (e.g., a vibration motor, a resistance mechanism), other signal generators, etc. The user input component 1026 may include an alphanumeric input component (e.g., a keyboard, a touch screen configured to receive alphanumeric input, an optical keyboard, or other alphanumeric input component), a pointing-based input component (e.g., a mouse, a touch pad, a trackball, a joystick, a motion sensor, or another pointing instrument), a tactile input component (e.g., a physical button, a touch screen or other tactile input component that provides location and force of a touch or touch gesture), an audio input component (e.g., a microphone), etc.
[0111] In another example, the I / O component 1038 may include a biometric component 1028, a motion component 1030, an environment component 1032, or a position component 1034, as well as a variety of other components. For example, the biometric component 1028 includes components for detecting expressions (e.g., hand expressions, facial expressions, voice expressions, body postures, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweat, or brain waves), identifying people (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or EEG-based recognition), etc. The motion component 1030 includes an acceleration sensor component (e.g., an accelerometer), a gravity sensor component, and a rotation sensor component (e.g., a gyroscope).
[0112] Environmental component 1032 includes, for example, one or more cameras (with still imaging / video and video capture capabilities), an illumination sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers that detect ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an acoustic sensor component (e.g., one or more microphones that detect background noise), a proximity sensor component (e.g., an infrared sensor that detects nearby objects), a gas sensor (e.g., a gas detection sensor that detects concentrations of hazardous gases for safety or measures pollutants in the atmosphere), or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment.
[0113] With respect to cameras, the client device 102 may have a camera system including, for example, a front-facing camera on the front surface of the client device 102 and a rear-facing camera on the rear surface of the client device 102. The front-facing camera may, for example, be used to capture still images and videos (e.g., "selfies") of a user of the client device 102, which may then be enhanced using the enhancement data (e.g., filters) described above. In examples where the front-facing camera is used with the viewfinder ring flash described herein, the user has the ability to use augmented reality facial filters in low light conditions or even in complete darkness because the viewfinder ring flash illuminates the user's face without blurring the output of the digital image sensor. The rear-facing camera may, for example, be used to capture still images and videos in a more traditional camera mode, where these images are similarly enhanced using the enhancement data. In addition to the front-facing camera and the rear-facing camera, the client device 102 may also include a 360° camera for capturing 360° photos and videos.
[0114] Additionally, the camera system of the client device 102 may include dual rear cameras (e.g., a main camera and a depth sensing camera), or even three, four, or five rear camera configurations on the front and back sides of the client device 102. For example, these multi-camera systems may include a wide-angle camera, an ultra-wide-angle camera, a telephoto camera, a macro camera, and a depth sensor.
[0115] The location component 1034 includes a positioning sensor component (e.g., a GPS receiver component), an altitude sensor component (e.g., an altimeter or barometer that detects air pressure, from which the altitude can be obtained), an orientation sensor component (e.g., a magnetometer), and the like.
[0116] Various technologies may be used to achieve communication. The I / O components 1038 also include a communication component 1036 that is operable to couple the machine 1000 to the network 1020 or the device 1022 via a corresponding coupling or connection. For example, the communication component 1036 may include a network interface component or another suitable device for interfacing with the network 1020. In other examples, the communication component 1036 may include a wired communication component, a wireless communication component, a cellular communication component, a near field communication (NFC) component, a Bluetooth component (e.g., Bluetooth low energy), a Wi-Fi component, and other communication components that provide communication via other modalities. The device 1022 may be another machine or any of a variety of peripheral devices (e.g., a peripheral device coupled via USB).
[0117] In addition, the communication component 636 can detect the identifier or include a component operable to detect the identifier. For example, the communication component 636 can include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting a one-dimensional barcode such as a universal product code (UPC) barcode, a multi-dimensional barcode such as a quick response (QR) code, an Aztec code, a data matrix, a data symbol (Dataglyph), a maximum code (MaxiCode), PDF410, UltraCode, UCC RSS-2D barcode, and other optical codes) or an acoustic detection component (e.g., a microphone for identifying a tagged audio signal). In addition, various information can be obtained via the communication component 1036, such as a location obtained via Internet Protocol (IP) geolocation, a location obtained via Wi-Fi signal triangulation, a location obtained via detecting an NFC beacon signal that can indicate a specific location, etc.
[0118] Various memories (e.g., main memory 1012, static memory 1014, and memory of processor 1002) and storage unit 1016 may store one or more sets of instructions and data structures (e.g., software) implemented or used by any one or more of the methods or functions described herein. These instructions (e.g., instructions 1008) when executed by processor 1002 cause various operations to implement the disclosed examples.
[0119] The instructions 1008 may be sent or received over the network 1020 using a transmission medium via a network interface device (e.g., a network interface component included in the communication component 1036) and using any of several well-known transmission protocols (e.g., Hypertext Transfer Protocol (HTTP)). Similarly, the instructions 1008 may be sent or received via a coupling (e.g., a peer-to-peer coupling) with the device 1022 using a transmission medium.
[0120] Software Architecture
[0121] Fig.11 1100 is a block diagram illustrating a software architecture 1104 that can be installed on any one or more of the devices described herein. The software architecture 1104 is supported by hardware, such as a machine 1102, including a processor 1120, a memory 1126, and an I / O component 1138. In this example, the software architecture 1104 can be conceptualized as a stack of layers, where each layer provides specific functionality. The software architecture 1104 includes layers such as an operating system 1112, a library 1110, a framework 1108, and an application 1106. In operation, the application 1106 calls an API call 11110 through the software stack and receives a message 11112 in response to the API call 11110.
[0122] The operating system 1112 manages hardware resources and provides public services. The operating system 1112 includes, for example, a kernel 1114, services 1116, and drivers 1122. The kernel 1114 acts as an abstraction layer between the hardware layer and other software layers. For example, the kernel 1114 provides functions such as memory management, processor management (e.g., scheduling), component management, network and security settings. Services 1116 can provide other public services to other software layers. Drivers 1122 are responsible for controlling or connecting to the underlying hardware. For example, drivers 1122 may include display drivers, camera drivers, or Low-power drivers, Flash drivers, Serial communication drivers (e.g., USB drivers), Drivers, audio drivers, power management drivers, etc.
[0123] The library 1110 provides a common low-level infrastructure used by the application 1106. The library 1110 may include a system library 1118 (e.g., a C standard library) that provides functions such as memory allocation functions, string manipulation functions, mathematical functions, etc. In addition, the library 1110 may include an API library 1124, such as a media library (e.g., a library that supports the rendering and manipulation of various media formats such as Moving Picture Experts Group Group 4 (MPEG4), Advanced Video Coding (H.264 or AVC), Moving Picture Experts Group Layer 3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio codec, Joint Photographic Experts Group (JPEG or JPG), or Portable Network Graphics (PNG)), a graphics library (e.g., an OpenGL framework for rendering graphics content in two dimensions (2D) and three dimensions (3D) on a display), a database library (e.g., SQLite that provides various relational database functions), a network library (e.g., WebKit that provides network browsing functions), etc. The library 1110 may also include a variety of other libraries 1128 to provide many other APIs to the application 1106.
[0124] The framework 1108 provides a common high-level infrastructure used by the applications 1106. For example, the framework 1108 provides various graphical user interface (GUI) functions, high-level resource management, and high-level positioning services. The framework 1108 can provide a wide range of other APIs that can be used by the applications 1106, some of which can be specific to a particular operating system or platform.
[0125] In an example, applications 1106 may include a broad classification of home applications 1136, contact applications 1130, browser applications 1132, book reader applications 1134, location applications 1142, media applications 1144, messaging applications 1146, game applications 1148, and other applications such as third-party applications 1140. Applications 1106 are programs that perform functions defined in the program. Various programming languages may be used to create one or more applications 1106 constructed in various ways, such as object-oriented programming languages (e.g., Objective-C, Java, or C++) or procedural programming languages (e.g., C or assembly language). In a specific example, third-party applications 1140 (e.g., applications developed using an ANDROID or IOS software development kit (SDK) by an entity other than a supplier of a specific platform) may be mobile software running on a mobile operating system such as IOS, ANDROID, WINDOWS phone, or another mobile operating system. In this example, third party application 1140 can call API calls 11110 provided by operating system 1112 to facilitate the functionality described herein.
[0126] Glossary
[0127] "Carrier signal" refers to any intangible medium that can store, encode or carry instructions for execution by the machine and includes digital or analog communications signals or other intangible media to facilitate communication of these instructions. Instructions may be sent or received over a network via a network interface device using a transmission medium.
[0128] "Client Device" refers to any machine that interfaces with a communications network to obtain resources from one or more server systems or other client devices. A client device may be, but is not limited to, a mobile phone, a desktop computer, a laptop computer, a portable digital assistant (PDA), a smart phone, a tablet computer, an ultrabook, a netbook, a laptop computer, a multiprocessor system, a microprocessor-based or programmable consumer electronics product, a game console, a set-top box, or any other communications device that a user may use to access a network.
[0129] "Communications network" means one or more parts of a network, which may be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), the Internet, a part of the Internet, a part of the public switched telephone network (PSTN), a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, A network, other type of network, or a combination of two or more such networks. For example, the network or a portion of the network may include a wireless network or a cellular network, and the coupling may be a code division multiple access (CDMA) connection, a global system for mobile communications (GSM) connection, or other type of cellular or wireless coupling. In this example, the coupling may implement any of various types of data transmission technologies, such as single carrier radio transmission technology (1xRTT), evolution data optimized (EVDO) technology, general packet radio service (GPRS) technology, enhanced data rate for GSM evolution (EDGE) technology, the third generation partnership project (3GPP) including 3G, fourth generation wireless (4G) network, universal mobile telecommunications system (UMTS), high speed packet access (HSPA), world wide interoperability for microwave access (WiMAX), long term evolution (LTE) standard, other standards defined by various standard setting organizations, other long distance protocols, or other data transmission technologies.
[0130] "Component" refers to a logic, device or physical entity with boundaries defined by function or subroutine calls, branch points, APIs or other technologies that provide partitioning or modularization for specific processing or control functions. Components can be combined with other components via their interfaces to perform machine processing. Components can be packaged functional hardware units designed for use with other components and can be part of a program that generally performs specific functions in related functions. Components can constitute software components (e.g., codes implemented on machine-readable media) or hardware components. "Hardware components" are tangible units that can perform certain operations and can be configured or arranged in a certain physical manner. In various example embodiments, one or more computer systems (e.g., independent computer systems, client computer systems, or server computer systems) or one or more hardware components (e.g., processors or processor groups) of a computer system can be configured by software (e.g., applications or application parts) to operate to perform certain operations as described herein. Hardware components can also be implemented mechanically, electronically, or in any suitable combination thereof. For example, a hardware component may include a dedicated circuit or logic that is permanently configured to perform certain operations. The hardware component may be a dedicated processor, such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). The hardware component may also include a programmable logic or circuit that is temporarily configured to perform certain operations by software. For example, the hardware component may include software executed by a general-purpose processor or other programmable processor. Once configured by such software, the hardware component becomes a specific machine (or a specific component of a machine) that is uniquely customized to perform the configured function, and is no longer a general-purpose processor. It should be understood that the decision to implement the hardware component mechanically, in a dedicated and permanently configured circuit or in a temporarily configured (e.g., configured by software) circuit can be driven by cost and time considerations. Therefore, the phrase "hardware component" (or "hardware-implemented component") should be understood to include a tangible entity, that is, an entity that is physically constructed, permanently configured (e.g., hardwired) or temporarily configured (e.g., programmed) to operate in some way or perform certain operations described herein. Considering an embodiment in which the hardware component is temporarily configured (e.g., programmed), it is not necessary to configure or instantiate each hardware component in the hardware component at any time. For example, where a hardware component includes a general purpose processor that is configured by software to be a special purpose processor, the general purpose processor may be configured as different special purpose processors (e.g., including different hardware components) at different times. Thus, software configures one or more specific processors to, for example, constitute a specific hardware component at one time and to constitute different hardware components at different times. Hardware components may provide information to other hardware components and receive information from other hardware components. Thus, the described hardware components may be considered to be communicatively coupled.In the case of multiple hardware components being present at the same time, communication can be achieved by signal transmission (e.g., by appropriate circuits and buses) between or among two or more hardware components in the hardware components. In an embodiment in which multiple hardware components are configured or instantiated at different times, communication between such hardware components can be achieved, for example, by storing information in a memory structure accessed by multiple hardware components and retrieving information in the memory structure. For example, a hardware component can perform an operation, and the output of the operation is stored in a memory device to which it is communicatively coupled. Then, other hardware components can access the memory device at a subsequent time to retrieve the stored output and process it. The hardware component can also initiate communication with an input device or an output device, and can operate on resources (e.g., the collection of information). The various operations of the example methods described herein can be performed at least in part by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform related operations. Whether it is temporarily configured or permanently configured, such a processor can constitute a processor-implemented component that operates to perform one or more operations or functions described herein. As used herein, "processor-implemented components" refer to hardware components implemented using one or more processors. Similarly, the method described herein can be implemented at least in part by a processor, wherein one or more specific processors are examples of hardware. For example, at least some of the operations in the operation of the method can be performed by one or more processors or the parts implemented by the processor. In addition, one or more processors can also operate to support the execution of related operations in the "cloud computing" environment or operate as "software as a service" (SaaS). For example, at least some of the operations in the operation can be performed by a computer group (as an example of a machine including a processor), wherein these operations can be accessed via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., API). The execution of certain operations can be distributed between processors, not only resident in a single machine, but deployed across multiple machines. In some example embodiments, a processor or a part implemented by a processor can be located in a single geographic location (e.g., in a home environment, an office environment, or a server cluster). In other example embodiments, a processor or a part implemented by a processor can be distributed across multiple geographic locations.
[0131] "Computer-readable storage media" refers to both machine storage media and transmission media. Thus, the term includes both storage devices / media and carrier / modulated data signals. The terms "machine-readable medium," "computer-readable medium," and "device-readable medium" mean the same thing and may be used interchangeably in this disclosure.
[0132] “Machine storage media” refers to a single or multiple storage devices and media (e.g., centralized or distributed databases, and associated caches and servers) that store executable instructions, routines, and data. Thus, the term should be considered to include, but is not limited to, solid-state memory and optical and magnetic media, including memory internal or external to the processor. Specific examples of machine storage media, computer storage media, and device storage media include: non-volatile memory, including, for example, semiconductor memory devices such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), FPGA, and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The terms “machine storage media,” “device storage media,” and “computer storage media” mean the same thing and may be used interchangeably in this disclosure. The terms “machine storage media,” “computer storage media,” and “device storage media” expressly exclude carrier waves, modulated data signals, and other such media, at least some of which are encompassed by the term “signal media.”
[0133] “Non-transitory computer-readable storage medium” refers to tangible media capable of storing, encoding, or carrying instructions to be executed by a machine.
[0134] "Signal medium" refers to any intangible medium that is capable of storing, encoding or carrying instructions executed by a machine, and "signal medium" includes digital or analog communication signals or other intangible media to facilitate the transfer of software or data. The term "signal medium" should be deemed to include any form of modulated data signal, carrier wave, etc. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode the information in the signal. The terms "transmission medium" and "signal medium" mean the same thing and may be used interchangeably in this disclosure.
Claims
1. A hybrid search method, comprising: storing the media content items in a database; Access the text input associated with the character count; identifying each item from the media content items as being suitable, unsuitable, or indeterminate with respect to the textual input; applying text matching logic to the text input with respect to each item identified as uncertain to identify the item as suitable or unsuitable; generating personalized media content items by incorporating the text input into each of the media content items identified as suitable; as well as generating an item selection user interface, the item selection user interface comprising one or more items from the personalized media content items, in: Each item from the media content items is associated with a first condition, a second condition, and a third condition; and Identifying an item from the media content items as suitable comprises: determining that the character count is greater than or equal to a value associated with the first condition and less than or equal to a value associated with the second condition, wherein identifying an item from the media content items as unsuitable comprises: determining that the character count is greater than a value associated with the third condition, Wherein identifying an item from the media content items as indeterminate comprises determining that the character count is greater than a value associated with the second condition and less than a value associated with the third condition.
2. The hybrid search method according to claim 1, wherein: Applying the text matching logic includes evaluating corresponding grapheme features of individual characters in the text input.
3. The hybrid search method according to claim 1, wherein: Each item from the media content items is adapted to incorporate one or more lines of text and is associated with a line count and a set of line count text presentation conditions, each member of the set of line count text presentation conditions comprising a first condition, a second condition, and a third condition, the method comprising: generating a plurality of split options based on the text input, each option from the plurality of split options comprising the text input split into one or more lines; and In response to determining that at least one from the plurality of split options is suitable for incorporation into a first item from the media content items based on the set of line count text presentation conditions associated with the first item, the first item is identified as suitable.
4. The hybrid search method according to claim 3, comprising: In response to determining that no split options from the plurality of split options are suitable for incorporation into a second item from the media content items based on the set of line count text presentation conditions associated with the second item, identifying the second item as unsuitable.
5. The hybrid search method according to claim 3, comprising, for a third item from the media content items: determining that no split option from the plurality of split options matches with respect to a second condition from the set of line count text presentation conditions associated with the third item; determining a split option from the plurality of split options that matches a third condition from the set of line count text presentation conditions associated with the third item as an indeterminate option; evaluating corresponding glyph features of individual characters in the uncertain options; as well as Responsive to a result of the evaluation, the third item is identified as being suitable relative to the uncertain option.
6. The hybrid search method according to claim 1, comprising: causing the item selection user interface to be presented on a display device; receiving an indication of an item selected from the personalized media content items in the item selection user interface; as well as Include the selected item in the contextual user interface.
7. The hybrid search method according to claim 6, wherein: The contextual user interface is a chat message sending and receiving user interface or an image editing user interface.
8. A hybrid search system comprising: one or more processors; as well as A non-transitory computer-readable storage medium comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising: storing the media content items in a database; Access the text input associated with the character count; identifying each item from the media content items as being suitable, unsuitable, or indeterminate with respect to the textual input; applying text matching logic to the text input with respect to each item identified as uncertain to identify the item as suitable or unsuitable; generating personalized media content items by incorporating the text input into each of the media content items identified as suitable; and generating an item selection user interface, the item selection user interface comprising one or more items from the personalized media content items, in: Each item from the media content items is associated with a first condition, a second condition, and a third condition; and Identifying an item from the media content items as suitable comprises: determining that the character count is greater than or equal to a value associated with the first condition and less than or equal to a value associated with the second condition, wherein identifying an item from the media content items as unsuitable comprises: determining that the character count is greater than a value associated with the third condition, Wherein identifying an item from the media content items as indeterminate comprises determining that the character count is greater than a value associated with the second condition and less than a value associated with the third condition.
9. The hybrid search system according to claim 8, wherein: Applying the text matching logic includes evaluating corresponding grapheme features of individual characters in the text input.
10. The hybrid search system according to claim 8, wherein: Each item from the media content items is adapted to incorporate one or more lines of text and is associated with a line count and a set of line count text presentation conditions, each member of the set of line count text presentation conditions comprising a first condition, a second condition, and a third condition, the operations comprising: generating a plurality of split options based on the text input, each option from the plurality of split options comprising the text input split into one or more lines; and In response to determining that at least one from the plurality of split options is suitable for incorporation into a first item from the media content items based on the set of line count text presentation conditions associated with the first item, the first item is identified as suitable.
11. The hybrid search system according to claim 10, wherein the operations include: In response to determining that no split options from the plurality of split options are suitable for incorporation into a second item from the media content items based on the set of line count text presentation conditions associated with the second item, identifying the second item as unsuitable.
12. The hybrid search system of claim 10, wherein the operations include, for a third item from the media content items: determining that no split option from the plurality of split options matches with respect to the second condition from the set of line count text presentation conditions associated with the third item; determining a split option from the plurality of split options that matches a third condition from the set of line count text presentation conditions associated with the third item as an indeterminate option; evaluating corresponding glyph features of individual characters in the uncertain options; as well as Responsive to a result of the evaluation, the third item is identified as being suitable relative to the uncertain option.
13. The hybrid search system according to claim 8, wherein the operations include: causing the item selection user interface to be presented on a display device; receiving an indication of an item selected from the personalized media content items in the item selection user interface; as well as Include the selected item in the contextual user interface.
14. A machine-readable non-transitory storage medium having instruction data, the instruction data being executable by a machine to cause the machine to perform an operation, the operation comprising: storing the media content items in a database; Access the text input associated with the character count; identifying each item from the media content items as being suitable, unsuitable, or indeterminate with respect to the textual input; applying text matching logic to the text input with respect to each item identified as uncertain to identify the item as suitable or unsuitable; generating personalized media content items by incorporating the text input into each of the media content items identified as suitable; as well as generating an item selection user interface, the item selection user interface comprising one or more items from the personalized media content items, in: Each item from the media content items is associated with a first condition, a second condition, and a third condition; and Identifying an item from the media content items as suitable comprises: determining that the character count is greater than or equal to a value associated with the first condition and less than or equal to a value associated with the second condition, wherein identifying an item from the media content items as unsuitable comprises: determining that the character count is greater than a value associated with the third condition, Wherein identifying an item from the media content items as indeterminate comprises determining that the character count is greater than a value associated with the second condition and less than a value associated with the third condition.
