Intelligent overlay generation

Abstract

The invention provides intelligent overlay generation. Various systems, methods, and devices are described for an AI platform that may utilize a machine learning model configured to generate one or more overlays associated with received inputs. In an example, a system and method of generating one or more overlays may include receiving a media item and an input. The input may be natural language text or natural language audio associated with the user. A machine learning model may be used to determine a context associated with the input. Based on determining the context of the input, the machine learning model may generate one or more overlays. The user may select from the one or more overlays, indicating the one or more overlays to the user in conjunction with the media item. The combined media may be provided to the user, wherein the selected one or more overlays may be superimposed on the media item.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 585,533, filed on September 26, 2023, and entitled “Intelligent Sticker Generation,” and U.S. Non-Provisional Patent Application No. 18 / 889,083, filed on September 18, 2024, the entire contents of both applications are incorporated herein by reference. Technical Field

[0003] Examples of the present disclosure may generally relate to methods, apparatus, and computer program products for generating image overlays using artificial intelligence. Background Art

[0004] Electronic devices continue to change and evolve to provide users with flexibility and adaptability. As electronic devices become more versatile, users increasingly carry them with them in a variety of daily activities. This trend has caused many users to seek ways to express themselves. One such way is to capture and share images, videos, or audio recordings. While these methods allow users to record their environment, they may also limit the user's ability to fully express themselves. For example, users may wish to modify or interact with the media they have captured in order to further convey their unique perspectives. Summary of the invention

[0005] Disclosed herein is a method, apparatus, or system for generating an overlay associated with an electronic device based on input using an artificial intelligence (AI) platform. The AI ​​platform may be a social media platform or a messaging platform, or may be associated with a social media platform or a messaging platform. In some examples, the AI ​​platform may generate one or more overlays that may be added to a media item, such as an image, video, or graphics interchange format (GIF). The AI ​​platform may enable collaborative customization between one or more users, because one or more inputs associated with the one or more users may be received to generate one or more overlays of a media item.

[0006] In one example, a method, system, apparatus, and / or computer-readable medium may be provided for: receiving a media item; receiving an input via a device associated with a user; analyzing the context of the input using a machine learning model; generating one or more overlays associated with the context of the input based on using the machine learning model; receiving one or more selections associated with the one or more overlays; and generating combined media based on the media item and the one or more overlays. The combined media may be sent. For example, the combined media may be sent to a display of a device associated with the user.

[0007] Additional advantages will be partially described in the following description or may be learned through practice. These advantages will be realized and obtained through the elements and combinations specifically pointed out in the appended claims. It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and are not restrictive as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The Summary of the Invention and the Detailed Description below may be further understood when read in conjunction with the accompanying drawings. To illustrate the disclosed subject matter, exemplary embodiments of the disclosed subject matter are shown in the accompanying drawings; however, the disclosed subject matter is not limited to the specific methods, compositions, and apparatus disclosed. In addition, the drawings are not necessarily drawn to scale. In the drawings:

[0009] Figure 1 An example system according to an example of the present disclosure is presented.

[0010] Figure 2A An example method according to an example of the present disclosure is shown.

[0011] Figure 2B An example method according to an example of the present disclosure is shown.

[0012] Figure 3A , Figure 3B , Figure 3C and Figure 3D Example implementations of the disclosed subject matter of the present disclosure are shown.

[0013] Figure 4 Example machine learning and training models according to examples of the present disclosure are shown.

[0014] Figure 5 An example communication device according to examples of the present disclosure is shown.

[0015] Figure 6 An example computing system according to examples of the present disclosure is shown.

[0016] These drawings depict various embodiments for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods shown herein may be employed without departing from the principles described herein. DETAILED DESCRIPTION

[0017] Some embodiments of the present disclosure will now be described more fully below with reference to the accompanying drawings, in which some but not all embodiments of the present disclosure are shown. In fact, the various embodiments of the present disclosure can be implemented in many different forms and should not be construed as being limited to the embodiments set forth herein. The same reference numerals always refer to the same elements.

[0018] As referred to herein, media items or media may refer to content items that may be shared or published between one or more user devices or user profiles via a platform (e.g., a social media platform or a messaging platform, etc.). As examples, media may include, but are not limited to, images, videos, or stories, etc.

[0019] Electronic devices are constantly changing and evolving to provide users with flexibility and adaptability. As the adaptability of electronic devices increases, users may want to carry and maintain their electronic devices in various daily activities. This may cause many users to want to share their ideas and express themselves. Users may want to express themselves via messaging platforms or online platforms (e.g., social media platforms). Some studies have shown that the best online self-expression may rely on excellent visuals. Visual expressions can be deeply contextual, which may cause users to want more creative control over the media (e.g., text, images / photos, audio, graphics interchange format (GIF), videos, or stories, etc.) that users use to express themselves.

[0020] In some systems, visual self-expression in a messaging platform or online platform may be limited to predefined media, such as static images, pre-generated stickers, videos and / or audio, etc., which may become obsolete over time. In some examples, the platform can provide users with methods or systems for adding some personality traits and self-expression to images, GIFs, videos or stories. One such example that one or more platforms can utilize is stickers. The method for applying stickers can be to utilize a pre-generated sticker database, which is created and does not change based on any user input. The method may minimize the possibility of contextualization or creative control of media. The system may also exacerbate some problems that users may face when expressing themselves through the collected media. Studies have shown that many users may lack the comfort of sharing or expressing themselves or their creativity through the collected media items (these media items can be shared via the platform or third-party applications). Therefore, creating a way to customize its image for users can allow users to better express themselves, which can enable users to have a richer and more attractive experience to the platform and interact with other users over time.

[0021] The present disclosure generally relates to systems and methods for an artificial intelligence platform in which artificial intelligence (AI) can be used to modify or generate overlays (e.g., images or maps, etc.) associated with one or more portions of a received media item. Examples of the present disclosure may include using an artificial intelligence (AI) platform that uses machine learning to generate one or more overlays to capture a user's imagination and self-expression. In one example, the AI ​​platform may include generating one or more overlays based on received input, adding or removing previously generated overlays to or from a combined media, replacing one or more overlays associated with the combined media, adjusting one or more overlays associated with the combined media, or constructing ideas or illustrating a user's self-expression of the media, etc.

[0022] In one example, the AI ​​platform can implement collaborative customization between one or more users associated with the AI ​​platform. For example, a first user on a first device can create an initial customized media item including a media item and an initially generated overlay, and share the initial customized media item with a second user associated with a second user device. Then, the second user can provide their own input to the initial customized media item to further customize the media item. The server can generate new overlay suggestions based on the input associated with the second user, thereby allowing users (e.g., the first user and the second user) to iteratively enhance the shared media item. The collaborative customization feature of the AI ​​platform can enable new forms of creative interaction through shared media items between one or more users of a plurality of users. Collaborative customization can enhance user engagement on the AI ​​platform 110 (e.g., a social media platform or a messaging platform, etc.).

[0023] Figure 1 An example smart overlay system 100 according to an example of the present disclosure is shown. The smart overlay system 100 may host an AI platform 110 (e.g., a social media platform or a messaging platform, etc.). The smart overlay system 100 may be able to facilitate communication between users or provide content between users. The smart overlay system 100 may include a device 101, a device 102, a device 103, a server 107, a data repository 108, and an AI platform 110. As shown, for simplicity, the AI ​​platform 110 may be located on the server 107. It is contemplated that the AI ​​platform 110 may be located on or interact with one or more devices of the smart overlay system 100.

[0024] In a specific example, devices 101, 102, and 103 may be associated with individuals (e.g., users) that interact or communicate with AI platform 110. AI platform 110 may be considered a messaging platform (or social media platform). In a specific example, one or more users may use one or more devices (e.g., devices 101, 102, 103) to access, send data to, or receive data from AI platform 110, which may be located on server 107 or devices (e.g., devices 101, 102, 103), etc.

[0025] The present disclosure contemplates any suitable network 105. By way of example and not limitation, one or more portions of network 105 may include an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), or any combination thereof. Network 105 may include one or more wired links or wireless links to facilitate communication between devices of system 100. Network 105 may include one or more networks 105.

[0026] In a specific example, the device 101, 102, 103 may be an electronic device that includes hardware, software, or embedded logic components or a combination of two or more such components and is capable of performing appropriate functions implemented or supported by the device 101, 102, 103. As an example and not limitation, the device 101, 102, 103 may be a computer system, such as a desktop computer, a notebook or laptop computer, a netbook, a tablet computer (e.g., a smart tablet), an e-book reader, a global positioning system (GPS) device, a personal digital assistant (PDA), a handheld electronic device, a cellular phone, a smart phone, an augmented reality / virtual reality device, other suitable electronic devices, or any suitable combination thereof. The present disclosure contemplates any suitable device (e.g., device 101, 102, 103). The device 101, 102, 103 may enable a user to access the network 105. The device 101, 102, 103 may enable one or more users to communicate with other users at another device 101, 102, 103.

[0027] The AI ​​platform 110 may be a network-addressable computing system that may host an online messaging network or a social media network. The AI ​​platform 110 may generate, store, receive, or send information associated with a user, such as user profile data or other suitable data related to the AI ​​platform 110. One or more components of the smart overlay system 100 may access the AI ​​platform 110 directly or via the network 105. By way of example and not limitation, the device 101 may access the AI ​​platform 110 located on the server 107 directly or via the network 105 by using a web browser or a local application on the device 101 associated with the AI ​​platform 110 (e.g., a messaging application, a social media application, another suitable application, or any combination thereof).

[0028] The intelligent overlay system 100 may include one or more servers (e.g., server 107). Each server 107 may be a single server, or a distributed server across multiple computers or multiple data centers. The server 107 may be of various types, such as but not limited to a web server, a news server, a mail server, a message server, an advertising server, a file server, an application server, an exchange server, a database server, a proxy server, another server suitable for performing the functions or processes described herein, or any combination thereof. Each server 107 may include hardware, software, or embedded logic components, or a combination of two or more of these components, for performing the appropriate functions implemented or supported by the server 107. The intelligent overlay system 100 may include one or more data repositories (e.g., data repository 108). The data repository 108 may be used to store various types of information. The information stored in the data repository 108 may be organized according to a specific data structure. Each data repository 108 may be a relational database, a columnar database, an associative database, or other suitable database. Although the present disclosure describes or illustrates a specific type of database, the present disclosure contemplates any suitable type of database. Certain examples may provide a plurality of interfaces that enable a device 101 , 102 , 103 or another system (eg, a third party system) to manage, retrieve, modify, add, or delete information stored in the data repository 108 .

[0029] Users of AI platform 110 may have the ability to communicate and interact with other users. In a particular example, a user associated with a particular device (e.g., device 101) may join AI platform 110 and then add connections (e.g., relationships) to multiple other users (e.g., devices 102, 103) that constitute contacts of AI platform 110 with whom they want to communicate. In some examples, user connections or communications may be monitored for machine learning purposes. In an example, server 107 of AI platform 110 may receive, record, or otherwise obtain information associated with a communication or connection with a user (e.g., device 101, device 102, or device 103). In this document, the term contact (e.g., a known user, colleague, or friend club) may refer to any other user of AI platform 110 in which there is an indication of a connection or relationship.

[0030] The AI ​​platform 110 may provide users with the ability to take actions on various types of items. By way of example and not limitation, these items may include groups to which a user may belong, message boards that a user may be interested in, question forums, messages between one or more users, interactions with images, stories, videos, comments under posts, or other suitable items. Users may interact with anything that can be represented in the AI ​​platform 110. In a specific example, the AI ​​platform 110 may be able to link various users. By way of example and not limitation, the AI ​​platform 110 may enable users to interact with each other and receive content (e.g., media or text, etc. or any combination thereof) from their respective groups or contacts, where the group may refer to a selected plurality of users who are communicating or interacting with each other through an application programming interface (API) or other communication channel.

[0031] Figure 2A A method 120 of generating (e.g., creating) one or more overlays according to an example of the present disclosure is shown. The method 120 may be performed by a device (e.g., communication device 30, computing system 600, device 101, device 102, or device 103). For example, device 101 may perform method 120 in part using one or more of the following: a processor (e.g., processor 32, processor 91), a memory (e.g., non-removable memory 44, removable memory 46, read-only memory (ROM) 93, random access memory (RAM) 82), and / or a storage controller (e.g., storage controller 92).

[0032] At step 121, a media item (e.g., an image, GIF, story, or video, etc.) may be received. In some examples, a graphical user interface associated with device 101 may receive an instruction to upload or otherwise obtain a media item. The media item may be captured by a camera of device 101 or selected from existing content stored in device 101 or data repository 108 (e.g., cloud storage).

[0033] At step 122, input may be received via a device 101 associated with the user. The input may be used as a basis for generating a custom overlay added to the media item. The input may include one or more of natural language (e.g., text) or audio, etc. For example, the device 101 may receive a text or audio description, such as "cute bananas with heart-shaped eyes" in audio or text, to request a specific type of overlay.

[0034] In step 123, the machine learning model (e.g., Figure 4The machine learning model 410 can analyze the input or media item through AI. The machine learning model 410 can determine the context associated with the received input or media item through a nearest neighbor algorithm (e.g., nearest neighbor application or nearest neighbor technology) by analyzing the semantic meaning of the input and in the training data (e.g., Figure 4 The context associated with the received input or media item is determined by finding the most similar context in the training data 420). This can be performed by a process called contextualized embedding, in which the input is embedded in a high-dimensional vector space, allowing the machine learning model 410 to obtain the nuances in the meaning associated with the received input. Then, the nearest neighbor algorithm can search for the most similar vectors in the training data 420, which can represent one or more contexts that are closest in meaning to the received input. The adjacent context is used to inform the model's understanding of the input, so that the machine learning model 410 can accurately provide the relevant context associated with the received input. It can be expected that the nearest neighbor algorithm can use a variety of different mathematical techniques to search for nearest neighbors, such as but not limited to cosine similarity, Euclidean distance, or graph-based methods, depending on the specific model architecture and training data. The first result (e.g., context) can be further used in the machine learning model 410 or one or more other machine learning models 410. For example, a user can provide input such as "a cute banana with heart-shaped eyes". AI platform 110 may determine through machine learning model 410 that the context of the received input may be associated with a banana including or having heart-shaped eyes. In some examples, the input may be analyzed in conjunction with or in addition to a media item, or a conversation between one or more users, or one or more media items associated with a conversation between one or more users (e.g., a group chat, etc.). In one example, a media item may be, for example, Figure 3D As shown, the image may show a person's heart-shaped fingers, which may be used as a contextual indication of a sticker to be displayed (e.g., a heart-shaped sticker or a sticker of similar format). Thus, in response to the analysis, considering the text input of the media item "cute banana" may cause the system to provide similar one or more overlays of a banana with heart-shaped eyes, such as Figure 3C shown.

[0035] In step 124, a machine learning model 410 (e.g., machine learning model 410) is used to consult a database (e.g., memory 44 or data repository 108) to generate one or more overlays associated with the input context (e.g., the first result). The machine learning model 410 can be configured to utilize the context to obtain (e.g., search) the nearest neighbor image-text pairs through the data repository 108 to generate one or more overlays, wherein these image-text pairs can be associated with the training data 420. The machine learning model 410 can utilize a nearest neighbor algorithm, wherein the machine learning model 410 can represent each image-text pair as a joint embedding. The joint embedding can capture both the visual features of the image and the semantic meaning of the text. In some examples, the joint embedding can be obtained by a deep learning architecture that combines a neural network (e.g., a convolutional neural network, etc.) for image processing and a large language model based on a transformer for text processing. Then, the nearest neighbor algorithm can search for multiple most similar image-text pairs for the context associated with step 123.

[0036] A plurality of most similar image-text pairs may be determined by any suitable mathematical method (e.g., but not limited to, cosine similarity or dot product) to compare the embedding (e.g., context) of the received input with the embeddings of all image-text pairs in the data repository 108. In some examples, a similarity metric may be used to measure the degree of match between the context associated with the received input and the meaning of each image-text pair. Based on the similarity metric, the machine learning model 410 may select the top M image-text pairs with the highest similarity scores, where 'M' may be any suitable number determined by the AI ​​platform 110, for example, M may be 10 image-text pairs. The top M image-text pairs may be selected and considered as the nearest neighbors of the context associated with step 123, and visual features associated with the top M image-text pairs may be utilized to facilitate the generation of one or more overlays that may be associated with the context of the received input. The machine learning model 410 may generate one or more overlays that may be provided via a graphical user interface associated with the device 101.

[0037] Continuing to refer to step 124, in one example, the AI ​​platform 110 can determine, through the machine learning model 410, an overlay associated with a banana with heart-shaped eyes based on the input "cute banana with heart-shaped eyes". Therefore, the machine learning model 410 can generate one or more overlays via a graphical user interface associated with the device 101. The one or more overlays generated can be associated with a banana with heart-shaped eyes (e.g., including one or more stickers of a banana with heart-shaped eyes). One of the one or more overlays can include a different rendition of the received input. In some examples, one or more overlays can be generated based on an analysis of the following items in combination or addition: a conversation between one or more users; or one or more media items associated with a conversation between one or more users (e.g., a group chat, etc.). In some examples, one or more overlays can be stored in a memory (e.g., non-removable memory 44, removable memory 46, ROM 93, or RAM 82) or a database (e.g., a data repository 108) for future use. In such an example, one of the one or more overlays may be saved in memory 44, where the one of the one or more overlays may be selected (e.g., determined) to be saved, or the overlay may have been used a number of times exceeding a selected threshold. The selected threshold may be determined by the AI ​​platform 110 or a setting associated with the device 101. It is contemplated that in some examples, a common (e.g., previously presented) context is associated with the input, and the machine learning model 410 may consult the memory 44 or data repository 108 to provide one or more previously stored overlays. As disclosed herein, one or more overlays may include one or more stickers—graphic elements that can be added to the original media item. The smart overlay system 100 may access the data repository 108 or other memory 44 to show items associated with the input (e.g., prompt), thereby potentially creating multiple overlay options that match the description.

[0038] At step 125, one or more selections associated with the one or more overlays may be received. In one example, a device 101 that may be associated with an AI platform 110 (e.g., a messaging platform and / or a social media platform, etc.) may receive one or more selections associated with one or more overlays. The one or more selections may indicate a first overlay in the one or more overlays to be used with the media item received in step 121. In some examples, in response to one or more selections of the one or more overlays, a representation may be performed on the device 101 to indicate that one or more selections of the one or more overlays have been selected. The representation may be provided via a graphical user interface of the device 101 or any other suitable component. The representation may include one or more of tactile feedback or highlighting of one of the one or more overlays, etc., to express that one of the one or more overlays has been selected.

[0039] In step 126, the device 101 may generate a combined media through a graphical user interface. The combined media may include one or more selections of one or more overlays superimposed on the media item received in step 121. It is contemplated that the selected one or more overlays (e.g., one or more selections of one or more overlays) superimposed on the media item may not overwrite or replace any pixel information or media data associated with the media item. In some examples, the selected one or more overlays may be adjusted relative to the media item. Adjusting the selected one or more overlays relative to the media item may refer to, but is not limited to, performing one or more of the following on the selected one or more overlays: zooming in; moving (e.g., left, right, up or down, or any combination thereof); reducing; rotating; flipping; reversing; or hiding a portion of the selected one or more overlays, etc.; or any combination thereof. It is contemplated that the adjustment of the selected one or more overlays may be performed relative to the media item by the device 101 or any suitable movement of the cursor associated with the device 101 in conjunction with the graphical user interface associated with the device 101. For example, by moving a finger associated with the user from right to left on the surface of the graphical user interface, one of the one or more selections may be moved from right to left.

[0040] At step 127, the combined media may be sent. The combined media may be sent to one or more user profiles associated with AI platform 110 (e.g., a social media platform) or device 102 via one or more graphical user interfaces associated with one or more users. In some examples, the combined media may be sent to one or more users (e.g., device 102 or AI platform 110) based on pressing a share button, wherein the share button may share the combined media on AI platform 110 via an image, a post, a video, or a story, or any combination thereof. The share button may also be configured as a send button between one or more users (e.g., a conversation between two or more users). In some examples, device 101 may send the combined media to a group of users (e.g., multiple users in a group chat). In some other examples, Figure 2A The steps need not be performed iteratively or simultaneously, and may be performed in any suitable manner that need not be performed sequentially. In some examples, method 120 may also include feedback selection, wherein the feedback selection may be used to help train machine learning model 410. The feedback selection may be configured to allow a user to agree or disagree with one or more overlays generated. The disclosed method may be performed on one device (e.g., device 101) or on multiple devices.

[0041] Figure 2B Method 10 of intelligent overlay generation according to an example of the present disclosure is shown. Device 101 can perform method 10. In step 11, the first device 101 associated with the first user can receive a media item (e.g., an image, a GIF, a story, or a video, etc.). In step 12, the first device 101 can receive a first input. The first input can include one or more of natural language (e.g., text) or audio, etc. In step 13, the first device 101 can generate a media item based on the machine learning model 410 (e.g., Figure 4 The first input may be analyzed using a machine learning model 410. Similar to step 123, the machine learning model 410 may determine a first context associated with the received first input.

[0042] At step 14, the first device 101 can utilize the machine learning model 410 to consult the data repository 108 (e.g., a memory or a database) to generate one or more first overlays associated with the first context (e.g., the first result). The one or more overlays generated can be associated with a banana with heart-shaped eyes (e.g., including one or more stickers of a banana with heart-shaped eyes). A first overlay in the one or more first overlays can include a different rendition of the received first input. Step 14 can be performed similarly to step 124.

[0043] At step 15, one or more first selections associated with the one or more first overlays may be received. In one example, a first device 101 that may be associated with an AI platform 110 (e.g., a messaging platform and / or a social media platform, etc.) may receive one or more first selections associated with the one or more first overlays. The one or more first selections may indicate one or more of the one or more overlays associated with the first user for use with the media item received at step 11. Step 15 may be similar to Figure 2A In step 16, the first device 101 may generate a first combined media. The first combined media may include one or more first selections superimposed on the media item received in step 11. Step 16 may be similar to Figure 2A Step 126 is performed.

[0044] At step 17, first device 101 may facilitate sending the first combined media to the second user via a graphical user interface associated with a second device (e.g., device 102). In some examples, the first combined media may be sent to the second device 102 based on pressing a share button, wherein the share button may share the first combined media on the AI ​​platform 110 via an image, a post, a video, or a story, or any combination thereof. The share button may also be configured as a send button between a first user and a second user (e.g., a conversation between two or more users) of a plurality of users associated with the AI ​​platform 110. In some examples, the first user (e.g., first device 101) may send the first combined media to the AI ​​platform 110, in which case, as a result, the AI ​​platform 110 may send the first combined media to the second device 102 associated with the second user.

[0045] In step 18, a second input may be received. The second input may include one or more of natural language (e.g., text) or audio, etc. The second input may be received on the AI ​​platform 110 or device 102 associated with the second user. In step 19, the second input may be analyzed based on the machine learning model 410. Similar to step 13, the machine learning model 410 may determine a second context associated with the received second input. Similar to what is described in step 13, the second context associated with the received second input may be determined by a nearest neighbor algorithm (e.g., a nearest neighbor application or nearest neighbor technique) by analyzing the semantic meaning of the second input and finding the most similar second context in the training data 420. The second context associated with the second input (e.g., a second result) may be further used in the machine learning model 410 or one or more other machine learning models 410. For example, the second device 102 may provide a second input, such as "anime character kicks". The AI ​​platform 110 may determine through the machine learning model 410 that the second context of the received second input may be associated with an anime character performing a kick.

[0046] At step 20, the second device 102 (machine learning model 410) can be used to consult the data repository 108 (e.g., memory 44 or database) to generate one or more second overlays associated with the second context (e.g., the second result). The machine learning model 410 can be configured to utilize the second context to obtain the nearest neighbor image-text pairs through the data repository 108 to generate one or more second overlays, wherein these image-text pairs can be associated with the training data 420. The machine learning model 410 can utilize a nearest neighbor algorithm similar to the algorithm described in step 14. As described in step 14, based on the similarity metric, the machine learning model 410 can select the top M image-text pairs with the highest similarity scores, wherein 'M' can be any suitable number determined by the AI ​​platform 110, for example, M can be 10 image-text pairs. The top M image-text pairs can be selected and regarded as the nearest neighbors of the second context associated with step 19, and the visual features associated with the top M image-text pairs can be utilized to facilitate the generation of one or more second overlays that can be associated with the second context of the second input received at step 18. The machine learning model 410 can generate one or more second overlays that can be provided via a graphical user interface associated with the second device 102. For example, the second user can provide a second input, such as "anime character kicking". The AI ​​platform 110 can determine through the machine learning model 410 that the second user may want an overlay associated with the anime character kicking. Therefore, the machine learning model 410 can provide one or more second overlays via a graphical user interface associated with the second device 102. The one or more second overlays generated can be associated with the anime character kicking (e.g., including one or more stickers of the anime character kicking). One or more of the one or more second overlays can include different renditions of the received second input.

[0047] At step 21, one or more second selections associated with the one or more second overlays may be received. The one or more second selections may indicate one or more of the one or more second overlays associated with the second user for use with the received combined media item associated with step 17. In some examples, in response to the one or more second selections of one or more of the one or more second overlays, a representation may be provided to the second device 102 that may indicate that one or more of the second overlays (e.g., the one or more second selections) have been selected. The representation may be provided via a graphical user interface associated with the second device 102 or any other suitable component of the device (e.g., the communication device 30).

[0048] In step 22, the second device 102 may display the second combined media. The second combined media may include one or more second selections and one or more first selections superimposed on the media item. It is contemplated that the one or more second selections superimposed on the first combined media (e.g., the selected one or more second overlays) may not overwrite or replace any pixel information, media data, or overlays associated with the first combined media. In some examples, the one or more second selections may be adjusted relative to the first combined media. Adjusting the one or more second selections (e.g., the selected one or more second overlays) relative to the first combined media may refer to, but is not limited to, performing one or more of the following on the selected one or more second overlays: zooming in; moving (e.g., left, right, up, or down, or any combination thereof); zooming out; rotating; flipping; reversing; or hiding a portion of the selected one or more second overlays, etc. In an alternative example, the second user may be able to adjust one or more first selections associated with the first user or one or more second selections associated with the second user. It is contemplated that the one or more second selections (e.g., the selected one or more second overlays) may be adjusted relative to the first combined media by any suitable movement associated with the second device 102. For example, by moving a finger associated with the second user from right to left on the surface of the graphical user interface, a second selection of the one or more second selections may be moved from right to left.

[0049] In step 23, the second device 102 may facilitate sending the second combined media to the first user via a graphical user interface associated with the first user (e.g., the first device 101). The second combined media may include one or more second selections superimposed on the first combined media. In some examples, the second combined media may be shared on the platform 110 through images, posts, videos, or stories, etc., or any combination thereof. There may be a share button that is configured as a send button between the first user and the second user (e.g., a conversation between two or more devices 101, 102, 103). In some examples, the second user (e.g., the second device 102) may facilitate sending the second combined media to a graphical user interface associated with the first user (e.g., the first device 101).

[0050] It can be expected that Figure 2BThe steps need not be performed iteratively or simultaneously, and may be performed in any suitable manner that does not necessarily need to be performed sequentially. In some examples, method 10 may also include feedback selection, wherein the feedback selection may be used to help train the machine learning model 410. The feedback selection may be configured to allow the first user and the second user to agree or disagree with the generated one or more overlays. In some examples, one or more first overlays and one or more second overlays may be generated based on an analysis in combination with or in addition to: a conversation between the first user and the second user; or one or more media items associated with the conversation between the first user and the second user.

[0051] It can be expected that although Figure 2A and Figure 2B The machine learning model 410 may be discussed as a single machine learning model 410, but Figure 2A and Figure 2B The steps and methods may be performed by one or more machine learning models 410. It is contemplated that, as disclosed, the machine learning model 410 may be one or more language models. In an example, the machine learning model 410 may be any suitable large language model.

[0052] Reference Figure 3A , an example graphical user interface 300 may be shown. In an example, graphical user interface 300 may refer to a login page associated with AI platform 110. Graphical user interface 300 may include one or more example overlays. Figure 3B There may be shown input 302 associated with graphical user interface 300. Input 302 as shown may be natural language text. Figure 3C One or more example overlays 303 generated based on input 302 via machine learning may be shown. Figure 3D Combined media 304 may be shown. The combined media may include one or more overlays superimposed on the received media item in the manner as described in step 121 .

[0053] Figure 4 An example framework 400 that can be used by a system 100 associated with machine learning is shown. The framework 400 can be hosted remotely. Alternatively, the framework 400 can be located at a Figure 1The AI ​​platform 110 shown in the figure may be processed by a device (e.g., computing system 600, communication device 30, device 101, device 102, or device 103). The machine learning model 410 may be operably coupled to the training data 420 stored in the training database 403 (e.g., data repository 108). In some examples, the machine learning model 410 may be associated with other operations. The machine learning model 410 may implement one or more machine learning models 410 (e.g., the machine learning model 410 associated with step 205, or the machine learning model 410 associated with step 210) and / or another device (e.g., server 107, device 101, device 102, or device 103).

[0054] In another example, the training data 420 may include properties of thousands of objects. For example, the objects may be smartphones, people, books, newspapers, news articles, signs, cars, audio, images, movies, TV shows, other videos, and other items. Attributes may include, but are not limited to, the size, shape, orientation, and position of the object. The training data 420 used by the machine learning model 410 may be fixed or periodically updated (e.g., by the computing system 600, the communication device 30). Alternatively, the training data 420 may be updated in real time based on the evaluation performed by the machine learning model 410 in a non-training mode. This is shown by the bidirectional arrow connecting the machine learning model 410 and the stored training data 420.

[0055] The machine learning model 410 can be designed to determine the context associated with the received input. The context can include semantic meanings associated with the received input. The machine learning model 410 can be designed to find vector embeddings similar to the context associated with the received input, as well as image-text pair embeddings. The machine learning model 410 can be a large language model that generates a representation (e.g., vector space) or embedding of natural language or visual image data. These machine learning models 410 can be trained (e.g., pre-training and / or real-time training) on ​​a large amount of text data and / or data collection of various language patterns and semantics. The machine learning model 410 can understand and represent the context of words, terms, and / or phrases in a high-dimensional space, thereby effectively collecting / determining semantic similarities between different words, contexts, and situations, even when they are not exactly the same. For example, an input associated with "big wheel bicycle" can be associated with another input that has been previously determined (e.g., "big wheel motorcycle").

[0056] The machine learning model 410 may be designed to develop and predict associations between one or more images and one or more semantic meanings of text. In one example, the machine learning model 410 may utilize the training data 420 to develop and predict associations between one or more of the following: user interactions with media associated with the AI ​​platform 110, previously selected overlays, context associated with a conversation, context of input, media or image data, etc. The training data 420 may be historical data, or data associated with one or more media, inputs, previous inputs, contextual data, or user interactions with media on the AI ​​platform 110, etc. The training data 420 may also include user profile data, wherein the user profile data may include one or more of the following: previous inputs associated with one of the one or more users of the AI ​​platform 110 or associated with a particular “style” associated with the user, previously used overlays, or user media interaction data associated with the AI ​​platform 110, etc. In this example, the training data 420 may include associations between user profile data (e.g., style). The training data 420 may also include user interactions with content such as, but not limited to, captions, tags, or likes to help form image-text pairs. In some examples, user interactions with content items associated with the AI ​​platform 110 may provide information to the machine learning model 410 to generate one or more overlays. As an example, one or more overlays that may be shared frequently may manipulate the machine learning model 410 to generate similar overlays. Therefore, the training data 420 may also include sharing rates associated with one or more previously generated overlays.

[0057] In some examples, the machine learning model 410 can implement a neural network. The neural network can help utilize the training data 420 or help the machine learning technology to analyze the context associated with the input, and pair the context with an image-text pair that can be associated with historical data. In one example, the neural network can be trained based on historical data indicating the joint embedding between the visual features of one or more images and one or more semantic meanings of the text. The historical data may include, but is not limited to, books, movies, news articles, magazines, television programs, previous overlay selections by users or other users, or previous inputs, etc. The neural network can be configured to process one or more images using conventional architectures such as, but not limited to, residual neural networks. In some examples, the neural network can be modified to adapt to specific image processing requirements. The neural network can include a language model based on a converter for processing text, which can be modified to adapt to specific text processing requirements associated with the AI ​​platform. Text embedding can be used as a condition for an image generation model (e.g., an image generation model based on diffusion), which can enable the machine learning model 410 to collect semantic relationships between text and image pairs.

[0058] In operation, the machine learning model 410 can evaluate the association between one or more inputs and the overlay. For example, the input (e.g., search, interaction with a content item, etc.) can be compared with corresponding attributes of the stored training data 420 (e.g., pre-stored objects).

[0059] In operation, the machine learning model 410 can evaluate the properties of media (e.g., text, images, video, or audio, etc.) obtained by hardware (e.g., devices 101, 102, 103). For example, the properties of the extracted media (e.g., features from one or more images, one or more videos, one or more reels, one or more posts, stories, and / or text, etc.) can be compared with corresponding properties of the stored training data 420 (e.g., pre-stored objects).

[0060] Figure 5 A block diagram of an example hardware / software architecture of a user equipment (UE) 30 is shown. Figure 5 As shown, UE 30 (also referred to herein as node 30) may include: processor 32; non-removable memory 44; removable memory 46; speaker / microphone 38; keypad 40; display, touchpad and / or indicator 42; power supply 48; global positioning system (GPS) chipset 50; and other peripherals 52. UE 30 may also include camera 54. In an example, camera 54 is a smart camera that is configured to sense images that appear within one or more bounding boxes. UE 30 may also include communication circuitry, such as transceiver 34 and transmit / receive element 36. It will be appreciated that UE 30 may include any sub-combination of the aforementioned elements while remaining consistent with the example.

[0061] The processor 32 may be a dedicated processor, a digital signal processor (DSP), a plurality of microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), and a state machine, etc. In general, the processor 32 may execute computer executable instructions stored in a memory (e.g., a non-removable memory 44 and / or a removable memory 46) of the node 30 to perform various required functions of the node. For example, the processor 32 may perform signal encoding, data processing, power control, input / output processing, and / or any other function that enables the node 30 to operate in a wireless environment or a wired environment. The processor 32 may run an application layer program (e.g., a browser) and / or a radio access layer (RAN) program and / or other communication programs. For example, the processor 32 may also perform security operations such as authentication, security key negotiation, and / or encryption operations, for example, at the access layer and / or the application layer.

[0062] Processor 32 is coupled to its communication circuitry (eg, transceiver 34 and transmit / receive element 36). Processor 32 may control the communication circuitry by executing computer executable instructions to enable node 30 to communicate with other nodes via communication network 28 to which it is connected.

[0063] The send / receive element 36 can be configured to send signals to other nodes or network devices or receive signals from other nodes or network devices. For example, in one example, the send / receive element 36 can be an antenna configured to send and / or receive radio frequency (RF) signals. The send / receive element 36 can support various networks and air interfaces, such as wireless local area network (WLAN), wireless personal area network (WPAN) and cellular, etc. In another example, the send / receive element 36 can be configured to send and receive both RF signals and optical signals. It will be appreciated that the send / receive element 36 can be configured to send and / or receive any combination of wireless signals or wired signals.

[0064] The transceiver 34 may be configured to modulate signals to be transmitted by the transmit / receive element 36 and to demodulate signals received by the transmit / receive element 36. As described above, the node 30 may have multi-mode functionality. Thus, for example, the transceiver 34 may include multiple transceivers for enabling the node 30 to communicate via multiple radio access technologies (RATs), such as universal terrestrial radio access (UTRA) and Institute of Electrical and Electronics Engineer (IEEE 802.11).

[0065] The processor 32 can access information from any type of suitable memory, such as non-removable memory 44 and / or removable memory 46, and store data therein. For example, as described above, the processor 32 can store session contexts in its memory. The non-removable memory 44 can include RAM, ROM, a hard disk, or any other type of memory storage device. The removable memory 46 can include a subscriber identity module (SIM) card, a memory stick, and a secure digital (SD) memory card, etc. In other examples, the processor 32 can access information from a memory that is not physically located on the node 30 (e.g., located on a server or a home computer) and store data in the memory.

[0066] The processor 32 may receive power from the power source 48 and may be configured to distribute power to and / or control power to other components in the node 30. The power source 48 may be any suitable device for powering the node 30. For example, the power source 48 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0067] Processor 32 may also be coupled to a GPS chipset 50, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of node 30. It will be appreciated that node 30 may acquire location information by any suitable location determination method while remaining consistent with the example.

[0068] Figure 6is a block diagram of an exemplary computing system 600. In some exemplary embodiments, the network device 110 may be the computing system 600. The computing system 600 may include a computer or a server and may be primarily controlled by computer-readable instructions, which may be in the form of software, regardless of where such software is stored or accessed, or in any manner. Such computer-readable instructions may be executed within a processor (e.g., a central processing unit (CPU) 91) to cause the computing system 600 to operate. In many workstations, servers, and personal computers, the central processing unit 91 may be implemented by a single-chip CPU called a microprocessor. In other machines, the central processing unit 91 may include multiple processors. The coprocessor 81 may be an optional processor different from the main CPU 91 that performs additional functions or assists the CPU 91.

[0069] In operation, the CPU 91 fetches instructions, decodes and executes them, and transmits information to and receives information from other resources via the computer's main data transmission path (system bus 80). Such a system bus connects the components in the computing system 600 and defines the medium for data exchange. The system bus 80 typically includes data lines for sending data, address lines for sending addresses, and control lines for sending interrupts and operating the system bus. An example of such a system bus 80 is a Peripheral Component Interconnect (PCI) bus.

[0070] The memory coupled to the system bus 80 includes RAM 82 and ROM 93. Such memory may include circuits that allow information to be stored and retrieved. ROM 93 generally contains stored data that is not easily modified. The data stored in RAM 82 can be read or changed by CPU 91 or other hardware devices. Access to RAM 82 and / or ROM 93 can be controlled by a memory controller 92. The memory controller 92 can provide an address translation function that converts a virtual address into a physical address when an instruction is executed. The memory controller 92 can also provide a memory protection function that isolates multiple processes within the system and isolates system processes from user processes. Therefore, a program running in the first mode can only access memory mapped by the virtual address space of its own process; it cannot access memory within the virtual address space of another process unless memory sharing has been established between the processes.

[0071] In addition, the computing system 600 may include a peripheral device controller 83 that is responsible for transmitting instructions from the CPU 91 to peripheral devices, such as a printer 94 , a keyboard 84 , a mouse 95 , and a disk drive 85 .

[0072] The display 86 controlled by the display controller 96 is used to display the visual output generated by the computing system 600. Such visual output may include text, graphics, animated graphics, and video. The display 86 may be implemented as a cathode ray tube (CRT) based video display, a liquid crystal display (LCD) based flat panel display, a gas plasma based flat panel display, or a touch pad. The display controller 96 includes the electronic components required to generate the video signals sent to the display 86.

[0073] In addition, the computing system 600 may include communication circuits, such as a network adapter 97, which may be used to connect the computing system 600 to an external communication network (e.g., Figure 5 The communication network 28) enables the computing system 600 to communicate with other nodes of the network (eg, the UE 30).

[0074] It should be appreciated that, in application, the examples of the multiple methods and multiple devices described herein are not limited in application to the details of the construction and arrangement of the multiple parts set forth in the following description or shown in the accompanying drawings. These methods and devices can be implemented in other examples and can be practiced or can be performed in various ways. Examples of specific embodiments are provided herein only for illustrative purposes and are not intended to limit the examples of these specific embodiments. In particular, the multiple actions, multiple elements, and multiple features described in conjunction with any one or more examples are not intended to be excluded from similar effects in any other examples.

[0075] It should be understood that the methods and systems described herein are not limited to specific methods, specific components or specific implementations. It should also be understood that the terminology used herein is only for the purpose of describing specific examples and is not intended to be limiting.

[0076] Some parts of this specification sheet describe multiple embodiments in terms of the application and symbolic representation of the operation of information. The technical personnel in the field of data processing usually use these application descriptions and representations to effectively convey the essence of their work to other technical personnel in the field. Although these operations are described functionally, computationally or logically, these operations should be understood to be implemented by computer programs or equivalent circuits, or microcodes, etc. In addition, without loss of generality, it is sometimes convenient to refer to the arrangement of these operations as components. The described operations and their associated components can be implemented in software, firmware, hardware or any combination thereof.

[0077] Any of the steps, operations or processes described herein may be performed or implemented individually or in combination with other devices using one or more hardware or software components. In one embodiment, the software component is implemented using a computer program product including a computer-readable medium containing computer program code that may be executed by a computer processor to perform any or all of the steps, operations or processes described herein.

[0078] Each example may also relate to an apparatus for performing the operations herein. The apparatus may be specially constructed for the desired purpose, and / or the apparatus may include a computing device that is selectively activated or reconfigured by a computer program stored in a computer. Such a computer program may be stored in a non-transitory tangible computer-readable storage medium that can be coupled to a computer system bus, or in any type of medium suitable for storing electronic instructions. In addition, any computing system mentioned in this specification may include a single processor, or may be an architecture that employs a multi-processor design for increased computing power.

[0079] As referred to herein, the term "publishing" may refer to creating and publishing original content on a platform for viewing by one or more users. Publishing may be an act of content creation that may include writing text, uploading images or videos, or sharing links. In some instances, publishing may make content visible to a specific audience or the public.

[0080] As used herein, the term "sharing" can refer to the act of redistributing existing content to expand the reach and visibility of existing content. In some instances, sharing can involve taking content created by oneself or others and disseminating it through various online channels, often with the addition of personal commentary or context.

[0081] The terms "publishing" and "sharing" mentioned herein may differ primarily in their origin and purpose. In one example, publishing may generally involve creating new content and initiating conversations, while sharing may focus on amplifying existing content and participating in ongoing discussions. In some instances, publishing may require more effort in content creation, while sharing may be a faster way to interact with and spread information across platforms or networks. However, in some instances, publishing and sharing may be used interchangeably.

[0082] As referred to herein, the term "story" may refer to a content item associated with a social media platform that may allow one or more users to share one or more images, one or more photos, one or more videos, etc. that are visible for a limited time. In one example, the limited time may be 24 hours, but it is contemplated that the limited time may be any suitable time increment. In one example, a story may often appear at the top of a user's feed and may include various enhancements to the shared one or more images, one or more photos, or one or more videos. These enhancements may be one or more of: filters; stickers; or text, etc.; or any combination thereof.

[0083] As used herein, according to examples of the present disclosure, the terms "data," "content," "information," and similar terms may be used interchangeably to refer to data that can be sent, received, and / or stored. In addition, the term "exemplary" as used herein is not intended to express any qualitative assessment, but merely to express an illustration of an example. Therefore, any such terms should not be used to limit the spirit and scope of the examples of the present disclosure.

[0084] As defined herein, "computer-readable storage media" may be distinguished from "computer-readable transmission media," which refers to non-transitory physical or tangible storage media (e.g., volatile storage devices or non-volatile storage devices) and "computer-readable transmission media," which refers to electromagnetic signals.

[0085] As referred to herein, an "application" may refer to a computer software package that can perform specific functions for a user and / or in some cases for another application or applications. One or more applications can be run using an operating system (OS) and other supporting programs. In some examples, one or more applications can request one or more services from other entities via an application programming interface (API) and communicate with these entities. It is contemplated herein that "user" and "device" are generally used interchangeably.

[0086] As referred to herein, "artificial reality" may refer to an immersive form of reality that has been adjusted in some manner before being presented to a user, and artificial reality may include, for example, virtual reality, augmented reality, mixed reality, hybrid reality, metaverse reality, or some combination and / or derivative thereof. Artificial reality content may include entirely computer-generated content or computer-generated content combined with captured (e.g., real-world) content. In some instances, artificial reality may also be associated with applications, products, accessories, services, or some combination thereof, which are used, for example, to create content in artificial reality and / or otherwise be used in artificial reality (e.g., to perform activities in artificial reality).

[0087] As referred to herein, “artificial reality content” may refer to content such as video, audio, tactile feedback, or some combination thereof, any of which may be presented in a single channel or in multiple channels (e.g., stereoscopic video that creates a three-dimensional (3D) effect for the viewer).

[0088] As referred to herein, a metaverse may represent an immersive virtual space or world in which devices may be used in a network in which one or more social connections may, but need not, exist between users in the network or with environments in the virtual space or world. A metaverse or metaverse network may be associated with a three-dimensional (3D) virtual world, an online game (e.g., a video game), one or more content items (e.g., images, videos, non-fungible tokens (NFTs)), and wherein the content items may, for example, be purchased with digital currency (e.g., cryptocurrency) and other suitable currencies. In some examples, a metaverse or metaverse network may allow for the generation and provision of an immersive virtual space in which remote users may socialize, collaborate, learn, shop, and / or participate in various other activities within the virtual space, including through the use of augmented reality / virtual reality / mixed reality.

[0089] As used herein, "or" is inclusive and not exclusive, unless expressly indicated otherwise or the context indicates otherwise. Thus, as used herein, "A or B" means "A, B, or both," unless expressly indicated otherwise or the context indicates otherwise. Furthermore, "and" is both joint and several, unless expressly indicated otherwise or the context indicates otherwise. Thus, as used herein, "A and B" means "A and B, jointly or severally," unless expressly indicated otherwise or the context indicates otherwise.

[0090] The foregoing description of various examples has been presented for the purpose of illustration; the foregoing description is not intended to be exhaustive or to limit the patent rights to the precise form disclosed. Those skilled in the relevant art may recognize that many modifications and variations are possible in light of the present disclosure.

[0091] The scope of the present disclosure covers all changes, substitutions, variations, alterations and modifications to the exemplary examples described or shown herein that will be understood by those skilled in the art. The scope of the present disclosure is not limited to the exemplary examples described or shown herein. In addition, although the present disclosure describes and illustrates the various examples herein as including specific parts, elements, features, functions, operations or steps, any example in these examples may include any combination or arrangement of any of the parts, elements, features, functions, operations or steps described or shown anywhere herein that will be understood by those of ordinary skill in the art. In addition, the reference to a device or system suitable for, arranged to, capable of, configured to, enabled to, able to operate or operable to perform a specific function, or a component in a device or system in the appended claims includes the device, system, component, whether the device, system, component or the specific function is activated, turned on or unlocked, as long as the device, system or component is so suitable for, arranged to, capable of, configured to, enabled to, able to operate or operable. Furthermore, although this disclosure describes or illustrates a particular example as providing particular advantages, the particular example may provide none, some, or all of those advantages.

[0092] Finally, the language used in this specification is selected primarily for readability and instructional purposes, and may not be selected to describe or limit the subject matter of the present invention. Therefore, it is intended that the scope of the patent rights is not limited by the specific implementation, but by any claims published in this application based thereon. Therefore, the disclosure of each example is intended to illustrate, not limit, the scope of the patent rights set forth in the appended claims.

[0093] A method, system, or apparatus may be provided for: receiving a media item; receiving an input via a device associated with a user; analyzing the context of the input using a machine learning model; generating one or more overlays associated with the context of the input based on the use of the machine learning model; receiving one or more selections associated with the one or more overlays; and creating combined media based on the media item and the one or more overlays; and sending the combined media. The combined media may be sent to a display of a device associated with the user. The combined media may include one or more selections associated with the one or more overlays superimposed on the media item. The input may be natural language text or natural language audio. An indication to share or publish the combined media may be performed. The indication to share or publish may cause the combined media to be sent to one or more other users. The one or more overlays may include one or more stickers. The one or more overlays may be configured to adjust relative to the media item. The combined media may be configured to be iteratively adjusted in response to one or more inputs associated with one or more users.

[0094] A system or apparatus may include executable instructions that can be performed by a processor to: receive a media item and an input via a device associated with a user; analyze the context of the input using a machine learning model; generate one or more overlays associated with the context of the input based on the use of the machine learning model; receive one or more selections associated with the one or more overlays; create combined media based on the media item and the one or more overlays; send the combined media. The input may be natural language text or natural language audio. The combined media may be sent to a display of a device associated with the user. The media item may be an image or a video. The combined media may include one or more selections associated with the one or more overlays superimposed on the media item. An indication to share or publish the combined media may cause the combined media to be sent to one or more other users. The one or more overlays may include one or more stickers. The one or more overlays may be configured to adjust relative to the media item. The combined media may be configured to be iteratively adjusted in response to one or more inputs associated with the one or more users.

[0095] A method, system, or apparatus may be provided for: receiving a media item and a first input via a first device associated with a first user; applying a machine learning model to analyze the context of the first input; generating a first list of overlays associated with the context of the first input through the machine learning model; receiving one or more first selections associated with one or more overlays in the generated first list of overlays; providing a first combined media via a graphical user interface of a first device or a second device in one or more devices; receiving a second input associated with the first combined media via a second device associated with a second user; applying a machine learning model to analyze the context of the second input; generating a second list of overlays associated with the context of the second input through the machine learning model; receiving one or more second selections associated with one or more overlays in the generated second list of overlays; and providing a second combined media via a graphical user interface of a first device or a second device in one or more devices. All combinations of this paragraph and the above paragraphs (including deletion or addition of steps) are considered in a manner consistent with other parts of the specific implementation.

Claims

1. A method comprising: receiving a media item; receiving input via a device associated with a user; using a machine learning model to analyze the context of the input; generating one or more overlays associated with the context of the input based on using the machine learning model; receiving one or more selections associated with the one or more overlays; generating combined media based on the media item and the one or more overlays; and The combined media is transmitted.

2. The method according to claim 1, wherein: The input includes natural language text or natural language audio.

3. The method according to claim 1, wherein: The combined media is sent to a display of the device associated with the user.

4. The method according to claim 1, wherein: The combined media includes the one or more selections associated with the one or more overlays superimposed on the media item.

5. The method according to claim 1, further comprising: An indication to share the combined media is received.

6. The method according to claim 1, wherein: The one or more overlays include one or more tiles.

7. The method according to claim 1, wherein: The one or more overlays are configured to adjust relative to the media item.

8. The method according to claim 1, wherein: The combined media is configured to be iteratively adjusted in response to one or more inputs associated with one or more users.

9. The method according to claim 1, further comprising: Feedback associated with the one or more overlays is received to iteratively train the machine learning model.

10. An apparatus comprising: one or more processors; as well as at least one memory storing instructions that, when executed by the one or more processors, cause the apparatus to: receiving a media item; receiving input via a device associated with a user; using a machine learning model to analyze the context of the input; generating one or more overlays associated with the context of the input based on using the machine learning model; receiving one or more selections associated with the one or more overlays; generating combined media based on the media item and the one or more overlays; and The combined media is transmitted.

11. The device according to claim 10, wherein: The input includes natural language text or natural language audio.

12. The device according to claim 10, wherein: The combined media is sent to a display of the device associated with the user.

13. The device according to claim 10, wherein: The media item includes an image or a video.

14. The device according to claim 10, wherein: The combined media includes the one or more selections associated with the one or more overlays superimposed on the media item.

15. The device according to claim 10, wherein: When the one or more processors further execute the instructions, the apparatus is configured to: An indication to share the combined media is received.

16. The device according to claim 10, wherein: The one or more overlays include one or more tiles.

17. A non-transitory computer readable medium storing instructions which, when executed, cause: receiving a media item; receiving input via a device associated with a user; using a machine learning model to analyze the context of the input; generating one or more overlays associated with the context of the input based on using the machine learning model; receiving one or more selections associated with the one or more overlays; generating combined media based on the media item and the one or more overlays; and The combined media is transmitted.

18. The non-transitory computer readable medium of claim 17, wherein: The combined media is configured to be iteratively adjusted in response to one or more inputs associated with one or more users.

19. The non-transitory computer readable medium of claim 17, wherein: When the instructions are executed, they also cause: Feedback associated with the one or more overlays is received to iteratively train the machine learning model.

20. The non-transitory computer readable medium of claim 17, wherein: The input includes natural language text or natural language audio.

Images