Generate prompts for user linked notes

Abstract

Improve understanding of search results from search result pages. A system and method for generating a prompt for user data input entry can include obtaining context data. The context data can be processed to determine whether an input entry interface is to be provided. In response to determining that an input entry interface is to be provided, the context data or other data associated with the content display instance can be processed with a generative model to generate a prompt that can be provided to a user. User input data can then be obtained and stored for provision to other users.

Description

[Technical field]

[0001] Related Applications This application claims priority to U.S. Nonprovisional Application No. 18 / 392,648, filed December 21, 2023, and the benefit of U.S. Provisional Application No. 63 / 596,484, filed November 6, 2023. Applicants claim priority to and the benefit of each of such applications, each of which is incorporated by reference in its entirety.

[0002] FIELD OF THE DISCLOSURE This disclosure relates generally to generating prompts for taking link notes and, more specifically, to determining when and how to prompt a user to provide notes regarding links associated with web resources, which may then be provided to other users. [Background technology]

[0003] Understanding search results from a search result page can be difficult because titles and text fragments may provide limited information that may not be relevant to the user's interests, leading to time-consuming review of web resources that may not yield the desired information. Obtaining additional information about a web resource can be difficult and may involve additional searches that may or may not identify relevant information.

[0004] Additionally, user insight can be difficult to obtain. In particular, users may have difficulty deciding which words to use. Furthermore, the words may not be directed to points of interest to other users and / or may not be rich enough to produce the desired results. Summary of the Invention

[0005] Aspects and advantages of embodiments of the disclosure will be set forth in part in the description that follows, or may be learned from the description, or may be learned by practice of the embodiments.

[0006] One exemplary aspect of the present disclosure is directed to a computing system for generating comment prompts and retrieving input. The system may include one or more processors and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations. The operations may include obtaining content data. The content data may be associated with a web resource. The operations may include processing the content data with a generative model to generate a predictive prompt. The prompt may include a predicted text string associated with a comment to the web resource. The operations may include providing an input prompt interface for display to the predictive prompt. The input prompt interface may be configured to receive input. The operations may include obtaining comment input data from a user computing system via the input prompt interface. In some implementations, the comment input data may include user-generated comments for the web resource. The operations may include storing data associated with the comment input data with data associated with the web resource. The data associated with the comment input data may be stored in a searchable database that is provided for display in response to the web resource being provided as a search result.

[0007] In some implementations, the operations may include obtaining user data. The user data may be associated with a particular user. The user computing system may be associated with the particular user. Processing the content data with the generative model to generate the predictive prompt may include processing the content data and the user data with the generative model. The user data may include user search history data. The generative model may generate the predictive prompt based on the particular user previously searching for information associated with the topic of the web resource. In some implementations, the user data may include user browser history data. The generative model may generate the predictive prompt based on the particular user previously viewing other web resources that include information associated with the topic of the web resource. The operations may include generating a graphics card based on the user data, the content data, and the comment input data. The graphics card may include a user profile identifier for the particular user and data associated with the comment input data. The operations may include storing the graphics card. The graphics card may include a graphic background generated with the image generation model based on the comment input data.

[0008] In some implementations, the operations can include obtaining a search query, determining that a web resource is associated with the search query, and providing a particular search result for display. The particular search result can include a link to the web resource, a title of the web resource, and data associated with the comment input data. Storing the data associated with the comment input data with the data associated with the web resource can include generating a web resource note and storing the web resource note with a plurality of other web resource notes associated with the web resource. In some implementations, the operations can include providing the web resource note and the plurality of other web resource notes to a note interface, which provides the web resource note and the plurality of other web resource notes to a plurality of graphics cards. The generative model can include an autoregressive language model. The generative model can be prompted to generate a question describing a request for information about the web resource.

[0009] Another exemplary aspect of the present disclosure is directed to a computer-implemented method for linked note prompts. The method can include obtaining, by a computing system including one or more processors, context data. The context data can be associated with a particular content display instance. The particular content display instance can include a particular user viewing a particular content item. The method can include determining, by the computing system, an input prompt action based on the context data. The input prompt action can include providing an input entry interface to a user to obtain user input. The method can include processing, by the computing system, the context data with a generative language model to generate a predictive prompt. In some implementations, the predictive prompt can include a natural language request for information generated based on the context data. The method can include providing, by the computing system, a predictive prompt in the input entry interface, and obtaining, by the computing system, user-generated content via the input entry interface. The method can include generating, by the computing system, a linked note based on the user-generated content. In response to the particular content item being determined as a search result, a linked note can be generated that is provided for display in the search result interface.

[0010] In some implementations, the context data may be associated with a type of content being provided for display. The context data may be associated with a particular user associated with a particular content display instance. The context data may include search history data. The content being provided for display may be associated with a particular web resource. The context data may be associated with interaction data of links of the particular web resource of a plurality of social networking platforms. The prompt action may be determined based on the interaction data. In some implementations, the context data may include user data and content data. The prompt action may be determined based on a topic associated with the content being provided for display being one of a plurality of topics for which the particular user has knowledge determined based on the user data. The context data may include previous notes generated by the particular user. The predictive prompt may include a structure based on a previous structure for the previous notes.

[0011] Another exemplary aspect of the present disclosure is directed to one or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations. The operations can include obtaining a first search query at a first time and determining that a web resource is responsive to the first search query. The operations can include obtaining content data. The content data can be associated with the web resource. The operations can include processing the content data with a generative model to generate a predictive prompt. The prompt can include a predicted text string associated with a comment to the web resource. The operations can include providing a predictive prompt for display within an input prompt interface. The input prompt interface can include an input entry box. The operations can include obtaining comment input data from a user computing system via the input prompt interface. The comment input data can include user-generated content. The operations can include storing the user-generated content. The operations can include obtaining a second search query at a second time. The second time can be different from the first time. The operations can include determining that the web resource is responsive to the second search query and providing the user-generated content in a search result interface with data describing the web resource.

[0012] Another exemplary aspect of the present disclosure is directed to a computing system for graphics card generation. The system may include one or more processors and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations. The operations may include obtaining card data. The card data may describe content in the graphics card. The content may be associated with one or more topics. The operations may include processing the card data to determine one or more entity tags associated with the content. The one or more entity tags may be associated with one or more topics. The operations may include accessing a media content item database to obtain one or more media content items. Based on a determination that the one or more media content items are associated with one or more entity tags related to the content, the one or more media content items may be obtained. The operations may include providing the one or more media content items for display. The one or more media content items may be provided for display in an interactive user interface. The one or more media content items may be selectable to be inserted into the graphics card.

[0013] In some implementations, the graphics card may be associated with a linked note. The linked note may include user-generated content tagged to a particular web resource. The actions may include obtaining an input selection associated with one or more media content items, generating an enhanced graphics card, and providing the enhanced graphics card for display. The enhanced graphics card may include at least a portion of the content of the graphics card and at least a portion of the one or more media content items. In some implementations, the actions may include obtaining an adjustment input. The adjustment input may be associated with a request to enhance the enhanced graphics card. The actions may include generating an updated graphics card based on the adjustment input. The updated graphics card may include an enhanced graphics card with one or more adjustments. The actions may include providing the updated graphics card for display. The one or more adjustments may include at least one of changing the layout of the enhanced graphics card, changing the cropping of the one or more media content items, changing the size, color, or template of the one or more content items.

[0014] In some implementations, the media content item database may include a user-specific database. The user-specific database may be associated with a particular user. The particular user may have generated at least a portion of the content. In some implementations, the user-specific database may include an image gallery associated with the particular user. The image gallery may be stored on a server computing system associated with the particular content item storage platform. In some implementations, the user-specific database may include a local storage database of a user computing device. The media content item database may include a plurality of media content items. In some implementations, the plurality of content items may be pre-processed to generate a plurality of respective metadata sets.

[0015] Another exemplary aspect of the present disclosure is directed to a computing system. The system may include one or more processors and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations. The operations may include providing an input draft interface for display. The input draft interface may include a graphical user interface including a plurality of attribute options and a text entry box. The plurality of attribute options may be associated with a plurality of candidate attributes for content item generation. The operations may include obtaining a selection of a particular attribute option of the plurality of attribute options via the input draft interface. The particular attribute option may be associated with a particular candidate attribute. The operations may include obtaining a text input via a text entry box of the input draft interface. The text input may be associated with a prompt intent for content item generation. The operations may include processing the particular attribute option and the text input with a generative model to generate a model-generated content item. The mode-generated content item may include the particular candidate attribute. In some implementations, the model-generated content item may be associated with the prompt intent. The operations may include providing the model-generated content item for display via the input draft interface.

[0016] In some implementations, the operations may further include obtaining an input selection via the input draft interface and generating an enhanced graphics card based on the input selection. The enhanced graphics card may include a graphics card enhanced to include the model-generated content items. The operations may include providing the enhanced graphics card for display. The plurality of candidate attributes may include a plurality of different styles. The plurality of different styles may be associated with at least one of a plurality of different artistic styles or a plurality of different writing styles.

[0017] In some implementations, the plurality of candidate attributes may include a plurality of different tones. The plurality of different tones may be associated with at least one of a plurality of different emotions or a plurality of different pace types. In some implementations, the generative model may be retrieved from the generative model database based on the selection of the particular attribute option. The particular attribute soft prompt may be retrieved based on the selection of the particular attribute option. The particular attribute soft prompt may include a set of learned parameters. The set of learned parameters may be processed with the generative model to generate the model-generated content item.

[0018] In some implementations, the first search query and the second search query may be different. The comment entry data may include multi-modal data. The multi-modal data may include text data and image data.

[0019] Other aspects of the present disclosure are directed to various systems, apparatus, non-transitory computer-readable media, user interfaces, and electronic devices.

[0020] These and other features, aspects, and advantages of various embodiments of the present disclosure will become better understood with reference to the following description and appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the present disclosure and, together with the detailed description, serve to explain associated principles.

[0021] Detailed descriptions of embodiments directed to those skilled in the art are set forth herein with reference to the accompanying drawings. [Brief description of the drawings]

[0022] [Figure 1] 1 illustrates a block diagram of an exemplary linked notes generation system according to an exemplary embodiment of the present disclosure. [Diagram 2] 1 illustrates a block diagram of an exemplary user prompt system, in accordance with an exemplary embodiment of the present disclosure. [Diagram 3] 1 illustrates a flowchart diagram of an exemplary method for performing linked notes prompts according to an exemplary embodiment of the present disclosure. [Figure 4] 1 illustrates a diagram of an exemplary prompt, according to an exemplary embodiment of the present disclosure. [Figure 5A] 1 illustrates an example note interface diagram with topic prompts according to an exemplary embodiment of the present disclosure. [Figure 5B] 1 illustrates an exemplary notes interface with similar article prompts according to an exemplary embodiment of the present disclosure. [Figure 5C] 1 illustrates an example predictive prompt diagram according to an example embodiment of the present disclosure. [Figure 6A] 1 illustrates a diagram of an exemplary Linked Notes entry point according to an exemplary embodiment of the present disclosure. [Figure 6B] 1 illustrates a diagram of an exemplary Linked Notes entry point according to an exemplary embodiment of the present disclosure. [Figure 6C] 1 illustrates a diagram of an exemplary Linked Notes entry point according to an exemplary embodiment of the present disclosure. [Figure 7]1 illustrates a flowchart diagram of an exemplary method for performing linked notes generation, according to an exemplary embodiment of the present disclosure. [Figure 8] 1 illustrates a flowchart diagram of an exemplary method for providing linked notes display, according to an exemplary embodiment of the present disclosure. [Figure 9A] 1 illustrates a diagram of an exemplary graphics card interface, according to an exemplary embodiment of the present disclosure. [Figure 9B] 1 illustrates a diagram of an exemplary graphics card interface, according to an exemplary embodiment of the present disclosure. [Figure 9C] 1 illustrates a diagram of an exemplary graphics card interface, according to an exemplary embodiment of the present disclosure. [Figure 9D] 1 illustrates a diagram of an exemplary graphics card interface, according to an exemplary embodiment of the present disclosure. [Figure 9E] 1 illustrates a diagram of an exemplary graphics card interface, according to an exemplary embodiment of the present disclosure. [Figure 9F] 1 illustrates a diagram of an exemplary graphics card interface, according to an exemplary embodiment of the present disclosure. [Figure 9G] 1 illustrates a diagram of an exemplary graphics card interface, according to an exemplary embodiment of the present disclosure. [Figure 10] 1 illustrates a diagram of an exemplary card generation interface according to an exemplary embodiment of the present disclosure. [Figure 11] 1 illustrates a diagram of an exemplary content item generation interface, according to an exemplary embodiment of the present disclosure. [Figure 12] 1 illustrates a diagram of an exemplary image suggestion interface, according to an exemplary embodiment of the present disclosure. [Figure 13] FIG. 2 illustrates a flowchart diagram of an exemplary method for making image suggestions, according to an exemplary embodiment of the present disclosure. [Figure 14] 1 illustrates a flowchart diagram of an exemplary method for performing content item generation, according to an exemplary embodiment of the present disclosure. [Figure 15A]1 illustrates a block diagram of an exemplary computing system that performs linked notes prompting, according to an exemplary embodiment of the present disclosure. [Figure 15B] 1 illustrates a block diagram of an exemplary computing system that performs linked notes prompting, according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Reference numbers repeated among the drawings are intended to identify like features in the various embodiments.

[0024] In general, the present disclosure is directed to generating prompts for user data entry. In particular, the systems and methods disclosed herein can leverage context determination (e.g., determining a context in which a user is likely to provide a note and / or determining a comment gap and / or a content gap for a particular link) to determine an input entry interface (e.g., a link note input entry interface) and can leverage generative models (e.g., large-scale language models) to generate prompts based on user data (e.g., user search history and / or user browse history) and / or content data (e.g., content topic and / or content type). For example, a user can be prompted to provide notes on a particular web resource (and / or other content item) on a search result page, during review of the web resource, and / or on a next search instance. Prompts can be generated based on previous user notes, previously viewed content, content topic, and / or content type, and can provide a prompt to the user requesting information in a format that generates insightful notes.

[0025] Linked notes can provide additional information about a web resource without reviewing the web resource, and linked notes can be provided by other users. The system and method can determine when to provide a linked note prompt to a user based on a context determined to be associated with valuable note capture. For example, a particular user may provide more authoritative and / or more detailed information about a particular topic based on previously acquired knowledge and / or based on previously generated notes. Additionally and / or alternatively, a particular content type can be determined to be associated with a user's comments and / or user confusion.

[0026] The prompts provided to the user can "inspire" the user to provide more detailed information and / or instruct the user to leave notes on specific topics and / or features of the web resource. The generative model can process user data and / or content data to generate predictive prompts. In particular, the generative model can leverage the user's search history, the user's browsing history, the user's previous notes, and / or other user data to generate suggested notes, prompting questions, and / or note templates. Alternatively and / or additionally, the generative model can leverage a semantic understanding of the web resource, topic classifications, content type classifications, other notes associated with the web resource, and / or other content data to generate suggested notes, prompting questions, and / or note templates.

[0027] The input entry interface can provide predictive prompts to the user. The input entry interface can then obtain input from the user (e.g., comment input data) to generate user-generated content describing the linked note. In some implementations, a graphic card can be generated based on the linked note. The graphic card can include the user-generated content of the linked note, a user profile identifier (e.g., name and / or image), link information, and / or a graphic background. The linked note and / or graphic card can be stored in association with the web resource. The stored linked note and / or graphic card can then be retrieved in response to one or more users searching the web resource and / or one or more users interacting with the note interface.

[0028] Understanding search results from a search results page can be difficult because titles and text snippets can provide limited information that may not be relevant to a user's interests, leading to time-consuming review of web resources that may not yield desired information. Additionally and / or alternatively, obtaining additional information about a web resource can be difficult, which may involve additional searches where relevant information may or may not be identified. Social media posts, blog posts, and / or reviews of the web resource and / or entities associated with the web resource may lack detail, may be misguided, and / or may lack context and / or perspective.

[0029] Linked notes (e.g., linked notes obtained from a user and / or generated by a generative model) can provide additional information about a web resource that may inform other users of its relevance to their request. Linked notes may be provided on a search results page and / or displayed in a notes interface that may be accessed from the search results page and / or the web resource. Linked notes may be provided on a graphics card, in a text panel inline with a snippet of text, and / or in other formats.

[0030] Determining when and how to prompt a user to generate a note can be based on a determination of note inconsistencies (e.g., for an article, there are many notes on the blogging platform and / or social media platform, but relatively few for the search platform's note interface), a determination of the user's specific interests (e.g., is the resource similar to other articles the user has viewed in the past), resource trends (e.g., has this resource and / or similar resources been commented on previously), a determination of resources to note (e.g., does the note provide utility), and / or other determinations. The prompts can be generated based on generative language model processing, which can include processing previous notes (e.g., other notes by the user and / or other users), processing search queries, processing web resources, and / or processing other data.

[0031] Linked notes prompts can be utilized to initiate and / or prompt collection of information about web resources that can then be provided to other users, which may identify user opinions, user summaries, and / or specific details of other users. The retrieved notes may then be provided in a search results interface and / or discovery feed. Linked notes prompts can be determined based on determination of note mismatches, determination of user specific interests, resource trends, determination of resources to note, and / or other determinations.

[0032] Obtaining additional information from other users can be useful to users to judge the topic and quality of search results that may be difficult to discern from traditional search result displays. However, useful detailed information can be difficult to obtain. An interface can be provided that obtains more information from relevant users by determining when to prompt the user to create a note and generating context-aware prompts.

[0033] When a user views a search result, it may be difficult to prompt the user to provide information (e.g., comments, reviews, insights, etc.). A prompt generation system may be utilized to target the right user with the right prompt at the right time / place based on the insight. In particular, the prompt may aid in creating a post with desired characteristics (e.g., a desired level of detail on a desired topic, and / or other characteristics). The prompt generation system disclosed herein may prompt a particular user to generate (or create) a note and / or a particular web resource (and / or content item) to generate (or create) a note via the prompt generation system.

[0034] In some implementations, the systems and methods disclosed herein can be utilized to prompt users to generate standalone content. The standalone content can include user recipes, user tutorials, user graphics, life updates, link shares, and / or other user-generated content. The standalone content can be generated free-form and / or based on model-generated prompts. In some implementations, one or more machine-learned models can be utilized to generate content templates and / or to enhance user-provided content (e.g., reconstruct and / or restyle text, images, audio, interface elements, and / or video).

[0035] In some implementations, the Link Notes and / or interactions with the Link Notes may be utilized to adjust the ranking of the web resource, the tagging of the web resource, the embedding of the web resource, and / or the indexing of the web resource. For example, in some implementations, the Link Notes may be processed to determine the quality of the web resource. The quality determination may be determined based on processing the Link Notes with one or more machine-learned models (e.g., sentiment analysis models, language models, classification models, etc.). The Link Notes may be processed with one or more machine-learned models to identify topics related to the web resource, identify the bias of the web resource, the usefulness of the web resource, and / or the direction of the web resource. The Link Notes may be utilized to suggest additional content, may be embedded for embedding-based search, and / or may be utilized for query suggestions.

[0036] Linked notes in the notes interface may be ranked and / or displayed based on interactions, quality as determined by machine learning models, responsiveness to queries, level of detail, and / or other attributes. In some implementations, linked notes generated by a user may be provided to all other users, only users within the user's social network, and / or only users determined to be associated with the user based on interests, location, and / or activity.

[0037] Linked Notes may be utilized for multiple different content items and may not be limited to web resources. For example, the systems and methods disclosed herein may be utilized to generate prompts and / or interfaces for obtaining, inspiring, and / or generating Linked Notes for local files (e.g., documents, images, videos, etc. on a device), intranet files, and / or other content item sources, which may include folders on external drives, documents in the cloud, etc.

[0038] In some implementations, the input interface may include an open input interface that provides one or more options for providing user input. Alternatively and / or additionally, the input interface may include multiple features and / or options for generating user-generated content that may be utilized for linked notes and / or standalone content. The input interface may include an independent content item user interface that may enable a user to add images, links, and / or different template type content and may be interactive. The interactive user interface may include image suggestions, template suggestions, text suggestions, layout suggestions, link suggestions, widget suggestions, template suggestions, and / or other options (e.g., other types of suggestions).

[0039] Image suggestions may include processing the user input text, data associated with the web resource, the generated prompt, a stock photo library, and / or an image database associated with the user (e.g., an online image gallery associated with the user, and / or local images on the user computing device) to determine images relevant to a particular context (e.g., those associated with the user input, the web resource, and / or the generated prompt). Image suggestions may include determining one or more entities, topics, and / or features associated with the web resource and / or the user input, and then processing a stock photo library and / or image database(s) associated with the user to determine one or more particular images associated with the one or more entities, topics, and / or features associated with the web resource and / or the user input. For example, if the web resource describes a recipe for pasta, a stock image gallery and / or a user image gallery may be searched for images depicting pasta, cooking, pasta ingredients, and / or a kitchen. Other examples may include determining the text of a generated prompt that may be associated with a trip to Mexico, where one or more images from the user's image gallery may be identified and suggested based on location metadata, feature detection, optical character recognition, and / or other determination techniques that may be utilized to identify one or more images associated with a trip to Mexico. In some implementations, the image suggestions may be based on generating prompt embeddings and / or web resource embeddings and then performing an embedding search based on multiple image embeddings associated with one or more image databases. The determination and display of suggestions may be performed for images, videos, document files, audio, text data, templates, and / or other data.

[0040] Additionally and / or alternatively, the interactive input interface can include a "Help Me Write" feature. The "Help Me Write" feature can be a selectable user interface feature that can provide a generative language model interface for generating text for user-generated content. The "Help Me Write" feature can include drop-down menus for selecting a particular tone, style, format, length, and / or other attributes for the model-generated text. The "Help Me Write" feature can process user input to adjust and / or change the style, tone, format, language, vocabulary, length, and / or level of conciseness of the input text. For example, a user can select a tone from multiple tone options and enter a text string, and the input interface can provide the text string and the selected tone prompt to a generative language model (e.g., a large-scale language model) to generate a model-generated text response that can then be utilized in user-generated content (e.g., linked notes and / or stand-alone content). Alternatively and / or additionally, the input interface can interface with different generative language models associated with different attributes in response to selection of different attribute options. Different generative models may be trained and / or tuned for specific attributes.

[0041] The systems and methods of the present disclosure provide several technical effects and advantages. As an example, the systems and methods can provide an interactive user interface that can be utilized to generate prompts and obtain user input data. In particular, the systems and methods disclosed herein can leverage one or more machine learning models to determine when to request linked notes and generate prompts to request information. For example, the generative model can process user data, content data, and / or other contextual data to determine that a request for information action should be made. Additionally and / or alternatively, the generative model can generate a prompt to request information based on the user data, content data, and / or other contextual data. The prompt can be provided to the user, the user input can be received, and the linked notes can be generated and stored.

[0042] Another technical advantage of the systems and methods of the present disclosure is that user data and content data can be leveraged to determine which users may provide authoritative information regarding a particular web resource and / or when to prompt the user to provide information. For example, a user may be determined to be knowledgeable on a particular topic and / or to be a common note poster of a given type of content. Based on the determination, the user may be prompted to provide a link note to the given web resource. Alternatively and / or additionally, the topic of the content, the type of content, and / or other interactions with the content may be utilized to determine that the web resource is "proficient" to comment on. Prompts may be generated with a generative model to provide prompts that are both user-aware and content-aware.

[0043] The systems and methods disclosed herein address problems encountered by computer systems that acquire, process, and transmit data from multiple databases from multiple sources. The sheer volume of data available to users can potentially result in misinformation, misdirection, and / or lack of verification. Text snippets, titles, and / or example images in a search results interface may provide some detail about the content of a web resource. However, information from other users can provide further insight into the topic, credibility, and / or expectations, which can be utilized to reduce instances of irrelevant web resources that users navigate and review.

[0044] Other examples of technical effects and benefits relate to improved computational efficiency and improved functionality of computing systems. For example, the systems and methods disclosed herein can leverage note generation to provide an interface that provides information about links, which may mitigate redundant review of search results by providing user-based validation. The reduced amount of follow-up queries and the reduced amount of page redirects may reduce latency on user devices and reduce computational costs for search engines. Exemplary embodiments of the present disclosure will now be described in more detail with reference to the figures.

[0045] 1 illustrates a block diagram of an exemplary linked notes generation system 10 according to an exemplary embodiment of the present disclosure. In some implementations, the linked notes generation system 10 is configured to receive and / or obtain context data 12 describing user data, content data, and / or other context data associated with a web page and / or viewing stance, and, as a result of receiving the context data 12, generate, determine, and / or provide predictive prompts 18 describing generated natural language requests for information from a user. Thus, in some implementations, the linked notes generation system 10 may include a generative model 16 operable to process the context data 12 to generate predictive prompts 18 including text strings describing questions, commands, templates, and / or suggested comments.

[0046] In particular, the linked notes generation system 10 can obtain context data 12. The context data 12 can include user data (e.g., data associated with a user viewing a search results page, entering a search query, and / or viewing a discovery feed), content data (e.g., data associated with the content of the web resource 14), and / or other context data (e.g., time, query trends, comment trends, news, etc.). The context data 12 can include user search history data, user browsing history data, user purchase history data, user profile data, user note history data, topic label data for the web resource, content type labels for the web resource, other notes about the web resource, and / or other data. The context data 12 can be generated using a personalized machine-learned model and / or one or more other machine-learned models.

[0047] The contextual data 12 may be obtained based on web resources 14 that have been offered for display, previously reviewed, and / or associated with search results on a search result page. The contextual data 12 may be obtained and / or generated based on one or more user interactions, one or more global trends, and / or based on web resources 14 that are associated with a particular type of content (e.g., editorials, tutorials, blogs, news articles, sports score trackers, etc.).

[0048] A generative model 16 (e.g., an autoregressive language model, a diffusion model, and / or one or more other generative models) may process the context data 12 to generate a predictive prompt 18. The generative model 16 may include a language model (e.g., a large-scale language model, a visual language model, and / or other language models), a text-to-image generative model, and / or other generative models. The predictive prompt 18 may include text data, image data, audio data, latent encoding data, and / or multimodal data. The predictive prompt 18 may include a question to which the user may respond, a template for drafting a note, and / or one or more selectable note options. For example, the predictive prompt 18 may include a question generated based on a semantic analysis and / or topic determination of the web resource, a template generated based on previously generated notes by a particular user and / or other users, and / or a selectable note option based on previous comments provided about similar web resources. In some implementations, the predictive prompt 18 may describe a request for a particular type of information regarding the web resource 14. Alternatively and / or additionally, the predictive prompt 18 may describe general information regarding the web resource 14. The predictive prompt 18 may include a new text string not previously provided by the user and / or not provided in association with the web resource 14. The predictive prompt 18 may include a number of predicted characters, words, pixels, signals, and / or structures.

[0049] The predictive prompts 18 may be provided for display in the input entry interface. A user may interact with the input entry interface to generate user-generated content 20 that may be transmitted to a server computing system (e.g., a search engine computing system). The user-generated content 20 may include text data, image data, audio data, video data, latent encoding data, and / or multimodal data. The user-generated content 20 may describe notes about the web resource 14. The notes may describe interpretations, opinions, reviews, validations, and / or indications of quality and / or topics. The user-generated content 20 may include notes displayed on a graphic card having one or more graphics, one or more widgets, one or more links, one or more media content items, and / or a graphic background.

[0050] The linked note generation system 10 can index the linked notes 22 with the web resources 14. The indexing can be utilized to provide user-generated content 20, including linked notes, for display when providing search results for the web resources 14. Alternatively and / or additionally, the user-generated content 20 can be stored in a notes database and displayed in a notes interface when selected by one or more users.

[0051] 2 illustrates a block diagram of an exemplary user prompt system 200 according to an exemplary embodiment of the present disclosure. The user prompt system 200 is similar to the linked note generation system 10 of FIG. 1, except that the user prompt system 200 further includes an action decision block 230.

[0052] The user prompt system 200 can retrieve content data 224 and user data 226. The data can be retrieved in response to a search query, an on-back event to a search results page (e.g., returning to a search results page after viewing a web resource), a next search instance, a next instance of a web resource that is a search result, and / or based on other triggering events. The content data 224 can describe the content of the web resource and can include text, images, video, layout, audio files, transitions, potential encoding data, related links / web resources, interaction history of the web resource, and / or other data related to the web resource and / or other similar web resources. User data 226 may include user search history (e.g., a log of previous queries obtained from the user), user browsing history (e.g., a log of previously visited web pages and / or platforms), user application history (e.g., a log of previous interactions with applications), user purchasing history (e.g., a log of previously obtained products and / or services), user profile (e.g., a user identifier, user preferences, user title, user account, and / or user contacts), note history (e.g., a log of previously provided / generated notes), and / or data describing social media networks and / or activity.

[0053] The content data 224 and / or the user data 226 can be processed in a context determination block 228 to determine a context. The context determination block 228 can include one or more machine-learned models and / or one or more deterministic functions. The context determination block 228 can generate context data.

[0054] The context data may be processed in an action decision block 230 to determine that an input prompt action is to be taken. The action decision block 230 may include one or more machine-learned models and / or one or more deterministic functions. The context decision and / or the action decision may be based on heuristics.

[0055] The prompting action may include generating a prompt and providing a user with an input entry interface with the prompt to obtain linked notes for a given web resource. The prompting action may be determined based on the likelihood of the user responding, the user's credibility, the user's experience, the user's knowledge, users with different associations with previous note providers, content gaps describing differences in notes for a particular web resource versus similar web resources, comment gaps describing differences between interactions with links on one or more blogs or social media platforms versus the amount of notes, the topic of the web resource, the topic of the search, the content type, the intent of the content, and / or other data.

[0056] Certain content types (e.g., news articles, short stories, movies, skits, blog posts, and / or social media posts) may be determined to be more likely to be interacted with for Linked Notes generation and / or to benefit more from Linked Notes. Additionally and / or alternatively, interactions with web resources on other platforms may be determined. If the quantity and / or quality of interactions on the other platforms are determined to meet a threshold difference compared to the current platform, prompt actions may be determined more frequently. For example, the threshold for the prompt action may be adjusted based on the interactions on the other platforms. Alternatively and / or additionally, the threshold may be adjusted based on search and / or viewing trends associated with the web resource.

[0057] The prompt action may occur immediately after the decision and / or may be provided later as a "nudge", which may be a time determined to be higher for a response (e.g., when the user is in a particular location (e.g., at home), when the user's calendar is empty, at a particular time of day when there is increased phone activity, and / or at the next user search instance). The "nudge" may be provided via device notification, email, and / or application-based notification.

[0058] The user prompt system 200 can then leverage the context data to generate a prompt to request notes from the user based on the determination of the prompting action. The context data can include user data 226 (e.g., data associated with a user viewing a search results page, entering a search query, and / or viewing a discovery feed), content data 224 (e.g., data related to the content of the web resource), and / or other context data (e.g., time, query trends, comment trends, news, etc.). The context data can include user search history data (e.g., a list of previously searched search queries, which may include queries related to the same topic as the web resource), user browsing history data (e.g., a list of previously viewed web pages, which may include web pages associated with the same topic as the web resource), user purchase history data, user profile data (e.g., the user's name, occupation, education, preferences, etc.), user note history data, topic label data for the web resource, content type labels for the web resource, other notes about the web resource, and / or other data. The context data can be generated using a personalized machine-learned model and / or one or more other machine-learned models.

[0059] The contextual data may be obtained based on web resources that have been offered for display, previously reviewed, and / or associated with the search results on a search result page. The contextual data may be obtained and / or generated based on one or more user interactions, one or more global trends, and / or based on web resources associated with a particular type of content (e.g., editorials, tutorials, blogs, news articles, sports score trackers, etc.).

[0060] A generative model 216 (e.g., a text generation model, an image generation model, an audio generation model, a video generation model, and / or a multimodal media content item generation model) can process the context data to generate a predictive prompt 218. The generative model 216 can include a language model (e.g., a large-scale language model, a visual language model, and / or other language models), a text-to-image generation model, and / or other generative models. The predictive prompt 218 can include text data, image data, audio data, latent encoding data, and / or multimodal data. The predictive prompt 218 can include questions to which the user may respond, templates for drafting notes, and / or one or more selectable note options. For example, the predictive prompt 218 can include questions generated based on a semantic analysis and / or topic determination of the web resource (e.g., for an article about an unsolved case, the prompts can include "What do you think about the analysis of the unsolved case?", "Who do you think committed the crime?", "Was the article easy to understand and comprehensive with respect to the forensic evidence?", etc.). In some implementations, the predictive prompts 218 may include templates generated based on notes previously generated by a particular user and / or other users (e.g., if a user typically prepositions his or her notes, the prompt may include a template that begins with a preposition that emulates the previous sound style and tone). Additionally and / or alternatively, the predictive prompts 218 may include selectable note options based on previous comments provided on similar web resources (e.g., "The analysis of the incident was comprehensive and understandable, leading to reasonable conclusions," "The forensic analysis lacked a basis for scientific validation," "This article is more of a fan fiction than an actual article," etc.).In some implementations, the predictive prompt 218 may describe a request for a particular type of information about the web resource (e.g., for a biography about a politician, "What did you think about their background?", "What is your opinion on the epilogue?", "In my experience as a congressional historian, this biography is accurate / unreliable / well written / poorly constructed", etc.). Alternatively and / or additionally, the predictive prompt 218 may describe general information about the web resource. The predictive prompt 218 may include a novel text string not previously provided by the user and / or not provided in association with the web resource. The predictive prompt 218 may include a number of predicted characters, words, pixels, signals, and / or structures.

[0061] In some implementations, specific terms, details, and / or structure of a search query may be utilized to determine a user's level of experience and / or knowledge on a particular topic. The search query may be included in the context data, and the generative model 216 may generate predictive prompts 218 that reflect the determined level of experience and / or knowledge. Additionally and / or alternatively, previous search queries may be leveraged to determine search chain queries and determine search intent. The search intent may then be utilized to generate predictive prompts 218 associated with the search intent.

[0062] In some implementations, the predictive prompts 218 may differ based on the user's tendency to provide linked notes, based on the user's previous notes, based on the user's credibility, and / or based on other user data. If the user has not provided linked notes before and / or has provided only a few linked notes before, the predictive prompts 218 may consist of general prompts, multiple choice prompts, and / or format conversations. For experienced users, the predictive prompts 218 may be generated to provide the user with direct prompts, note templates, and / or options based on previous interactions.

[0063] The predictive prompts 218 may be provided for display in the input entry interface. A user may interact with the input entry interface to generate user-generated content 220 that may be sent to a server computing system (e.g., a search engine computing system). The user-generated content 220 may include text data, image data, audio data, video data, latent encoding data, and / or multimodal data. The user-generated content 220 may write notes about a web resource. The notes may write interpretations, opinions (e.g., "I believe the trades discussed in this article were fair based on the long-term outcomes of both teams"), reviews (e.g., "This short story is poorly paced and directed, and has no development of the main character"), validations (e.g., "The facts in this article are consistent with other reliable sources"), and / or indications of quality and / or topic (e.g., "This is a very well-written play about the perils of love in a war-torn town"). User-generated content 220 may include one or more graphics, one or more widgets, one or more links, one or more media content items, and / or notes displayed on a graphic card with a graphic background. For example, the text of a linked note may be provided in stylized text with a color that lines up with the model-generated image used as the background.

[0064] In some implementations, the generative model 216 can process the user-generated content 220 and generate a follow-up prompt that may request additional information and / or provide options for further customization.

[0065] The user prompt system 200 can store notes 222 with the web resources. Indexing can be utilized to provide user-generated content 220, including linked notes, for display when providing search results for the web resources. Alternatively and / or additionally, the user-generated content 220 can be stored in a notes database and displayed in a notes interface when selected by one or more users.

[0066] For example, a particular user and / or other users may enter a search query. The search engine system may determine that web resources are responsive to the search query. Search results associated with the web resources may be provided in a search result interface that includes data describing the web resource's title, media snippet, and link notes (e.g., graphics card).

[0067] 3 illustrates a flow chart diagram of an exemplary method for functioning in accordance with an exemplary embodiment of the present disclosure. Although FIG. 3 illustrates steps performed in a particular order for purposes of illustration and explanation, the method of the present disclosure is not limited to the order or arrangement specifically shown. Various steps of method 300 may be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of the present disclosure.

[0068] At 302, a computing system can obtain content data. The content data can be associated with a web resource. The content data can describe the content of the web resource and can include text data, image data, video data, audio data, latent encoding data, and / or multimodal data. The content data can include a topic of the web resource, a type of content, other notes received from other users, metadata of the web resource, a creator of the web resource, and / or descriptive data of entities related to the web resource. The content data can include content labels, the entire web resource content, a summary of the content, a media snippet, and / or content embeddings.

[0069] At 304, the computing system can process the content data with the generative model to generate a predictive prompt. The prompt can include a predicted text string associated with a comment on the web resource. The generative model can include an autoregressive language model. In some implementations, the generative model can be prompted to generate a question describing a request for information about the web resource. The generative model may include a Transformer model. The generative model may have been trained, configured, and / or prompted to perform semantic understanding on the web resource and then generate a prompt (e.g., a question) based on the semantic understanding. The predictive prompt can include a number of predicted characters that can be specifically determined based on the content data. The predictive prompt may ask about the quality of the web resource. The predictive prompt may ask about opinions and / or reviews of the web resource.

[0070] In some implementations, the computing system can obtain user data. The user data can be associated with a particular user. Processing the content data through the generative model to generate the predictive prompt can include processing the content data and the user data through the generative model. The user data can include user search history data, user browsing history data, a user's social network, user preferences, user profile information, a user's location, a user purchase history, and / or user connections. The generative model can generate the predictive prompt based on a particular user previously searching for information associated with the topic of the web resource. Alternatively and / or additionally, the generative model can generate the predictive prompt based on a particular user previously viewing other web resources that include information related to the topic of the web resource. The generative model can determine that a particular user is associated with a particular topic, a particular type of content, a particular opinion, and / or a particular context for commenting, and can generate the predictive prompt based on the determination.

[0071] At 306, the computing system can provide the predictive prompt for display using an input prompt interface. The input prompt interface can be configured to receive input. The input prompt interface can include a plurality of selectable user interface elements. In some implementations, the input prompt interface can include an input entry box for receiving input from a user. The input prompt interface can include an upload element for uploading a media content item (e.g., a document, an image, text, a video, an audio file, etc.). In some implementations, the input prompt interface can include an interface for a user to provide input to the generative model to generate a model-generated note based on the user input. Alternatively and / or additionally, the plurality of selectable user interface elements can include one or more selectable templates for generating user-generated content (e.g., user-generated notes). The one or more templates can be generated based on content items previously generated by a particular user (e.g., previously generated notes).

[0072] At 308, the computing system may obtain comment input data from the user computing system via the input prompt interface. The comment input data may include text data, image data, audio data, latent encoding data, and / or multimodal data. The comment input data may include one or more selections, one or more text strings, and / or one or more uploaded files. The comment input data may include user-generated comments about the web resource. The user-generated comments may include comments about the quality of the web resource, the topic of the web resource, and / or other aspects of the web resource.

[0073] At 310, the computing system may store data associated with the comment entry data with data associated with the web resource. Storing the data associated with the comment entry data with data associated with the web resource may include generating a web resource note and storing the web resource note with a plurality of other web resource notes associated with the web resource. The comment entry data may be indexed in association with the web resource and stored in a database and search results for the web resource may be provided. The data associated with the comment entry data may be stored in a searchable database and provided for display in response to a web resource being determined and / or provided as a search result.

[0074] In some implementations, the computing system can provide the web resource note and a plurality of other web resource notes to a note interface that provides the web resource note and a plurality of other web resource notes to a plurality of graphics cards.

[0075] Additionally and / or alternatively, the computing system can generate a graphics card based on the user data, the content data, and the comment input data. The graphics card can include a user profile identifier for the particular user and data associated with the comment input data. The computing system can then store the graphics card. The graphics card can include a graphical background generated with an image generation model based on the comment input data.

[0076] In some implementations, a computing system can obtain a search query, determine that a web resource is associated with the search query, and provide specific search results for display. The specific search results can include a link to the web resource, a title of the web resource, and data associated with the comment input data.

[0077] 4 illustrates an example prompt diagram, according to an example embodiment of the present disclosure. In particular, an example input entry interface 402 is illustrated in FIG. The example input entry interface 402 may be provided in response to determining that an input request action should be performed and that a predictive prompt should be generated. The input entry interface 402 may include links and / or references to web resources 404, a configuration panel 406 for displaying inputs to be provided during generation of user-generated content, one or more selectable predictive prompts 408, and / or one or more user interface elements for providing audio input and / or multimedia input (e.g., images).

[0078] The one or more selectable predictive prompts 408 can include predictive prompts generated by processing content of the web resource 404 with a generative language model. Alternatively and / or additionally, the one or more selectable predictive prompts 408 can include predictive prompts generated by processing notes associated with articles similar to articles provided by the web resource 404 with a generative language model.

[0079] The suggested predictive prompts may be provided in multiple formats. Additionally or alternatively, the number and / or length of predictive prompts may vary based on the content, the user, and / or other contextual data. For example, three options may be provided 412, or ten options may be provided 414. The note options may include multiple selectable prompts that the user can select as and / or as part of the linked note.

[0080] The input entry interface 402 may be utilized to receive text input (e.g., via a graphical keyboard interface), audio input (e.g., via one or more microphones), selections (e.g., selection of a user interface element associated with a predictive prompt note option), and / or input of a media content item (e.g., uploading an image). The received input may be provided for display in a preview window of the composition panel 406 and may then be utilized to generate a user-generated content item, which may include linked notes.

[0081] FIG. 5A illustrates a diagram of an exemplary note interface prompting for topics, according to an exemplary embodiment of the present disclosure. The exemplary note interface can be utilized to learn more about the web resource 502, read other users' thoughts on the web resource 502, and / or generate and provide new notes on the web resource 502. Specifically, FIG. 5A illustrates a link to the web resource 502, a previously provided note 504, one or more prompts 506 for requesting information from the user, and a recommendation user interface element 508. The link to the web resource 502 can include a thumbnail, a URL, and a title. The previously provided note 504 can include images and / or text provided by other users. The previously provided note 504 can be provided with interaction data including recommendations, comments, and the like. The one or more prompts 506 for requesting information from the user can include topics of discussion for the user to respond to the note. The recommendation user interface element 508 can be utilized to interact with the web resource 502 and / or the previously provided note 504.

[0082] FIG. 5B illustrates a diagram of an exemplary note interface for prompting similar articles, according to an exemplary embodiment of the present disclosure. The exemplary note interface can be utilized to learn more about the web resource 502, read other users' thoughts about the web resource 502 (e.g., opinions about the topic of the web resource and / or the quality of the web resource), and / or generate and provide new notes about the web resource 502 (e.g., a user can provide their own details and / or thoughts about the web resource). Specifically, FIG. 5B illustrates a link to the web resource 502, a previously provided note 504, a user interface element 510 for commenting on articles similar to the web resource 502, and a suggested article 512. The link to the web resource 502 can include a thumbnail, a URL, and a title. The previously provided note 504 can include images and / or text provided by other users. The previously provided note 504 can be provided with interaction data including recommendations, comments, and the like. A user interface element 510 for commenting on articles similar to web resource 502 may be selectable to open an input entry interface for providing link notes regarding one or more similar web resources, which may include suggested article 512.

[0083] FIG. 5C illustrates an example predictive prompt diagram, according to an example embodiment of the present disclosure. Specifically, content data related to the web resource and / or user data related to the particular user being prompted can be processed with a generative model to generate one or more predictive prompts. FIG. 5C illustrates an example question prompt 514 and a starting point prompt 516. The question prompt 514 can include a question to ask the user to provide information about a particular topic and / or subtopic. The starting point prompt 516 can include an introductory sentence for the user to build from their linked notes. User-generated content, including linked notes, can include a starting prompt when provided in a search results interface and / or a notes interface.

[0084] 6A-6C illustrate diagrams of exemplary linked notes entry points according to an exemplary embodiment of the present disclosure. Entry points to linked notes generation interfaces may be provided in a number of different interfaces and / or mediums. Specifically, FIG. 6A illustrates an exemplary search results interface for an exemplary search result 602. Contextual data associated with a search instance (e.g., user data, search query, and / or search results) may be processed to determine that a linked notes prompt is provided. FIG. 6A illustrates a first exemplary prompt 604 that is provided in response to determining that a user has previously searched for this topic. Additionally, FIG. 6A illustrates a second exemplary prompt 606 that is provided in response to determining interaction tendencies and / or determining that a particular web resource is a content type that is often interacted with for note generation.

[0085] FIG. 6B illustrates three different entry points for prompting and creation of linked notes. The first entry point 608 may include a drop-down menu in a browser application. A user may select a drop-down of options and select a note creation option from the drop-down. The second entry point 610 may be included within a search results interface. A user may select an option within the search results interface to comment on previously viewed content. The third entry point 612 may include an overlay user interface element that may be provided on a viewing window of the content of the web resource. The overlay user interface element may be provided by the web resource, the browser, and / or the operating system of the user computing device.

[0086] 6C depicts a search review entry point. Specifically, a widget and / or graphical pane may be provided to the user to provide a review of previous search experiences, which may include selecting a drop-down list element 614 to select web resources from a viewed content list 616 to indicate which web resources were useful for a particular search. Selections may be utilized to generate linked notes, re-rank the web resources, and / or navigate to a linked note generation interface.

[0087] 7 shows a flow chart diagram of an exemplary method for performing in accordance with an exemplary embodiment of the present disclosure. Although FIG. 7 shows steps performed in a particular order for purposes of illustration and explanation, the methods of the present disclosure are not limited to the order or arrangement specifically shown. Various steps of method 700 may be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of the present disclosure.

[0088] At 702, the computing system can obtain contextual data. The contextual data can be associated with a particular content display instance. A particular content display instance can include a particular user viewing a particular content item. In some implementations, the contextual data can be associated with a type of content being offered for display (e.g., an article, a scholarly paper, a blog post, an encyclopedia entry, a video, a media content library, and / or other types of content). Additionally or alternatively, the contextual data can be associated with a particular user associated with a particular content display instance. The contextual data can include search history data, browsing history data, user profile data, purchase history data, social network data, and / or other user data.

[0089] At 704, the computing system can determine an input prompt action based on the context data. The input prompt action can include providing an input entry interface to a user to obtain user input. The content being provided for display can be associated with a particular web resource. The context data can be associated with interaction data (e.g., posting, commenting, reposting, liking, and / or mentioning the link) of the particular web resource in a plurality of social networking platforms. In some implementations, the input prompt action can be determined based on the interaction data. The context data can include user data and content data. Additionally or alternatively, the input prompt action can be determined based on a topic associated with the content being provided for display being one of a plurality of topics about which the particular user is determined to have knowledge based on the user data.

[0090] At 706, the computing system can process the context data with a generative language model to generate a predictive prompt. The predictive prompt can include a natural language request for information generated based on the context data. The context data can include previous notes generated by the particular user. The predictive prompt can include a structure based on previous structures for the previous notes. In some implementations, the generative model can process the content data to generate a predictive prompt based on the content of a particular web resource.

[0091] At 708, the computing system can provide a predictive prompt in an input entry interface. The predictive prompt can be provided adjacent an input entry box for receiving and displaying the input text and / or image. The input entry interface can include a panel adjacent search results for a particular web resource. Alternatively and / or additionally, the input entry interface can be provided in a pop-up interface and / or can be redirected based on one or more inputs.

[0092] At 710, the computing system can obtain user-generated content via an input entry interface. The user-generated content can include text data, image data, video data, audio data, latent encoding data, statistical data, and / or multi-modal data. The user-generated content can be obtained via an upload interface and / or via an input entry box.

[0093] At 712, the computing system can generate linked notes based on the user-generated content. In some implementations, the computing system can generate a graphic card that includes the linked notes. The graphic card can include a graphic background that can be selected by a user and / or automatically generated. The graphic background can be generated based on the content of the web resource, the content of the linked notes, and / or the type of note. The graphic card and / or the linked notes can be stored to provide data related to the web resource. In response to a particular content item being determined as a search result, a linked note can be generated that is provided for display in a search result interface.

[0094] 8 shows a flow chart diagram of an exemplary method for performing in accordance with an exemplary embodiment of the present disclosure. Although FIG. 8 shows steps performed in a particular order for purposes of illustration and explanation, the methods of the present disclosure are not limited to the order or arrangement specifically shown. Various steps of method 800 may be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of the present disclosure.

[0095] At 802, the computing system may obtain a first search query at a first time and determine that a web resource is responsive to the first search query. The first search query may include a text query, an image query, an audio query, an embedded query, and / or a multimodal query. The web resource may be identified by a search engine, which may perform a keyword search, an embedding-based search, and / or other search techniques. The web resource may be determined to be responsive to a topic, question, and / or intent of the first search query.

[0096] At 804, the computing system can obtain content data and process the content data with the generative model to generate a predictive prompt. The content data can be associated with a web resource. The prompt can include a predicted text string associated with a comment to the web resource. The content data can describe the entire content of the web resource, a media snippet, a summary of the content, content labels, metadata, and / or content of previously provided link notes associated with the web resource. The generative model can process the content data to determine topic, point of view, intent, subject matter, structure, intended audience, type of content, and / or other content details. A predictive prompt can then be generated based on the determination.

[0097] At 806, the computing system can provide predictive prompts for display within the input prompt interface and obtain comment input data from the user computing system via the input prompt interface. The input prompt interface can include an input entry box. In some implementations, the input prompt interface can include multiple user interface elements for drafting content (e.g., notes). The comment input data can include user-generated content. In some implementations, the comment input data can include multimodal data. The multimodal data can include text data and image data.

[0098] At 808, the computing system can store the user-generated content. The user-generated content can be indexed with links to web resources. Alternatively and / or additionally, the web resources can be indexed with the user-generated content. The user-generated content can be stored with other user notes associated with a particular web resource and / or a particular user.

[0099] At 810, the computing system may obtain a second search query at a second time and determine that the web resource is responsive to the second search query. The second time may be different from the first time. In some implementations, the first search query and the second search query may be different. The second search query may include a text query, an image query, an audio query, an embedded query, and / or a multimodal query. The web resource may be identified by a search engine, which may perform a keyword search, an embedded-based search, and / or other search techniques. The web resource may be determined to be responsive to a topic, question, and / or intent of the second search query.

[0100] At 812, the computing system can provide user-generated content to the search results interface with data describing the web resource. The user-generated content can be provided with links to the web resource, titles, and snippets of text.

[0101] 9A-9G show diagrams of example graphics card interfaces according to example embodiments of the present disclosure. The systems and methods disclosed herein can be utilized to generate graphics cards for user-generated content, which can include linked notes and / or standalone content.

[0102] 9A shows two exemplary graphics cards. The illustrated graphics cards may include a user profile identifier 902 (e.g., a user profile picture and name), a body of a card 904 (e.g., a link note of stylized text overlaid on a graphics background), widget interface elements 906 (e.g., selectable user interface elements for redirecting to web resources and / or additional content), and / or interaction information 908 (e.g., likes, comments, and / or saves to the graphics card). The body of the card 904 may be configured by a particular user and / or may be automatically generated based on the link note, web resources, and / or user preferences. The widget interface elements 906 may include links to web pages, links to image galleries, links to videos, selectable elements for opening popup interfaces, additional notes, and / or other data.

[0103] 9B shows an example multi-page user-generated content and an example video user-generated content. The multi-page user-generated content can include multiple graphics cards that can be cycled through to display the user-generated content. The video user-generated content can include video with graphics and / or text overlaid on the video.

[0104] The widget interface elements 906 may include a link to the web resource, links to one or more other web resources, a video element selectable to provide video for display, media content display elements for providing media content for display (e.g., videos, images, audio files, and / or other media), reviews of the web resource, links to other notes, structured content items (e.g., structured recipes and / or structured calculators), lists (e.g., ingredients lists), map place cards (e.g., links to maps and / or web applications associated with the web resource), knowledge panels, and / or links to shopping interfaces.

[0105] 9C illustrates example interactions with the graphics card. For example, a user profile identifier 902 can be selected to peek at the user's profile 910. Additionally and / or alternatively, a video widget element 912 can be selected to expand a video for playback and / or navigate to a video player interface. The graphics card can be selected to minimize an add-on 914. A knowledge panel widget element 916 can be selected to expand a knowledge panel to provide additional information for viewing. An interaction element 918 can be selected to like, comment, save, and / or share user-generated content.

[0106] 9D illustrates prominence levels of widget interface elements (e.g., add-on elements). Specifically, the illustrated smoothie ingredients and instructions may be provided in a medium prominence interface element 922 (e.g., in a detailed view state), a low prominence interface element 924 (e.g., in a collapsed state), and / or a high prominence graphic panel 926 (e.g., in an expanded state). In some implementations, a user can interact with the widget interface elements to transition between levels and / or sizes of detail.

[0107] 9E shows search results for notes in a search results interface 930. The search results for notes may be provided in a separate tab adjacent to other search results and / or in a categorized panel. The search results for notes 932 may be selectable to navigate to an immersive viewer 936 that displays a magnified view of the user-generated content.

[0108] 9F illustrates different graphics card displays and / or note interface displays. Graphic cards may be displayed in a vertically scrollable single width format 940, an offset vertically scrollable two width format 942, a horizontally scrollable carousel interface within other search result formats 944, and / or an aligned vertically scrollable two width format 946. The format may be based on topic, interface type, user preferences, and / or context.

[0109] FIG. 9G may show different customization options. For example, a graphics card customization interface may be provided for generating a graphics card, which may include editing text 952, editing layout 954, editing images 956, and / or other customization options. In particular, the interactive interface may include options (and / or features) for content generation. The interface features may include text, images, audio, video, templates, and / or other input options. The interface features may include one or more generative model interfaces for content suggestions, template suggestions, and / or generative model-assisted generation (e.g., large-scale language models for rewriting text and / or proactively generating text, image generation models for generating new images based on web resources, user input, and / or generated prompts, audio generation models for generating narration, songs, and / or other audio, and / or graphics card generation models for processing web resources, generated prompts, and / or user input to generate graphics cards that can be suggested to the user for linked notes and / or standalone content). The interface features may include customization options for customizing the layout, font(s), interface element size(s), image(s), text, transition(s), tone, shading, and / or other features of the user-generated content. The interface features may include an option to add action user interface elements to a graphics card of the user-generated content. The action user interface elements may include selectable options for performing one or more actions (e.g., API calls, navigation to different applications, search, content item generation using a generative model, etc.).

[0110] The systems and methods disclosed herein can include image suggestions and / or image generation for generating a graphics card. For example, the systems and methods can determine that an image from a database (e.g., a server database, a local database, and / or a user image gallery) is associated with a web resource, a prompt, and / or a linked note. The image can then be provided as a suggestion to be utilized in the graphics card. Alternatively and / or additionally, the systems and methods can provide an image generation model (e.g., a text-to-image generation model) interface to generate images for inclusion in the graphics card. For example, an image generation model interface can be provided to a user, where the user may provide prompts to the image generation model, which can then generate a model-generated image that can be utilized in the graphics card.

[0111] In some implementations, one or more machine-learned models may be utilized to fact-check web resources and / or link notes. The one or more machine-learned models may include one or more generative models that can utilize application programming interfaces for API calls to obtain information and / or interact with other applications.

[0112] In some implementations, the generative model may be utilized to generate one or more model-generated linked notes that can be indexed with web resources, provide examples of linked notes, and / or provide semantic understanding notes. Alternatively and / or additionally, the generative model may be utilized to rewrite and / or suggest linked notes and / or stand-alone content. The interactive user interface may include an interface for interacting with the generative model to generate content (e.g., text, image(s), and / or other data). The interactive user interface may include options for selecting tone, style, format, lexicon, genre, and / or other attributes for adjusting the generative model to generate content with specific attributes. For example, the interactive user interface may be configured to generate prompts for the generative model based on user input, linked note prompts, and / or web resources.

[0113] The search results interface and / or discovery interface may provide statistics regarding the volume of particular searches, the volume of web resource selections, and / or interaction trends for links and / or search queries.

[0114] In some implementations, the systems and methods may include training and / or utilizing one or more contribution propensity models. The contribution propensity models may learn and / or determine user credibility (e.g., a user's relevant experience, expertise, and / or trustworthiness) for a particular user and / or a particular set of users. Additionally and / or alternatively, the contribution propensity models may learn and / or determine a propensity to provide linked notes.

[0115] The contribution propensity model may be trained to detect the likelihood of contributing, the credibility, the usefulness of a note, and / or other attributes associated with a user, a web resource, and / or a context. The contribution propensity model may be trained on a labeled dataset, based on an unlabeled dataset, and / or based on a hybrid dataset. In some implementations, the contribution propensity model may be trained on interaction data for learning a contribution prediction task, on output from a validation model for a credibility determination task, and / or on click-rates for a usefulness determination task.

[0116] 10 illustrates a diagram of an example card generation interface 1000, according to an example embodiment of the present disclosure. Specifically, a user may select an option to generate a graphic card of the linked notes. Based on that selection, a template 1002 may be selected and provided for display. A particular template may be selected based on a web resource associated with the linked notes (e.g., based on the content of the web resource), based on user interaction history, based on query history, based on user profile data, and / or based on other data.

[0117] The card creation interface 1000 may include a pull-up menu 1004 associated with a number of suggested prompts for linked note creation, which may include topic ideas. A user may pull up the menu to provide an expanded view 1006 of the suggested prompts. The expanded view 1006 may include a number of selectable suggested prompts that may be selected to generate text, images, and / or layout to be inserted into the graphics card. For example, the "How to Water Your Monstera Like a Pro" suggestion may be selected. A content item associated with the selected prompt suggestion may be inserted into the graphics card, and the graphics card may transition to an editing interface 1008. The editing interface 1008 may include options for editing the text, style, layout, font, color, and / or other edits.

[0118] 11 illustrates a diagram of an example content item generation interface 1100, according to an example embodiment of the present disclosure. Specifically, when modifying a graphics card template, an input draft interface may be utilized to generate one or more content items.

[0119] For example, a user may select an option to open a content item generation interface 1100. At 1102, the user may select one or more attributes from a drop-down menu. The one or more attributes may be associated with requested attributes of the content item to be generated. The one or more attributes may be associated with the tone and / or style of the content. At 1104, the user may generate and / or provide a text input. The text input may be associated with topic, intent, information, and / or other prompt details. The one or more attributes and the text input may be processed by the generative model to generate a model-generated content item. The model-generated content item may have one or more attributes and may be targeted to the topic, intent, information, and / or other prompt details of the text input.

[0120] At 1106, the model-generated content item may be provided for display below the text input and may be provided with a number of options. The number of options may include editing one or more attributes, editing the text input, reprocessing the data, saving the model-generated content item, exiting the interface, inserting the model-generated content item into a graphics card, and / or other options. At 1108, a modified graphics card may be provided for display with the model-generated content item inserted into the graphics card based on the user selection. The user may then edit the layout, size, color, font, and / or orientation of the model-generated content item and / or other content on the graphics card.

[0121] 12 illustrates a diagram of an exemplary image suggestion interface 1200, in accordance with an exemplary embodiment of the present disclosure. Specifically, the image suggestion interface 1200 can obtain card data, context data, and / or input data. The card data, context data, and / or input data can then be processed to determine one or more images (and / or other media content items) to be provided as suggestions for insertion into the graphics card.

[0122] For example, at 1202, a graphics card may be provided for display with options to insert additional text, stickers, and / or images. The user may then select the add image option. At 1204, an image selection interface may be provided for display, where the image selection interface may include default images, camera roll images, and / or image suggestions based on the text of the graphics card, the content of web resources associated with the linked notes, user history, and / or other data. For example, a number of images from the user's image gallery may be determined to be relevant to the text of the graphics card based on determining that the images are associated with a location (e.g., Mexico) referenced in the text of the graphics card. At 1206, the identified images may be provided for display for selection. The user may select a particular image from the identified images, and the image may be processed and inserted into the graphics card. At 1208, the selected image may be cropped and inserted into the graphics card for display.

[0123] 13 illustrates a flow chart diagram of an exemplary method for performing in accordance with an exemplary embodiment of the present disclosure. Although FIG. 13 illustrates steps performed in a particular order for purposes of illustration and explanation, the methods of the present disclosure are not limited to the order or arrangement specifically shown. Various steps of method 1300 may be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of the present disclosure.

[0124] At 1302, the computing system can obtain card data. The card data can describe content in a graphic card. The content can be associated with one or more topics. The graphic card can be associated with linked notes. The linked notes can include user-generated content tagged to specific web resources. The graphic card can include a background, one or more images, one or more text strings, and / or one or more user interface elements. The background can include a single color, multiple colors, images, and / or other data. The one or more user interface elements can include selectable widgets to provide additional information for display and / or to perform one or more actions. The content can include text data, image data, video data, latent encoding data, multimodal data, and / or other data.

[0125] At 1304, the computing system can process the card data to determine one or more entity tags associated with the content. The one or more entity tags can be associated with one or more topics. The card data can be processed through one or more machine-learned models (e.g., generative models, classification models, and / or other models) to generate entity tags. The entity tags can be associated with one or more objects, one or more companies, one or more places, one or more individuals, one or more structures, and / or other entities.

[0126] At 1306, the computing system may access a media content item database to retrieve one or more media content items. The one or more media content items may be retrieved based on determining that the one or more media content items are associated with one or more entity tags related to the content. The media content item database may include a user-specific database. In some implementations, the user-specific database may be associated with a particular user. The particular user may have generated at least a portion of the content. The user-specific database may include an image gallery associated with the particular user. The image gallery may be stored on a server computing system associated with the particular content item storage platform. Alternatively and / or additionally, the user-specific database may include a local storage database of a user computing device. The media content item database may include a plurality of media content items. The plurality of content items may have been pre-processed to generate a plurality of respective metadata sets. Determining that the one or more media content items are associated with one or more entity tags associated with the content may include determining whether the one or more media content items include features associated with the entity tags. The features may be determined based on metadata, image processing, and / or other techniques. The one or more media content items may include one or more images, one or more videos, one or more animations, one or more audio files, and / or one or more other content items.

[0127] At 1308, the computing system may provide one or more media content items for display. The one or more media content items may be provided in an interactive user interface. The one or more media content items may be selectable for insertion into the graphics card. The interactive user interface may provide multiple media content items for display, which may include media content items, web media content items, and / or other media content items relevant to the user.

[0128] In some implementations, a computing system can obtain an input selection associated with one or more media content items and generate an enhanced graphics card. The enhanced graphics card can include at least a portion of the content of the graphics card and at least a portion of the one or more media content items. The computing system can then provide the enhanced graphics card for display.

[0129] Additionally or alternatively, the computing system can obtain an adjustment input. The adjustment input can be associated with a request to expand the expansion graphics card. The computing system can generate an updated graphics card based on the adjustment input. The updated graphics card can include the expansion graphics card with one or more adjustments. The computing system can then provide the updated graphics card for display. The one or more adjustments can include at least one of changing the layout of the expansion graphics card, changing cropping of one or more media content items, changing the size, color, or template of one or more content items.

[0130] 14 shows a flow chart diagram of an exemplary method for performing in accordance with an exemplary embodiment of the present disclosure. Although FIG. 14 shows steps performed in a particular order for purposes of illustration and explanation, the methods of the present disclosure are not limited to the order or arrangement specifically shown. Various steps of method 1400 may be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of the present disclosure.

[0131] At 1402, the computing system can provide an input draft interface for display. The input draft interface can include a graphical user interface including attribute options and a text entry box. The attribute options can be associated with candidate attributes for content item generation. The candidate attributes can include tone, style, length, content type, and / or other details. The input draft interface can include a preview window for viewing the current state of the graphics card. The graphics card can be associated with a linked note. In some implementations, the graphics card can include a card template that may have been modified based on one or more user inputs. For example, a user may have added images, text, audio, video, widgets, and / or other data.

[0132] At 1404, the computing system may obtain, via the input draft interface, a selection of a particular attribute option of the plurality of attribute options. The particular attribute option may be associated with a particular candidate attribute. In some implementations, the plurality of candidate attributes may include a plurality of different styles. The plurality of different styles may be associated with at least one of a plurality of different artistic styles or a plurality of different writing styles. Alternatively and / or additionally, the plurality of candidate attributes may include a plurality of different tones. The plurality of different tones may be associated with at least one of a plurality of different emotions and / or a plurality of different pace types. The particular candidate attribute may include a tone and / or style required to generate the content item. The selection may be obtained based on a selection of the particular attribute option from a drop-down menu providing a plurality of attribute options for display.

[0133] At 1406, the computing system can obtain text input via a text entry box of the input draft interface. The text input can be associated with a prompt intent for content item generation. In some implementations, the text input can be entered automatically based on the content of a graphics card, based on a user context, and / or based on a prompt suggestion.

[0134] At 1408, the computing system can process the particular attribute option and the text input with the generative model to generate a model-generated content item. The mode-generated content item can include a particular candidate attribute. The model-generated content item can be associated with a prompt intent. The model-generated content item can include text data, image data, audio data, multimodal data, and / or other data. In some implementations, the generative model can be retrieved from a generative model database based on the selection of the particular attribute option. For example, the generative model database can store a plurality of different generative models associated with a plurality of candidate attributes. Each of the plurality of different generative models can be configured, trained, and / or tuned to generate a content item associated with a respective candidate attribute. Alternatively and / or additionally, the generative model can be a general generative model trained for a plurality of content generation tasks. Additionally or alternatively, a particular attribute soft prompt can be retrieved based on the selection of the particular attribute option. The particular attribute soft prompt can include a set of learned parameters. The set of learned parameters can be processed with the generative model to generate the model-generated content item.

[0135] At 1410, the computing system can provide the model-generated content item for display via an input draft interface. Providing the model-generated content item for display can include providing an option to insert the model-generated content item into a graphics card. The input draft interface can include a number of post-processing editing options, which can include options to change size, color, font, cropping, resolution, saturation, tinting, and / or other details.

[0136] In some implementations, the computing system can obtain an input selection via the input draft interface and generate an enhanced graphics card based on the input selection. The enhanced graphics card can include a graphics card enhanced to include the model-generated content items. The computing system can then provide the enhanced graphics card for display.

[0137] 15A illustrates a block diagram of an example computing system 100 for performing linked note prompts, according to an example embodiment of the present disclosure. System 100 includes a user computing system 102, a server computing system 130, and / or a third computing system 150 communicatively coupled via a network 180.

[0138] The user computing system 102 may include any type of computing device, such as, for example, a personal computing device (e.g., a laptop or desktop), a mobile computing device (e.g., a smartphone or tablet), a game console or controller, a wearable computing device, an embedded computing device, or any other type of computing device.

[0139] The user computing system 102 includes one or more processors 112 and a memory 114. The one or more processors 112 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be a single processor or multiple processors operably connected. The memory 114 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 114 may store data 116 and instructions 118 that are executed by the processor 112 to cause the user computing system 102 to perform operations.

[0140] In some implementations, the user computing system 102 may store or include one or more machine learned models 120. For example, the machine learned models 120 may be or otherwise include various machine learned models, such as neural networks (e.g., deep neural networks), or other types of machine learned models, including non-linear and / or linear models. The neural networks may include feed-forward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks, or other types of neural networks.

[0141] In some implementations, one or more machine-learned models 120 may be received from server computing system 130 over network 180, may be stored in user computing device memory 114, and may then be used or otherwise implemented by one or more processors 112. In some implementations, user computing system 102 may implement multiple parallel instances of a single machine-learned model 120 (e.g., to perform parallel machine-learned model processing across multiple instances of input data and / or detected features).

[0142] More specifically, the one or more machine-learned models 120 may include one or more detection models, one or more classification models, one or more segmentation models, one or more augmentation models, one or more generative models, one or more natural language processing models, one or more optical property recognition models, and / or one or more other machine-learned models. The one or more machine-learned models 120 may include one or more transformer models. The one or more machine-learned models 120 may include one or more neural radiance field models, one or more diffusion models, and / or one or more autoregressive language models.

[0143] One or more machine-learned models 120 may be utilized to detect one or more object features. The detected object features may be classified and / or embedded. The classification and / or embedding may then be utilized to perform a search to determine one or more search results. Alternatively and / or additionally, one or more detected features may be utilized to determine whether an indicator (e.g., a user interface element indicating the detected feature) should be provided to indicate that the feature was detected. A user may then select an indicator to cause the feature to be classified, embedded, and / or searched. In some implementations, the classification, embedding, and / or search may occur before an indicator is selected.

[0144] In some implementations, the one or more machine-learned models 120 may process image data, text data, audio data, and / or latent encoding data to generate output data that may include image data, text data, audio data, and / or latent encoding data. The one or more machine-learned models 120 may perform optical character recognition, natural language processing, image classification, object classification, text classification, audio classification, context determination, action prediction, image enhancement, image enhancement, text enhancement, sentiment analysis, object detection, error detection, inpainting, video stabilization, audio correction, audio enhancement, and / or data segmentation (e.g., mask-based segmentation).

[0145] Additionally or alternatively, one or more machine-learned models 140 may be included in or otherwise stored and implemented by a server computing system 130 that communicates with the user computing system 102 according to a client-server relationship. For example, the machine-learned models 140 may be implemented by the server computing system 130 as part of a web service (e.g., a viewfinder service, a visual search service, an image processing service, an ambient computing service, and / or an overlay application service). Thus, one or more models 120 may be stored and implemented in the user computing system 102 and / or one or more models 140 may be stored and implemented in the server computing system 130.

[0146] The user computing system 102 may also include one or more user input components 122 that receive user input. For example, the user input component 122 may be a touch-sensitive component (e.g., a touch-sensitive display screen or touchpad) that is sensitive to the touch of a user input object (e.g., a finger or stylus). The touch-sensitive component may function to implement a virtual keyboard. Other exemplary user input components include a microphone, a traditional keyboard, or other means by which a user may provide user input.

[0147] In some implementations, the user computing system may store and / or provide one or more user interfaces 124 that may be associated with one or more applications. The one or more user interfaces 124 may be configured to receive input and / or provide data for display (e.g., image data, text data, audio data, one or more user interface elements, augmented reality experiences, virtual reality experiences, and / or other data for display). The user interfaces 124 may be associated with one or more other computing systems (e.g., the server computing system 130 and / or the third party computing system 150). The user interfaces 124 may include a viewfinder interface, a search interface, a generative model interface, a social media interface, and / or a media content gallery interface.

[0148] The user computing system 102 may include and / or receive data from one or more sensors 126. The one or more sensors 126 may be housed in a housing component that houses one or more processors 112, memory 114, and / or one or more hardware components that may store and / or perform one or more software. The one or more sensors 126 may include one or more image sensors (e.g., cameras), one or more lidar sensors, one or more audio sensors (e.g., microphones), one or more inertial sensors (e.g., inertial measurement units), one or more biological sensors (e.g., heart rate sensors, pulse sensors, retinal sensors, and / or fingerprint sensors), one or more infrared sensors, one or more position sensors (e.g., GPS), one or more touch sensors (e.g., conductive touch sensors and / or mechanical touch sensors), and / or one or more other sensors. The one or more sensors may be utilized to obtain data related to the user's environment (e.g., an image of the user's environment, a record of the environment, and / or the user's location).

[0149] The user computing system 102 may include and / or be part of a user computing device 104. The user computing device 104 may include a mobile computing device (e.g., a smartphone or tablet), a desktop computer, a laptop computer, a smart wearable, and / or a smart appliance. Additionally and / or alternatively, the user computing system may acquire data from and / or generate data using one or more of the user computing devices 104. For example, a smartphone camera may be utilized to capture image data describing the environment, and / or an overlay application of the user computing device 104 may be utilized to track and / or process data being provided to the user. Similarly, one or more sensors associated with a smart wearable may be utilized to acquire data about the user and / or about the user's environment (e.g., image data may be acquired by a camera housed in the user's smart glasses). Additionally and / or alternatively, data may be acquired and uploaded from other user devices that may be specialized in acquiring or generating data.

[0150] The server computing system 130 includes one or more processors 132 and a memory 134. The one or more processors 132 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be a single processor or multiple processors operably connected. The memory 134 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 134 may store data 136 and instructions 138 that are executed by the processor 132 to cause the server computing system 130 to perform operations.

[0151] In some implementations, server computing system 130 includes or is otherwise implemented by one or more server computing devices. When server computing system 130 includes multiple server computing devices, such server computing devices may operate according to a sequential computing architecture, a parallel computing architecture, or some combination thereof.

[0152] As described above, the server computing system 130 may store or otherwise include one or more machine-learned models 140. For example, the models 140 may be or otherwise include various machine-learned models. Exemplary machine-learned models include neural networks or other multi-layer nonlinear models. Exemplary neural networks include feed-forward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks. Exemplary models 140 are described with reference to FIG. 15B.

[0153] Additionally and / or alternatively, the server computing system 130 may include and / or be communicatively connected to a search engine 142 that may be utilized to crawl one or more databases (and / or resources). The search engine 142 may process data from the user computing system 102, the server computing system 130, and / or the third party computing system 150 to determine one or more search results associated with the input data. The search engine 142 may perform term-based searches, label-based searches, Boolean-based searches, image searches, embedding-based searches (e.g., nearest neighbor searches), multi-modal searches, and / or one or more other search techniques.

[0154] The server computing system 130 may store and / or provide one or more user interfaces 144 for obtaining input data and / or providing output data to one or more users. The one or more user interfaces 144 may include one or more user interface elements, which may include input fields, navigation tools, content tips, selectable tiles, widgets, data display carousels, dynamic animations, information popups, image augmentation, text to speech, speech to text, augmented reality, virtual reality, feedback loops, and / or other interface elements.

[0155] User computing system 102 and / or server computing system 130 can train models 120 and / or 140 through interaction with a third party computing system 150 that is communicatively coupled via network 180. Third party computing system 150 may be separate from server computing system 130 or may be part of server computing system 130. Alternatively and / or additionally, third party computing system 150 may be associated with one or more web resources, one or more web platforms, one or more other users, and / or one or more contexts.

[0156] The third party computing system 150 may include one or more processors 152 and memory 154. The one or more processors 152 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be a single processor or multiple processors operably connected. The memory 154 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 154 may store data 156 and instructions 158 that are executed by the processor 152 to cause the third party computing system 150 to perform operations. In some implementations, the third party computing system 150 includes or is otherwise implemented by one or more server computing devices.

[0157] Network 180 may be any type of communications network, such as a local area network (e.g., an intranet), a wide area network (e.g., the Internet), or some combination thereof, and may include any number of wired or wireless links. In general, communications over network 180 may occur over any type of wired and / or wireless connections, using a wide variety of communications protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or protection schemes (e.g., VPN, Secure HTTP, SSL).

[0158] The machine-learned models described herein may be used in a variety of tasks, applications, and / or use cases.

[0159] In some implementations, the input to the machine-learned model(s) of the present disclosure may be image data. The machine-learned model(s) may process the image data to generate an output. As an example, the machine-learned model(s) may process the image data to generate an image recognition output (e.g., recognition of the image data, latent embedding of the image data, an encoded representation of the image data, a hash of the image data, etc.). As another example, the machine-learned model(s) may process the image data to generate an image segmentation output. As another example, the machine-learned model(s) may process the image data to generate an image classification output. As another example, the machine-learned model(s) may process the image data to generate an image data correction output (e.g., alteration of the image data, etc.). As another example, the machine-learned model(s) may process the image data to generate an encoded image data output (e.g., an encoded and / or compressed representation of the image data, etc.). As another example, the machine-learned model(s) may process the image data to generate an upscaled image data output. As another example, the machine-learned model(s) may process the image data to generate a prediction output.

[0160] In some implementations, the input to the machine-learned model(s) of the present disclosure may be text or natural language data. The machine-learned model(s) may process the text or natural language data to generate an output. As an example, the machine-learned model(s) may process the natural language data to generate a language encoding output. As another example, the machine-learned model(s) may process the text or natural language data to generate a latent text embedding output. As another example, the machine-learned model(s) may process the text or natural language data to generate a transformation output. As another example, the machine-learned model(s) may process the text or natural language data to generate a classification output. As another example, the machine-learned model(s) may process the text or natural language data to generate a text segmentation output. As another example, the machine-learned model(s) may process the text or natural language data to generate a semantic output. As another example, the machine-learned model(s) may process the text or natural language data to generate an upscaled text or natural language output (e.g., text or natural language data that is of higher quality than the input text or natural language). As another example, the machine-learned model(s) may process the text or natural language data to generate a predicted output.

[0161] In some implementations, the input to the machine-learned model(s) of the present disclosure may be audio data. The machine-learned model(s) may process the audio data to generate an output. As an example, the machine-learned model(s) may process the audio data to generate a speech recognition output. As another example, the machine-learned model(s) may process the audio data to generate a speech conversion output. As another example, the machine-learned model(s) may process the audio data to generate a potential embedding output. As another example, the machine-learned model(s) may process the audio data to generate an encoded audio output (e.g., an encoded and / or compressed representation of the audio data, etc.). As another example, the machine-learned model(s) may process the audio data to generate an upscaled audio output (e.g., audio data of higher quality than the input audio data, etc.). As another example, the machine-learned model(s) may process the audio data to generate a text representation output (e.g., a text representation of the input audio data, etc.). As another example, the machine-learned model(s) may process the audio data to generate a predicted output.

[0162] In some implementations, the input to the machine-learned model(s) of the present disclosure may be sensor data. The machine-learned model(s) may process the sensor data to generate an output. As an example, the machine-learned model(s) may process the sensor data to generate a recognition output. As another example, the machine-learned model(s) may process the sensor data to generate a prediction output. As another example, the machine-learned model(s) may process the sensor data to generate a classification output. As another example, the machine-learned model(s) may process the sensor data to generate a segmentation output. As another example, the machine-learned model(s) may process the sensor data to generate a segmentation output. As another example, the machine-learned model(s) may process the sensor data to generate a visualization output. As another example, the machine-learned model(s) may process the sensor data to generate a diagnostic output. As another example, the machine-learned model(s) may process the sensor data to generate a detection output.

[0163] In some cases, the input includes visual data and the task is a computer vision task. In some cases, the input includes pixel data of one or more images and the task is an image processing task. For example, the image processing task may be image classification and the output is a set of scores, each score corresponding to a different object class and representing the likelihood that the one or more images depict an object belonging to that object class. The image processing task may be object detection and the image processing output identifies one or more regions of the one or more images and, for each region, the likelihood that the region depicts an object of interest. As another example, the image processing task may be image segmentation and the image processing output defines, for each pixel of the one or more images, a respective likelihood of each category in a set of predefined categories. For example, the set of categories may be foreground and background. As another example, the set of categories may be object classes. As another example, the image processing task may be depth estimation and the image processing output defines, for each pixel of the one or more images, a respective depth value. As another example, the image processing task may be motion estimation, where the network input includes multiple images and the image processing output defines, for each pixel of the input images, the motion of the scene represented in pixels between the images in the network input.

[0164] A user computing system may include several applications (e.g., Applications 1-N). Each application may include its own respective machine learning library and machine-learned model(s). For example, each application may include a machine-learned model. Exemplary applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, etc.

[0165] Each application can communicate with several other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, each application can communicate with each device component using an API (e.g., a public API). In some implementations, the API used by each application is specific to that application.

[0166] The user computing system 102 may include several applications (e.g., applications 1-N). Each application communicates with a central intelligence layer. Exemplary applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, etc. In some implementations, each application can communicate with the central intelligence layer (and the model(s) stored therein) using an API (e.g., a common API across all applications).

[0167] The central intelligence layer can include several machine-learned models. For example, each machine-learned model (e.g., a model) can be provided for each application and managed by the central intelligence layer. In other embodiments, two or more applications can share a single machine-learned model. For example, in some embodiments, the central intelligence layer can provide a single model (e.g., a single model) for all of the applications. In some embodiments, the central intelligence layer is included within or otherwise implemented by an operating system of computing system 100.

[0168] The central intelligence layer can communicate with a central device data layer, which may be a centralized repository of data for computing system 100. The central device data layer may also communicate with several other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, the central device data layer can communicate with each device component using an API (e.g., a private API).

[0169] FIG. 15B illustrates a block diagram of an exemplary computing system 50 that executes linked notes prompts, according to an exemplary embodiment of the present disclosure. In particular, the exemplary computing system 50 may include one or more computing devices 52 that may be utilized to acquire and / or generate one or more data sets that may be processed by the sensor processing system 60 and / or the output determination system 80 to provide feedback to the user that may provide information regarding characteristics of the one or more acquired data sets. The one or more data sets may include image data, text data, audio data, multimodal data, latent encoding data, and the like. The one or more data sets may be acquired via one or more sensors associated with the one or more computing devices 52 (e.g., one or more sensors of the computing device 52). Additionally and / or alternatively, the one or more data sets may be stored data and / or retrieved data (e.g., data retrieved from a web resource). For example, images, text, and / or other content items may be interacted with by the user. The interacted content items may then be utilized to generate one or more decisions.

[0170] The one or more computing devices 52 may acquire and / or generate one or more data sets based on image capture, sensor tracking, data storage retrieval, content download (e.g., downloading images or other content items over the Internet from a web resource), and / or via one or more other techniques. The one or more data sets may be processed in a sensor processing system 60. The sensor processing system 60 may perform one or more processing techniques using one or more machine-learned models, one or more search engines, and / or one or more other processing techniques. The one or more processing techniques may be performed in any combination and / or individually. The one or more processing techniques may be performed serially and / or in parallel. In particular, the one or more data sets may be processed in a context determination block 62, which may determine a context associated with one or more content items. The context determination block 62 can identify and / or process metadata, user profile data (e.g., preferences, user search history, user browsing history, user purchase history, and / or user input data), previous interaction data, global trend data, location data, time data, and / or other data to determine a particular context associated with the user. The context can be associated with events, determined trends, particular actions, particular types of data, particular environments, and / or other contexts associated with the user and / or retrieved or acquired data.

[0171] The sensor processing system 60 may include an image pre-processing block 64. The image pre-processing block 64 may be utilized to adjust one or more values ​​of the captured and / or received images to prepare the images for processing by one or more machine-learned models and / or one or more search engines 74. The image pre-processing block 64 may resize the images, adjust saturation values, adjust resolution, remove and / or add metadata, and / or perform one or more other operations.

[0172] In some implementations, the sensor processing system 60 may include one or more machine-learned models, which may include a detection model 66, a segmentation model 68, a classification model 70, an embedding model 72, and / or one or more other machine-learned models. For example, the sensor processing system 60 may include one or more detection models 66 that can be utilized to detect particular features in a processed dataset. In particular, one or more images may be processed with the one or more detection models 66 to generate one or more bounding boxes associated with features detected in the one or more images.

[0173] Additionally and / or alternatively, one or more segmentation models 68 may be utilized to segment one or more portions of a dataset from one or more datasets. For example, one or more segmentation models 68 may utilize one or more segmentation masks (e.g., one or more segmentation masks generated manually and / or based on one or more bounding boxes) to segment portions of an image, portions of an audio file, and / or portions of text. Segmentation may include isolating one or more detected objects and / or removing one or more detected objects from an image.

[0174] One or more classification models 70 may be utilized to process image data, text data, audio data, latent encoding data, multimodal data, and / or other data to generate one or more classifications. The one or more classification models 70 may include one or more image classification models, one or more object classification models, one or more text classification models, one or more audio classification models, and / or one or more other classification models. The one or more classification models 70 may process the data to determine one or more classifications.

[0175] In some implementations, the data may be processed with one or more embedding models 72 to generate one or more embeddings. For example, one or more images may be processed with one or more embedding models 72 to generate one or more image embeddings in the embedding space. The one or more image embeddings may be associated with one or more image features of the one or more images. In some implementations, the one or more embedding models 72 may be configured to process multimodal data to generate multimodal embeddings. The one or more embeddings may be utilized for classification, retrieval, and / or learning embedding spatial distributions.

[0176] The sensor processing system 60 may include one or more search engines 74 that can be utilized to perform one or more searches. The one or more search engines 74 can crawl one or more databases (e.g., one or more local databases, one or more global databases, one or more private databases, one or more public databases, one or more proprietary databases, and / or one or more general databases) to determine one or more search results. The one or more search engines 74 may perform feature matching, text-based search, embedding-based search (e.g., k-nearest neighbor search), meta-database search, multi-modal search, web resource search, image search, text search, and / or application search.

[0177] Additionally and / or alternatively, the sensor processing system 60 may include one or more multimodal processing blocks 76 that may be utilized to assist in processing the multimodal data. The one or more multimodal processing blocks 76 may include generating multimodal queries and / or multimodal embeddings that are processed by one or more machine-learned models and / or one or more search engines 74.

[0178] The output(s) of the sensor processing system 60 may then be processed by an output determination system 80 to determine one or more outputs to provide to a user. The output determination system 80 may include heuristic-based decisions, machine-learned model-based decisions, user-selection-based decisions, and / or context-based decisions.

[0179] The output determination system 80 may determine how and / or where to provide the one or more search results in the search result interface 82. Additionally and / or alternatively, the output determination system 80 may determine how and / or where to provide the output of the one or more machine-learned models in the machine-learned model output interface 84. In some implementations, the one or more search results and / or the output of the one or more machine-learned models may be provided for display via one or more user interface elements. The one or more user interface elements may be overlaid on the displayed data. For example, one or more detection indicators may be overlaid on the detected object in the viewfinder. The one or more user interface elements may be selectable to perform one or more additional searches and / or one or more additional machine-learned model processes. In some implementations, the user interface elements may be provided as user interface elements dedicated to a particular application and / or may be provided uniformly across different applications. The one or more user interface elements may include pop-up displays, interface overlays, interface tiles and / or chips, carousel interfaces, audio feedback, animations, interactive widgets, and / or other user interface elements.

[0180] Additionally and / or alternatively, data associated with the output(s) of the sensor processing system 60 may be utilized to generate and / or provide an augmented reality experience and / or a virtual reality experience 86. For example, one or more acquired datasets may be processed to generate one or more augmented reality rendering assets and / or one or more virtual reality rendering assets, which may then be utilized to provide the augmented reality experience and / or a virtual reality experience 86 to the user. The augmented reality experience may render information related to an environment into the respective environment. Alternatively and / or additionally, objects associated with the processed dataset(s) may be rendered within the user environment and / or the virtual environment. Generating the rendering dataset may include training one or more neural radiance field models to learn a three-dimensional representation of one or more objects.

[0181] In some implementations, one or more action prompts 88 may be determined based on the output(s) of the sensor processing system 60. For example, a search prompt, a purchase prompt, a create prompt, a reservation prompt, a call prompt, a redirect prompt, and / or one or more other prompts may be determined to be associated with the output(s) of the sensor processing system 60. The one or more action prompts 88 may then be provided to the user via one or more selectable user interface elements. In response to selection of the one or more selectable user interface elements, a respective action of the respective action prompt may be performed (e.g., a search may be performed, a purchasing application programming interface may be utilized, and / or other applications may be opened).

[0182] In some implementations, one or more datasets and / or output(s) of sensor processing system 60 may be processed by one or more generative models 90 to generate model-generated content items, which may then be provided to a user. Generation may be prompted based on user selection and / or may be automatic (e.g., automatically based on one or more conditions that may be associated with a threshold amount of unidentified search results).

[0183] The one or more generative models 90 may include language models (e.g., large language models and / or visual language models), image generation models (e.g., text-to-image generation models and / or image augmentation models), audio generation models, video generation models, graph generation models, and / or other data generation models (e.g., other content generation models). The one or more generative models 90 may include one or more transformer models, one or more convolutional neural networks, one or more recurrent neural networks, one or more feedforward neural networks, one or more generative adversarial networks, one or more self-attention models, one or more embedding models, one or more encoders, one or more decoders, and / or one or more other models. In some implementations, the one or more generative models 90 may include one or more autoregressive models (e.g., machine-learned models trained to generate predictions based on previous behavioral data) and / or one or more diffusion models (e.g., machine-learned models trained to generate predictions based on generating and processing distribution data associated with input data).

[0184] One or more generative models 90 may be trained to process input data and generate model-generated content items, which may include predicted words, pixels, signals, and / or other data. The model-generated content items may include new content items that are not identical to any existing work. The one or more generative models 90 may leverage learned representations, sequences, and / or probability distributions to generate content items, which may include phrases, storylines, settings, objects, characters, beats, lyrics, and / or other aspects not included in existing content items.

[0185] The one or more generative models 90 may include a visual language model.

[0186] A visual language model can be trained, tuned, and / or configured to process image data and / or text data to generate natural language output. A visual language model can leverage a pre-trained large-scale language model (e.g., a large-scale autoregressive language model) with one or more encoders (e.g., one or more image encoders and / or one or more text encoders) to provide fine-grained natural language output that emulates natural language produced by humans.

[0187] The visual language models may be utilized for zero-shot image classification, few-shot image classification, image captioning, multimodal query distillation, multimodal question answering, and / or may be tuned and / or trained for multiple different tasks. The visual language models may perform visual question answering, image caption generation, feature detection (e.g., content monitoring (such as inappropriate content)), object detection, scene recognition, and / or other tasks.

[0188] The visual language model may leverage a pre-trained language model, which may then be tuned for multimodal use. Training and / or tuning of the visual language model may include image-text matching, masked language modeling, multimodal fusion with cross-attention, contrastive learning, prefix language model training, and / or other training techniques. For example, the visual language model may be trained to process images and generate predictive text similar to ground truth text data (e.g., ground truth captions of images). In some implementations, the visual language model may be trained to replace masked tokens of natural language templates with text tokens that describe features depicted in the input image. Alternatively and / or additionally, the training, tuning, and / or model estimation may include multi-layer concatenation of visual and text embedding features. In some implementations, the visual language model may be trained and / or tuned by jointly learning image embeddings and generation of text embeddings, which may include training and / or tuning a system that maps embeddings to a joint feature embedding space that maps text features and image features to a shared embedding space. The joint training may include parallel embedding of image-text pairs and / or may include triplet training. In some implementations, images may be used and / or processed as prefixes to a language model.

[0189] The output determination system 80 may process one or more data sets and / or the output(s) of the sensor processing system 60 using a data augmentation block 92 to generate augmented data. For example, one or more images may be processed in the data augmentation block 92 to generate one or more augmented images. Data augmentation may include data correction, data cropping, removal of one or more features, addition of one or more features, resolution adjustment, illumination adjustment, saturation adjustment, and / or other enhancements.

[0190] In some implementations, one or more data sets and / or output(s) of the sensor processing system 60 may be stored based on the determination in the data storage block 94.

[0191] The output(s) of the output determination system 80 may then be provided to a user via one or more output components of the user computing device 52. For example, one or more user interface elements associated with the one or more outputs may be provided for display via a visual display of the user computing device 52.

[0192] The process may be performed iteratively and / or continuously: one or more user inputs to provided user interface elements may condition and / or affect the continuous processing loop.

[0193] The technology described herein refers to servers, databases, software applications, and other computer-based systems, as well as actions performed on and information sent to and from such systems. The inherent flexibility of computer-based systems allows for a wide variety of configurations, combinations, and divisions of tasks and functions among components. For example, the processes discussed herein can be implemented using a single device or component, or multiple devices or components working in combination. Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0194] Although the subject matter of the present disclosure has been described in detail with respect to various specific and exemplary embodiments thereof, each example is provided for the purpose of illustration and not for the purpose of limiting the present disclosure. Those skilled in the art, upon understanding the above content, can easily make modifications, variations, and equivalents of such embodiments. Thus, the present disclosure does not exclude the inclusion of such modifications, changes, and / or additions to the subject matter as would be readily apparent to those skilled in the art. For example, features illustrated or described as part of one embodiment can be used with other embodiments to produce yet other embodiments. Thus, the present disclosure is intended to cover such modifications, variations, and equivalents.

Claims

1. 1. A computing system for comment prompt generation and input retrieval, the system comprising: one or more processors; and one or more non-transitory computer-readable storage media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations, the operations including: Obtaining content data, the content data being associated with a web resource, the web resource including one or more content items; processing the content data with a generative model to generate predictive prompts, the predictive prompts including predictive text strings associated with commenting on the web resource; providing the predictive prompts for display on an input prompt interface, the input prompt interface configured to receive input; obtaining comment input data from a user computing system via the input prompt interface, the comment input data including user-generated comments on the web resource; obtaining user data, the user data being associated with a particular user, the user computing system being associated with the particular user; generating a graphics card based on the user data, the content data and the comment input data, the graphics card including a user profile identifier for the particular user and data associated with the comment input data, the graphics card including a graphical background generated with an image generation model based on the comment input data; storing the graphics card and data associated with the comment entry data together with data associated with the web resource, the data associated with the comment entry data being stored in a searchable database that is provided for display in response to the web resource being provided as a search result; A computing system comprising:

2. 2. The system of claim 1, wherein the user data includes user search history data, and the generative model generates the predictive prompts based on the particular user's previous searches for information associated with a topic of the web resource.

3. 2. The system of claim 1, wherein the user data includes user browser history data, and the generative model generates the predictive prompts based on the particular user's previous viewing of other web resources that contain information associated with a topic of the web resource.

4. The operation includes: Obtaining a search query; and determining that the web resource is associated with the search query; providing specific search results for display, the specific search results including a link to the web resource, a title of the web resource, and data associated with the comment input data; The system of claim 1 further comprising:

5. Storing the data associated with the comment entry data with the data associated with the web resource includes: generating a web resource note; and storing the web resource note with a plurality of other web resource notes associated with the web resource.

6. The operation, providing the web resource note and the plurality of other web resource notes to a note interface that provides the web resource note and the plurality of other web resource notes to a plurality of graphic cards; The system of claim 5 , further comprising:

7. The operation comprises: obtaining a selection of attributes requested to enhance said user-generated comments; processing the user-generated comments and data describing the requested attributes together with the Generative Model to generate a model-generated content item; extending said graphics card to include said model-generated content items; The system of claim 1 further comprising:

8. The operation comprises: processing the graphics card to determine one or more entity tags based on the determined topic of the graphics card; accessing a media content item database to obtain one or more additional media content items based on the one or more entity tags; providing said one or more additional media content items for display; and The system of claim 1 further comprising:

9. The operation comprises: providing a graphics card customization interface for display, the graphics card customization interface including a plurality of options for editing the graphics card. The system of claim 1.

10. The system of claim 1 , wherein the generative model comprises an autoregressive language model, and the generative model is prompted to generate a question that describes a request for information about the web resource.

11. 1. A computer-implemented method for linked note prompts, the method comprising: obtaining, by a computing system including one or more processors, context data associated with a particular content presentation instance, the particular content presentation instance including a particular user viewing a particular content item of a particular web resource; determining, by the computing system, a prompting action based on the context data, the prompting action including providing an input entry interface to a user for obtaining user input; processing, by the computing system, the contextual data with a generative language model to generate a predictive prompt, the predictive prompt including a natural language request for information generated based on the contextual data; providing, by the computing system, the predictive prompt in the input entry interface; obtaining, by the computing system, user-generated content via the input entry interface; obtaining, by the computing system, user data, the user data being associated with a particular user, and the user-generated content being associated with the particular user; generating, by the computing system, a graphics card based on the user data and the user-generated content, the graphics card including a user profile identifier for the particular user and data associated with the user-generated content, the graphics card including a graphical background generated with an image generation model based on the user-generated content; generating, by the computing system, a link note including the graphics card based on the user-generated content, the link note being generated such that the link note is provided for display in a search result interface in response to the particular content item being determined as a search result. method.

12. The method of claim 11 , wherein the contextual data is associated with a type of content being offered for display.

13. The method of claim 11 , wherein the contextual data is associated with the particular user associated with the particular content display instance, and the contextual data includes search history data.

14. 12. The method of claim 11, wherein the content being offered for display is associated with a particular web resource, the contextual data is associated with interaction data of links of the particular web resource across multiple social network platforms, and the input prompt action is determined based on the interaction data.

15. 12. The method of claim 11, wherein the contextual data includes user data and content data, and the prompt action is determined based on a topic associated with content being offered for display being one of a plurality of topics about which the particular user is determined to have knowledge based on the user data.

16. The method of claim 11 , wherein the contextual data includes previous notes generated by the particular user, and the predictive prompts include a structure based on previous structures for the previous notes.

17. One or more non-transitory computer-readable storage media that collectively store instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations, the operations including: Obtaining a first search query at a first time; determining that a web resource is responsive to the first search query, the web resource including one or more content items; obtaining content data, the content data being associated with the web resource; processing the content data with a generative model to generate predictive prompts, the predictive prompts including predictive text strings associated with commenting on the web resource; providing the predictive prompt for display within an input prompt interface, the input prompt interface including an input entry box; obtaining comment input data from a user computing system via the input prompt interface, the comment input data including user generated content; obtaining user data, the user data being associated with a particular user, the user computing system being associated with the particular user; generating a graphics card based on the user data, the content data and the comment input data, the graphics card including a user profile identifier for the particular user and data associated with the comment input data, the graphics card including a graphical background generated with an image generation model based on the comment input data; storing said graphics card; obtaining a second search query at a second time, the second time being different from the first time; determining that the web resource is responsive to the second search query; and providing data describing the web resource to the graphics card of a search result interface. One or more non-transitory computer-readable storage media.

18. 20. The one or more non-transitory computer-readable storage media of claim 17, wherein the first search query and the second search query are different.

19. 20. The one or more non-transitory computer-readable storage media of claim 17, wherein the comment input data comprises multi-modal data.

20. 20. The one or more non-transitory computer-readable storage media of claim 19, wherein the multimodal data comprises textual data and image data.

Images