Supplemental content and generative language model

Abstract

A computing system includes a processor and a memory storing instructions that, when executed by the processor, cause the processor to perform a number of actions. The actions include providing cues to the generative language model, where the generative language model generates an output based on the cues, identifies text to be associated with the supplemental content item in the output, and assigns a hyperlink to the text in the output. Upon selecting the hyperlink or hovering on the hyperlink, the supplemental content item is displayed.

Description

Background Art

[0001] Supplementary content is typically presented alongside the primary content on a web page. In an example, a search engine results page (SERP) typically includes hyperlinks to web pages (or web applications) identified as relevant to a received query, knowledge cards that provide information about entities referenced in the query, instant answers to questions raised in the query, and supplementary content. In some embodiments, the supplementary content is sponsored search results. In another example, a publisher publishes content on a web page. A web page may also include supplementary content that is relevant to the content of the web page and / or relevant to the interests of viewers of the web page.

[0002] Recently, generative models including generative language models (GLMs), also known as large language models (LLMs), have been developed. An example of a GLM is the Generative Pretrained Transformer 4 (GPT-4). Another example of a GLM is the BigScience Language Open Science Open Access Multilingual (BLOOM) model, which is also a transformer-based model. In short, a generative model is configured to generate an output (such as text in a human language, source code, music, video, etc.) based on a prompt word, where the prompt word typically includes input proposed by a user. The generative model generates output in near real time (e.g., within seconds of receiving the prompt word). The generative model also generates output based on training data that has been used to train the generative model.

[0003] Currently, practical applications of generative models are quite limited. For example, users of generative models often request that they generate poetry, provide biographical information about celebrities, or provide information about topics of interest to the user. Generative models have not been used to improve the provision of supplementary content to search engine users and / or web page viewers. Furthermore, to date, technologies configured to provide supplementary content to end users have not been meaningfully integrated with generative models. Summary of the Invention

[0004] The following is a brief summary of the subject matter that is described in greater detail herein. This summary is not intended to limit the scope of the claims.

[0005] This document describes various techniques for utilizing a generative model related to providing supplemental content to viewers of a web page (or web application). As used herein, the term "web page" is intended to cover both conventional web pages and web applications. The computing environment described herein includes a computing system and a client computing device that communicates with the computing system over a network. The computing system executes a generative model and a supplemental content provisioning system. The computing system also optionally executes a search engine. The supplemental content provisioning system identifies supplemental content to be provided to viewers of the web page on a web page (where the web page can be a search engine results page (SERP) or some other appropriate web page). As will be described in more detail herein, a generative model is employed to improve the provision of supplemental content to viewers of the web page.

[0006] In a non-limiting example, a search engine receives a query from a client computing device and generates a search engine results page (SERP) based on the query. A generative model is integrated with the search engine, and an interface for the generative model can be included in the SERP. Thus, a user of the client computing device can provide input to the generative model through the interface in the SERP. In addition, the generative model can be provided with at least some of the information in the SERP as input (as part of a prompt word), so that the output generated by the generative model is based on the content of the SERP.

[0007] The input provided to the generative model (e.g., by a user or from information in a SERP) can be related to a specific entity, such as a company that offers goods or services for acquisition, products available through the company, services available through the company, etc. The generative model can identify that the input is related to the company and can identify a second generative model unique to the company (e.g., trained to provide output related to the company, including information about products available from the company, offers from the company, etc.). The second generative model can then provide output about the entity to the user. In this example, the output of the second generative model can be supplemental content. Various other examples of providing supplemental content based on the output of a generative model are described herein.

[0008] The above summary presents a simplified summary of the invention in order to provide a basic understanding of some aspects of the systems and / or methods discussed herein. This summary is not an extensive overview of the systems and / or methods discussed herein. It is not intended to identify key / critical elements or to delineate the scope of such systems and / or methods. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that will be presented later. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a functional block diagram of a computing environment that facilitates supplying supplemental content based on output of a generative model.

[0010] Figure 2 is a schematic diagram depicting a graphical user interface (GUI) corresponding to a generative model unique to a company.

[0011] Figure 3 is a flow chart illustrating a method for providing an output generated by a generative model to a user, where the generative model is selected from among several generative models.

[0012] Figure 4 is a flow chart illustrating a method for providing output generated by a generative model to a user, where the generative model corresponds to a website.

[0013] Figure 5 is a schematic diagram depicting a GUI, during a session between a user and a generative model, identifying a second generative model, and further, wherein the second generative model continues the session with the user.

[0014] Figure 6 is a flow chart illustrating a method for presenting the output of a generative model to a user.

[0015] Figure 7 is a schematic diagram depicting a GUI in which hyperlinks are assigned to portions of output generated by a generative model, and further in which supplementary content is presented when a pointer is hovered over the hyperlink.

[0016] Figure 8 is a flow chart illustrating a method for assigning hyperlinks to text in an output of a generative model, where the hyperlinks correspond to supplemental content items.

[0017] Figure 9 is a schematic diagram depicting supplemental content related to a company, where the supplemental content includes information generated by a generative model.

[0018] Figure 10 is a flow chart illustrating a method for updating a supplemental content item to include content generated by a generative model.

[0019] Figure 11 is a schematic diagram depicting a GUI for a SERP, wherein the GUI includes supplemental content, and further, wherein the supplemental content includes an interface for interacting with a generative model.

[0020] Figure 12 is a schematic diagram depicting a GUI corresponding to a generative model, wherein supplementary content is inserted into a conversation between a user and the generative model.

[0021] Figure 13 An example computing device is depicted. DETAILED DESCRIPTION

[0022] Various techniques for utilizing a generative model (such as a GLM) for providing supplementary content on a web page will now be described with reference to the accompanying drawings, wherein like reference numerals are used to refer to like elements throughout. In the following description, for the purpose of explanation, many specific details are provided to provide a thorough understanding of one or more aspects. However, it will be apparent that such (multiple) aspects may be put into practice without these specific details. In other examples, well-known structures and devices are shown in block diagram form to facilitate description of one or more aspects. Further, it will be understood that the functionality described as being implemented by certain system components may be performed by multiple components. Similarly, for example, components may be configured to perform the functionality described as being implemented by multiple components.

[0023] Furthermore, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless specified otherwise or clear from the context, the phrase "X employs A or B" is intended to mean any of the natural inclusive permutations. That is, the phrase "X employs A or B" is satisfied by any of the following instances: X employs A; X employs B; or X employs both A and B. In addition, the articles "a" and "an" used in this application and the appended claims should generally be construed to mean "one or more" unless specified otherwise or clear from the context to be directed to a singular form.

[0024] Further, as used herein, the terms "component," "system," "engine," and "module" are intended to encompass a computer-readable data storage device that is configured with computer-executable instructions that, when executed by a processor, cause specific functionality to be performed. Computer-executable instructions may include routines, functions, etc. It is also understood that a component or system may be located on a single device or distributed across several devices.

[0025] The technology described herein relates to providing supplemental content to viewers of a web page based on the output of a generative model. In an example, the generative model can be a "sponsored" generative model assigned to a specific company, where the generative model is trained to engage in conversation with the user about products and / or services offered for acquisition by the company. In this case, the generative model can access a database corresponding to the company, can access the company's web page, etc. Thus, in this example, the output generated by the generative model is supplemental content. In another example, the generative model is trained to generate text related to the product or service referenced in the supplemental content. This text can be based on content from a web page on the website of the company that offers the product or service for acquisition. Various other examples are presented in greater detail herein.

[0026] Now refer to Figure 1 , illustrates a functional block diagram of a computing environment 100. Computing environment 100 includes a computing system 102 and a client computing device 104, wherein computing system 102 and client computing device 104 communicate with each other via a network 106. Computing system 102 includes a processor 108 and a memory 110, wherein memory 110 stores instructions executed by processor 108. Specifically, memory 110 includes a supplemental content provisioning system 112, which is configured to identify supplemental content to be presented to a user on a webpage (e.g., a SERF or other suitable webpage). Memory 110 also includes a generative model 114 that generates output based on a prompt word. Compared to conventional techniques, the prompt word provided to generative model 114 may also include information in addition to user-provided input. For example, the prompt word may include information obtained by a search engine based on user input, content displayed on a webpage, and so on. This additional information may be automatically provided to generative model 114 (e.g., without user intervention). Memory 110 also includes a search engine 116 that searches based on a query. The query may be generated by the user or by generative model 114 based on user input. For example, the generative model 114 receives user input and constructs a query; the generative model 114 provides the query to the search engine 116, which then performs a search based on the query. At least a portion of the search results identified by the search engine 116 can be provided to the generative model 114, which can then generate output based on this information. Thus, the prompts provided to the generative model 114 can include, but are not limited to: 1) user input presented to the generative model 114 and / or the search engine 116; 2) a query generated by the generative model 114 based on the user input; 3) information extracted from the search results identified by the search engine 116 based on the user input and / or the query; 4) information from previous prompts used by the generative model 114 to generate output; and 5) previous output of the generative model 114.

[0027] The computing system 102 additionally includes a data repository 118. The data repository 118 stores supplemental content 120 and user history data 122. For example, the supplemental content provisioning system 112 can identify supplemental content from the supplemental content 120 based at least in part on information in the user history 122. Further, as will be described in greater detail below, the generative model 114 can generate output based on the user history 122. The supplemental content provisioning system 112 can use this output to identify and provide supplemental content for presentation to the user on a webpage.

[0028] Various graphical user interfaces associated with generative models and / or web pages are presented below. Where appropriate, reference will be made to computing environment 100 with respect to these GUIs.

[0029] Now refer to Figure 2 , illustrates a GUI 200 for a company-specific generative model (e.g., the generative model is trained on company-identified training data and / or accesses the company's database). GUI 200 can be presented on a SERP and can also be displayed on a webpage of a company's website. According to an example, client computing device 104 can receive user input from a user (not shown) and send the user input to search engine 116. Supplemental content provisioning system 112 identifies supplemental content based on the user input, where the supplemental content corresponds to the company. Supplemental content 204 identified by supplemental content provisioning system 112 is included in GUI 200. That is, supplemental content provisioning system 112 can wrap supplemental content 201 with a company-specific GUI feature, where the GUI feature includes an input field 202 through which a user can provide input to the company-specific generative model. Thus, as illustrated, the GUI 200 includes supplemental content 204 identified by the supplemental content provisioning system 112 and an input field 202 through which a user can interact with a generative model that is unique to the company (e.g., the generative model accesses the company's database, is specifically trained to interact with the company's customers, etc.).

[0030] Additionally, when a browser executing on the client computing device 104 loads the company's webpage, the GUI 200 can be presented. Thus, even if the browser is not displaying a SERP, the GUI 200 can be presented to the user on the webpage. Thus, the techniques described herein provide the user with the same experience when interacting with the generative model, regardless of whether the user is interacting with the generative model through a SERP or through the company's webpage. Furthermore, it is contemplated that the generative model 114 can generate insights about the company from supplemental content 204 identified by the supplemental content provisioning system 112, as well as identify and summarize information from the company's website, so that the information provided to the user is personalized based on the user's preferences. Thus, the generative model is integrated on both the SERP and the company's webpage, whereby the generative model can generate insights and summary content for the user.

[0031] Furthermore, the generative model can be a distilled model that can help users navigate the pages of a company's website. For example, when a user is searching for faucets, the distilled generative model 114 can provide insights, recommendations, complementary products, etc. to help the user when searching. Compared to a general generative model, the distilled model has limited capabilities and is only focused on the company's products. Figure 3, depicts a flow diagram illustrating a method 300 for presenting supplemental content to a user. The method 300 begins at 302 and receives user input from a client computing device at 304; in an example, the user input is received at a search engine and is in the form of a query including keywords.

[0032] At 306, supplemental content from the provider is identified based on the user input received at 304 and / or based on search results identified by the search engine based on the query. In one example, the supplemental content may be identified based on the results of a keyword auction. In another example, the supplemental content may be identified based on entities identified in the search results returned by the search engine.

[0033] At 308, a generative model assigned to the provider is identified; for example, the generative model can be associated with supplemental content and / or the provider. The generative model is trained based on information identified by the provider (e.g., the content of the provider's website(s), information in the provider's database, etc.). At 310, a GUI for the generative model is presented on the SERP. The SERP includes, for example, links to web pages identified by the search results as relevant to the user input, information cards related to the user input, and a GUI for the generative model. The GUI for the generative model can include supplemental content; in other words, the supplemental content is packaged in the GUI for the generative model. The GUI for the generative model includes an input field through which a user can provide input related to the supplemental content and / or the provider. In another example, the GUI for the generative model includes a button that, when selected, allows voice input to be captured by a microphone of the client computing device and converted into text input that can be presented to the generative model.

[0034] At 312, user input is received via a user input field. At 314, the generative model generates output and presents the output in a GUI of the generative model. The generative model generates output based on a prompt word, which includes the user input received via the user input field. In addition, the prompt word may also include information from supplemental content presented in the GUI. Further, the prompt word may also include information from search results identified by a search engine. Moreover, the prompt word may include default information proposed by the provider. Furthermore, the prompt word may optionally include information from a user profile, such as historical information of the user, identified user preferences, etc. Method 300 completes at 316.

[0035] Now refer to Figure 4, illustrates a flow chart illustrating a method for presenting information generated by a generative model. Method 400 begins at 402, and at 404, an indication is received that a webpage of a company website has been accessed by a user of a client computing device. In an example, an indication is received that a web browser executing on the client computing device has loaded the webpage. In another example, an indication is received that an application (other than a web browser) has accessed the webpage to retrieve webpage content. For example, the application may be an application for a company and may be used to browse and / or obtain products or services offered by the company.

[0036] At 406, a GUI for the generative model for the company is presented on a webpage, wherein the GUI includes an input field through which user input can be received. In one example, the computing system executes the generative model in addition to several other generative models from several other providers, and the computing system selects a GUI for the company from among several potential GUIs based on the website identifier and / or the provider identifier. At 408, user input is received via the input field. For example, a web server sends the user input to the computing system, and a prompt word provided to the generative model includes the user input. The prompt word may additionally include the content of the webpage. The prompt word may optionally include the content of other webpages on the website. At 410, the generative model generates output based on the prompt word, and at 412, the output is presented in the GUI of the generative model on the webpage. For example, the output is sent from the computing system to the webserver, which inserts the output into the GUI of the generative model displayed on the webpage. In another example, the output is sent from the computing system to a client computing device, whereupon the output is displayed in the GUI of the generative model. Method 400 completes at 414.

[0037] Now refer to Figure 5, illustrates a GUI 500 corresponding to a generative model, wherein the generative model is specific to a company (provider). In the example, the generative model 114 included in the memory 110 is a general generative model. Further, although not illustrated, the memory 110 may include several other generative models, each of the generative models corresponding to a respective company. The GUI 500 includes an input field 202, whereby a user may provide input to the generative model. In the example, initially, user input is provided to the generative model 114, such as the input "I want to book a flight." The generative model 114 may construct an output (not presented to the user) based on the user input, wherein the output may identify entities related to the user input, wherein the output may be a query based on the user input, and so on. The output generated by the generative model may be provided to the supplemental content provision system 112, which may identify supplemental content and / or a provider of the supplemental content based on the output of the generative model 114. According Figure 5 In the depicted example, the output generated by generative model 114 indicates that the user's intent is to travel, and supplemental content provisioning system 112 is provided with this output. Supplemental content provisioning system 114 identifies a provider of travel services (e.g., an airline) and further identifies a second generative model (different from generative model 114) assigned to the provider. As previously described, the second generative model is trained based on information identifying the provider, accessing the provider's database, etc. The second generative model generates output based on a prompt term, where the prompt term includes user input provided via input field 202. The prompt term may additionally include other information, such as preference information in the user profile, search results identified by search engine 110 based on the user input, etc. The output generated by the second generative model may be displayed in GUI 500 along with the output generated by generative model 114. Subsequent user input may be provided to the second generative model, and the second generative model may generate subsequent output based on the subsequent user input.

[0038] As illustrated, the second generative model may generate the output "Hello - I'm an agent for a company. Where are you going?" In this example, the second generative model and / or the content provided by the second generative model may be supplemental content. In this example, the company corresponding to the second generative model is charged for the conversation between the user and the second generative model. After the user provides input indicating the completion of the conversation (e.g., requesting a new topic), the user input is provided to the generative model 114.

[0039] Go to Figure 6, presents a method 600 performed by a computing system executing multiple generative models. Method 600 begins at 602 and receives user input at a first generative model at 604. At 606, the first generative model generates an output based on the user input (and optionally based on other information, such as information extracted from search results identified by a search engine based on the user input and / or a query generated by the first generative model based on the user input). At 608, the output is provided to a supplemental content provision system, and at 610, the supplemental content provision system identifies a second generative model based on the output generated by the first generative model. As previously mentioned, the second generative model can be assigned to a provider and can be trained based on information associated with the provider and can access the provider's data. At 612, the second generative model generates a second output based on the user input received at 604, wherein the supplemental content provision system provides the user input to the second generative model. The second output is presented to the user via a GUI. Method 600 completes at 614.

[0040] Now refer to Figure 7 , another GUI 700 corresponding to the generative model 114 is presented. At the example GUI 700, the user is engaging in a conversation with the generative model 114 about cities to travel to in Greece. The generative model 114 generates output based on a prompt word, where the prompt word comprises input proposed by the user of the client computing device 104. Further, as described above, the generative model 114 can generate output based on search results identified by the search engine 116, where the search engine 116 identifies the search results based on the received user input and / or a query generated by the generative model 114 based on the received user input.

[0041] In addition, the output of generative model 114 can be analyzed to determine whether the content in the output corresponds to a supplementary content item (or a provider of a supplementary content item). In an example, generative model 114 (or another instance of generative model 114) can analyze the output and identify text corresponding to the supplementary content item. In another example, a second generative model is trained to identify text corresponding to the supplementary content item. In yet another example, the output generated by generative model 114 is provided to supplementary content supply system 112, and supplementary content supply system 112 identifies the supplementary content item based on the output. In yet another example, generative model 114 (or another instance of generative model 114) can generate a summary of the output of generative model 114 and provide the summary to supplementary content supply system 112. Supplementary content supply system 112 can identify the supplementary content item based on the summary.

[0042] According to an example, upon determining that text in the output corresponds to a supplemental content item, the generative model 114 may assign a hyperlink to such text (underlined for presentation to the user), wherein when the user hovers over the hyperlinked text, the identified supplemental content item 902 is presented. When the user selects the hyperlinked text in the conversation output, or when the user selects the supplemental content item 902, the web browser may load the webpage corresponding to the supplemental content item 902.

[0043] There are several methods that can be used to identify and highlight text corresponding to supplemental content items in the output. In a first example, the generative model 114 generates output based on input provided by a user of the client computing device 104 and, optionally, search results identified by a search engine 116 and presented to the user (based on a query provided by the user to the search engine 116), or based on search results identified by the search engine 116 based on input provided to the generative model 114 or a query generated by the generative model 114. Notably, the output is not based on supplemental content items. The session output generated by the generative model 114 is then provided to a second generative model that analyzes the output for text that may correspond to supplemental content items.

[0044] In the second example, generative model 114 is provided with user input, search results identified by search engine 116, and supplemental content item information. However, the supplemental content item information is marked as such in the prompt provided to generative model 114, and generative model 114 does not generate conversational output based on information about the supplemental content items. Once generative model 114 generates output, generative model 114 can incorporate information related to the supplemental content items in conjunction with identifying text corresponding to one or more supplemental content items in the conversational output. This approach eliminates the need to call another generative model because generative model 114 already has information related to the supplemental content items.

[0045] Now refer to Figure 8 , a flow chart illustrating a method 800 for assigning a hyperlink to text in an output of a generative model is illustrated. The method 800 begins at 802 and receives user input at the generative model at 804. Optionally, the generative model generates a query based on the user input and provides the query to a search engine, and the search engine identifies search results based on the query.

[0046] At 806, the generative model generates an output based on the user input (and optionally based on at least some search results identified by the search engine). At 808, text corresponding to the supplemental content item in the output generated by the generative model is identified. As described above, the generative model can analyze the output and identify the text therein that corresponds to the supplemental content item. In another example, a second generative model receives the output and identifies the text therein that corresponds to the supplemental content item. In yet another example, the output is provided to the supplemental content provisioning system 112, and the supplemental content provisioning system 112 identifies the text therein that corresponds to the supplemental content item.

[0047] At 810, a hyperlink is assigned to the text in the output, and the hyperlinked text is presented on the client computing device. When hovering over the hyperlinked text, the supplemental content item can be presented (e.g., as a pop-up window). When the hyperlinked text is selected (or the supplemental content item is selected), the web page corresponding to the supplemental content item (landing page) is loaded by the web browser and presented on the display of the client computing device. Method 800 is completed at 812.

[0048] In addition to the functionality discussed above, the generative model 114 can also generate content that can be included in a supplemental content item (e.g., the generative model 114 can at least partially construct the supplemental content item). Figure 9 , illustrates a GUI 900 for supplemental content, including at least a portion generated by generative model 114. According to an example, based on a query posed by a user of client computing device 104 to search engine 116, the supplemental content provisioning system 112 identifies supplemental content items to be presented to the user of client computing device 104 based on a webpage being viewed by the user of client computing device 104 or other appropriate information. For example, the supplemental content items may relate to a company that wishes to indicate at least one product or service that may be of interest to the user of client computing device 104. Generative model 114 is provided with the supplemental content items identified by supplemental content provisioning system 112 and / or the identity of the company corresponding to the supplemental content items. When generative model 114 is provided with the identity of the company, a webpage of the company's website may be provided to generative model 114. In an example, generative model 114 may construct a query and cause such query to be provided to search engine 116. Search engine 116 may retrieve search results based on the query, wherein the search results include webpages belonging to the company's website. Information extracted from the search results may be provided to generative model 114. Based on the supplemental content items identified by the supplemental content provisioning system 112 and the web pages of the company's website, the generative model 114 may generate insights about the company, products offered for acquisition by the company, services offered for acquisition by the company, and the like.

[0049] The insights referenced above can be or include interesting offers made by the company, the relevance of (multiple) supplemental content items related to the company to the query, highlighting information related to the user's search content, etc. In a non-limiting example, the supplemental content items identified by the supplemental content provisioning system 112 can be related to a product. A web page of a company's website that offers a product for acquisition can indicate that free shipping is available for the product purchased online (and the supplemental content item fails to include information about shipping costs). Although the supplemental content item does not reference free shipping, the generative model 114 can generate an insight indicating that free shipping is available from the company. Accordingly, the GUI 900 includes an insights field 902 that includes insights about the company and / or product and / or service related to the supplemental content item identified by the supplemental content provisioning system 112.

[0050] In yet another example, the generative model 114 is well suited for use in conjunction with a mobile computing device. For example, a voice agent can be triggered on the mobile computing device when the user of the mobile computing device is inactive for a certain threshold time. Thus, through voice interaction, the generative model 114 can help the user parse search results by using voice as an interaction mode. In an example, the voice-based agent (generative model 114) facilitates supplementary content items, search results identified by the search engine 116, and the like. The voice agent can be triggered based on one or more metrics, such as the amount of time the user views the web page or other indications that the user may need additional assistance when viewing content on the web page. The generative model 114 can generate voice-based conversational output based on, for example, search results presented on a SERP, web page content presented on a mobile device, and the like. The generative model 114 can summarize the content on the web page in a meaningful way, rather than simply reading the results.

[0051] Generative model 114 can also be used to provide users with additional information related to companies offering products or services for purchase. Conventional search engines include verticals related to shopping. When a user issues a query to such a vertical, the search engine presents supplemental content items from various retailers offering such products or services for purchase. While some of the companies corresponding to these supplemental content items are generally well-known, others may be smaller retailers with lesser reputations, and users may need to conduct independent research before deciding to purchase from such retailers. Generative model 114 can be used to provide users with additional information about such retailers, allowing them to make informed decisions about product purchases. For example, generative model 114 can obtain information related to company reviews from social media sites, from the company's website, from the knowledge graph used by search engine 116, and so on. Furthermore, generative model 114 can obtain information about recent news events, social media posts, and the like to generate semantically meaningful information about the company. In this example, search engine 116 retrieves this information based on the query generated by generative model 114.

[0052] refer to Figure 10 , illustrates a method 1000 performed by a generative model. The method begins at 1002 and receives as input a supplemental content item that has been selected for presentation to a user of a client computing device and / or receives as input information about the supplemental content item at 1004. This information may include the identity of a product, the identity of a service, the identity of a product provider, the identity of a webpage corresponding to the product or service, and the like. At 1006, the generative model generates output based on the supplemental content item and / or information received at 1002. At 1006, the supplemental content item is updated to include the output generated by the generative model. Method 1000 completes at 1008.

[0053] Now refer to Figure 11, a GUI 1100 of a SERP is presented. The SERP includes a query field 1102 through which a user of the client computing device 104 can pose a query. Based on the query, the search engine 116 searches across various data sources to generate search results. For example, the search results may include a knowledge card 1104 depicting information about an entity referenced in the query, a field 1106 including a link to a web page identified by the search engine 116 as relevant to the query, a widget 1108 that the search engine 116 has identified as relevant to the query, and supplemental content items 1110 identified by the supplemental content provisioning system 112 (e.g., based on the query). In contrast to conventional approaches, the supplemental content items 1110 include features that allow for interaction with the generative model 114 (e.g., if the generative model 114 is assigned to a company corresponding to the supplemental content item 1100). For example, supplemental content item 1110 includes suggestion chips 1112 and 1114, wherein selection of a suggestion chip causes the conversational input represented in the suggestion chip to be provided to generative model 114 (thereby initiating a conversation with generative model 114 regarding a product, service, company, etc. related to supplemental content item 1110). Further, supplemental content item 1110 may include or be graphically associated with an input field 1116 through which a user of client computing device 104 may provide conversational input to generative model 114. In other words, supplemental content item 1110 may be visually packaged with features associated with generative model 114. In this example, generative model 114 generates suggestion chips 1112 and 1114 based on information related to supplemental content item 1110 and / or information related to a provider associated with such item 1110. The aforementioned information may be obtained directly from supplemental content item 1110 and / or a webpage associated with the provider.

[0054] In this case, the supplemental content item 1110 and / or the graphical feature can indicate to the user of the client computing device 104 that the generative model 114 is assigned to the company. The generative model 114 can be customized by the company (so that it can perform a soft sell, a hard sell, access the company's inventory, etc.). Although the supplemental content item 1110 is illustrated as being on a SERP, it is understood that the supplemental content item packaged with the feature associated with the generative model 114 can appear on any appropriate web page or web application that presents the supplemental content item.

[0055] In an example, the application programming interface of the generative model 114 is exposed to a company, thereby allowing such company to insert content for the generative model 114 to use in a conversation (such as price feeds, availability, deep web data, data behind databases, product support databases, specifications, features, sales platforms for products, etc.). Further, meta HTML tags can be exposed on the page to provide the generative model 114 with additional information about how to talk to the user, what to say to the user, conversation rules, politeness, etc. In another example, the generative model 114 is customized for the company, thereby allowing the company to control the content output by the generative model 114. Now referring to Figure 12 , a GUI 1200 is presented that is associated with the generative model 114. The GUI 1200 indicates that supplemental content items 1202 may be presented within a session conducted between a user of the client computing device 104 and the generative model 114. Figure 12 In the example shown, a user of client computing device 104 is conversing with generative model 114 about a particular type of car found in a movie that the user of client computing device 104 is interested in. Generative model 114 is provided with information identifying the make and year of the car. As illustrated, generative model 114 receives input "What car is the car from 'movie'?" Generative model 114 can construct a query (e.g., "cars from the movie 'movie'") and provide the query to search engine 116. The search engine obtains search results based on the query and provides at least some of the search results to generative model 114.

[0056] The generative model 114 may also determine, based on the content of the conversation (and optionally a portion of the user history 122), that the user of the client computing device 104 has expressed interest in such a car, and may indicate to the supplemental content provisioning system 112 that the user is interested in the car. In an example, the generative model 114 provides the generated query to the supplemental content provisioning system 112, and the supplemental content provisioning system 112 identifies a supplemental content item based on the query. The supplemental content provisioning system 112 causes the supplemental content item 1202 to be presented within the flow of the conversation. For example, the supplemental content provisioning system 112 provides the supplemental content item 1202 to the generative model 114, and the generative model 114 inserts the supplemental content item 1202 into the conversation flow between the user of the client computing device 104 and the generative model 114. In an example, a fee structure may be set up such that a company corresponding to supplemental content item 1202 may be evaluated for the position of supplemental content item 1202 in a conversation, may be evaluated per conversation turn related to cars, may be evaluated when supplemental content item 1202 is selected, and so on.

[0057] While some examples of functionality that the supplemental content supply system 112, generative model 114, and search engine 116 can perform have been described above, various other functionalities are also contemplated. In one example, the generative model 114 generates a narrative associated with a multi-click supplemental content item. Specifically, today, sites generate a "Top Deals for X" list consisting of a set of supplemental content items. These form a broad category of double-click experiences. In addition to the supplemental content items themselves, which can be supplemented by the output of the generative model 114, local content generated by the generative model 114 can also be placed, which is configured to influence user behavior regarding the supplemental content items. For example, the generative model 114 can generate the equivalent of an engaging story about how a specific product is used at a key moment. To this end, the generative model 114 can be given the context of existing supplemental content items that form the core of the local experience, and a narrative can be generated based on this set of supplemental content items. In this example, the prompt words used by the generative model 114 include information from the supplemental content items and instructions for generating a narrative associated with such items.

[0058] In yet another example, the GLM 114 generates a semantic representation of a user's interests based on a history of the user's interactions with web pages, supplemental content items, search results, and the like. For example, a user of the client computing device 104 may provide consent for their history to be analyzed, such that the user history 122 includes information about the user. The generative model 114 receives this user history and generates a semantic representation of the user's interests based on the user history. The generative model 114 provides an advantage over conventional methods because the generative model 114 is able to propose the reasoning behind the user's user history. Thus, given a user history, the generative model 114 can summarize the history into a few phrases / sentences that summarize the user's interests. The supplemental content provisioning system 112 can be provided with this summary and can employ this information when selecting supplemental content items to provide to the user of the client computing device 104.

[0059] Furthermore, the generative model 114 can be configured to assign tags to supplementary content items in the supplementary content 120. Conventionally, a significant amount of time and resources are required to obtain data regarding the relevance of supplementary content items to a query, the relevance of supplementary content items to keywords, the relevance of supplementary content items to a user or set of users, and the like. The generative model 114 can be provided with supplementary content items in the supplementary content 120 and assign tags to the supplementary content items that indicate relevance to a particular query, keyword, set of users, and the like. Furthermore, the generative model 114 can translate supplementary content items from one language to another, such that the generative model 114 can use a single cue word to assign tags to supplementary content items in different languages. In this case, the cue word provided to the generative model 114 can be designed to cause the generative model 114 to perform these tasks.

[0060] Furthermore, the generative model 114 can be configured to generate the supplemental content item in its entirety, or it can be configured to generate portions of the supplemental content item (title, description, etc.). Conventionally, titles of supplemental content items are automatically generated for various products and / or services; however, conventionally, the generated content is not personalized (either to the end user to whom the supplemental content item is provided or to the company corresponding to the supplemental content item). The generative model 114 can generate content for the supplemental content item, such as a title, description, site link, etc., wherein such content can be generated based on user information (e.g., a set of users corresponding to a particular demographic), information about the company, information extracted from a web page (e.g., describing the product), etc. For example, the supplemental content item can be generated offline, and content from the user history 122 can be used to identify suitable supplemental content items having portions generated by the generative model 114. To generate such supplemental content item portions, the generative model 114 can access the user history 122 and, for example, a landing page for a product, information about the company, etc., and the generative model 114 can generate such portions based on such information. Additionally, when the user consents to analyzing the input, the input provided by the user of the client computing device 104 to the generative model 114 can be utilized in conjunction with building a user profile. For example, based on rich contextual information from the conversation between the user and the generative model 114, the generative model 114 can build an embedding for the user to allow for more targeted and personalized retrieval of supplementary content for the user. In an example, the context from the conversation between the user and the generative model 114 can be incorporated into the user representation as a near-real-time signal for identifying supplementary content items for presentation to the user.

[0061] In conjunction with providing supplemental content items to users, agents for companies typically bid for space on a web page where the supplemental content items are to be displayed. Currently, ad networks and advertisers employ estimates for bids before launching an ad campaign, combining estimates of demand for the supplemental content items, revenue corresponding to the supplemental content items, and the like. In an example, the generative model 114 can be used to provide predictive modeling for estimating bids corresponding to supplemental content items. That is, richer feedback from the conversational content between the generative model 114 and the user can be utilized to enhance the features utilized for estimating bids. Thus, the contextual embedding vectors generated by the generative model 114 can be features used to represent user engagement, interest, and click feedback, which in turn can be used to estimate bids.

[0062] Generative model 114 can also be used in conjunction with generating images for supplemental content items. Images are often quite important for certain types of electronic advertising. However, a company may not associate images with all supplemental content items that the company intends to use. Thus, generative model 114 can be used to generate queries used by search engine 116 to retrieve images that can be assigned to textual supplemental content items, and / or can be used to generate images to be assigned to textual supplemental content items.

[0063] Various other features are also envisioned. For example, when a user of the client computing device 104 mentions a particular word or entity (or something related to a word or entity) in a conversation with the generative model 114, a supplemental content item can be triggered. For example, the supplemental content item can be presented in the conversation as an arbitrary widget (image) or a more subtle conversational reply, such as "Have you considered car Y?" The generative model 114 can be configured to perform intent detection to trigger the identification of the supplemental content item and to generate a response with specific content. The conversation output generated by the generative model 114 associated with the supplemental content item can be marked to indicate that the supplemental content item is related to the company (e.g., sponsored). In this case, the supplemental content item can appear in the conversation flow or can be displayed on other services on the web page.

[0064] Similarity-based ad bidding is also contemplated. Advertisers can have a comprehensive textual description of the type of user and / or search and conversation activity they are trying to target. This textual description (paragraph, sentence, etc.) can be embedded into a vector by the generative model 114. Advertisers can bid for locations based on the vector similarity of their description and user activity (e.g., conversation history, search history, etc.). The embedding allows for disambiguation of search intent (whether the term "tiger" refers to an animal or a sports team). The generative model 114 can appropriately weigh content from user-generated conversation turns with content from conversation turns generated by the generative model 114, and the embedding can be multimodal and jointly trained to embed a combination of text, speech, visual content, and / or non-visual metadata that exists. This contextual embedding can be sent directly to the search engine 116 to further improve search relevance.

[0065] Generative model 114 can fairly accurately represent the semantics of large passages of text. When users converse in a conversation, individual keywords may not be particularly useful for receiving bids from advertisers looking to attract users interested in a specific topic. In contrast, recent conversation history embeddings capture many properties of the conversation, such as user interests and preferences. Preprocessing of the text and training / validation of the embeddings can be performed to remove certain biases from the embeddings and ensure that unacceptable target criteria are removed. Generative model 114 can perform these tasks to generate training data to adjust the embeddings to be unbiased for such limitations.

[0066] Advertisers can submit bids for any embedded block of text. Nearest neighbor similarity lookups can be performed on user content in real or near real time, and advertisers can be charged proportionally to the similarity. This model can be integrated into existing keyword auction mechanisms. Depending on the similarity and number of bids, nonlinear pricing functions can be applied.

[0067] Further, the generative model 114 can be used in conjunction with optimizing advertising to help advertisers generate content so that the generative model 114 can sell products instead of competitors' products. In an example, a second generative model can be trained to imitate / simulate human users. User personalities can be extracted from real user conversations (on other topics) and distilled into customer profiles. Categories of users can be learned from conversation logs. The results of such simulated conversations can be monitored and used to modify supplemental content items - for example, whether the simulated user engaged with the advertisement, the simulated user's response sentiment, etc. can be tracked and used in conjunction with modifying the advertisement. According to the example, the advertisement can be automatically modified until the desired result is achieved.

[0068] Now refer to Figure 13, illustrates a high-level diagram of an exemplary computing device 1300 that can be used in accordance with the systems and methods disclosed herein. For example, computing device 1300 can be used in a system configured to provide content displayed in a web browser to a GLM as at least part of a prompt word. By way of another example, computing device 1300 can be used in a system configured to present supplemental content items to a user. Computing device 1300 includes at least one processor 1302 that executes instructions, the instructions being stored in memory 1304. For example, the instructions can be instructions for implementing the functionality described as being implemented by one or more of the components discussed above or instructions for implementing one or more of the methods described above. Processor 1302 can access memory 1304 via system bus 1306. In addition to storing executable instructions, memory 1304 can also store prompt words, images, supplemental content items, user history, conversations, and the like.

[0069] Computing device 1300 additionally includes a data repository 1308 accessible by processor 1302 via system bus 1306. Data repository 1308 may include executable instructions, instant answers, web indexes, and the like. Computing device 1300 also includes an input interface 1310 that allows external devices to communicate with computing device 1300. For example, input interface 1310 may be used to receive instructions from an external computer device, from a user, and the like. Computing device 1300 also includes an output interface 1312 that enables computing device 1300 to interface with one or more external devices. For example, computing device 1300 may display text, images, and the like via output interface 1312.

[0070] It is envisioned that external devices communicating with the computing device 1300 via the input interface 1310 and the output interface 1312 can be included in an environment providing substantially any type of user interface with which the user can interact. Examples of user interface types include graphical user interfaces, natural user interfaces, and the like. For example, a graphical user interface can accept input from a user using (multiple) input devices (such as a keyboard, mouse, remote control, etc.) and provide output on an output device (such as a display). Further, a natural user interface can enable a user to interact with the computing device 1300 in a manner that is not constrained by the input devices (such as a keyboard, mouse, remote control, etc.). In contrast, a natural user interface can rely on voice recognition, touch and stylus recognition, gesture recognition on and near the screen, air gestures, head and eye tracking, sound and voice, vision, touch, gestures, machine intelligence, and the like. Additionally, although illustrated as a single system, it is to be understood that the computing device 1300 can be a distributed system. Therefore, for example, several devices can communicate via a network connection and can jointly perform the tasks described as being performed by the computing device 1300.

[0071] The various functions described herein can be implemented in hardware, software, or any combination thereof. If implemented in software, the function can be stored as one or more instructions or codes on a computer-readable medium or sent via a computer-readable medium. Computer-readable media include computer-readable storage media. Computer-readable storage media can be any available storage medium that can be accessed by a computer. By way of example and not limitation, such computer-readable storage media can include RAM, ROM, EEPROM, CD-ROM, or other optical disk storage devices, magnetic disk storage devices, or other magnetic storage devices, or any other medium that can be used to carry or store desired program codes in the form of instructions or data structures and can be accessed by a computer. As used herein, disks and optical disks include compact disks (CDs), laser disks, optical disks, digital versatile disks (DVDs), floppy disks, and Blu-ray disks (BDs), wherein disks typically reproduce data magnetically, while optical disks typically reproduce data optically using lasers. Further, the propagated signal is not included within the scope of computer-readable storage media. Computer-readable media also include communication media, which include any media that facilitates transmitting a computer program from one place to another. For example, a connection can be a communication medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies (such as infrared, radio, and microwave), then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies (such as infrared, radio, and microwave) are included in the definition of communications media. Combinations of the above should also be included within the scope of computer-readable media.

[0072] Alternatively or additionally, the functionality described herein may be performed, at least in part, by one or more hardware logic components. For example, but not limited to, illustrative types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0073] This document discloses various methods related to generative models and supplementary content items. Although the method is shown and described as a series of actions performed in sequence, it is to be understood and appreciated that the method is not limited by the order of the sequence. For example, some actions may occur in an order different from that described herein. In addition, an action may occur concurrently with another action. Further, in some instances, not all actions may be required to implement the method described herein.

[0074] Furthermore, the actions described herein may be computer-executable instructions that can be implemented by one or more processors and / or stored on one or more computer-readable media. Computer-executable instructions may include routines, subroutines, programs, execution threads, etc. Furthermore, the results of the method actions may be stored on a computer-readable medium, displayed on a display device, etc.

[0075] Additionally, various techniques are described herein related to presenting supplemental content according to at least the following examples:

[0076] (A1) In one aspect, a method is disclosed herein, wherein the method includes providing a prompt word to a generative model. The generative model is configured to generate an output based on the prompt word. The generative model is further configured to identify that text in the output is to be associated with a supplementary content item. The generative model is additionally configured to assign a hyperlink to the text in the output. The method also includes causing the output to be displayed on a display of a client computing device, wherein the text in the displayed output has the hyperlink assigned to it, and further wherein upon hovering over the hyperlink, the supplementary content item is displayed on the display concurrently with the output.

[0077] (A2) In some embodiments of the method of (A1), when the hyperlink is selected, a web page related to the supplemental content is loaded by a web browser of the client computing device and presented on the display.

[0078] (A3) In some embodiments of the method of at least one of claims (A1) to (A2), the method further includes receiving user input, wherein the user input is included in the prompt word, and further, wherein the user input is received before the prompt word is provided to the generative model.

[0079] (A4) In some embodiments of the method of (A3), the generative model generates a query based on user input. Additionally, the method further includes providing the query to a search engine, wherein the search engine identifies search results based on the query. The method further includes including at least a portion of the search results in the prompt word.

[0080] (A5) In some embodiments of the method of at least one of (A1) to (A4), the method further includes several actions occurring after causing the output to be displayed on a display of the client computing device. The actions include providing user input to the generative model, wherein the generative model is configured to generate a query based on the user input. The actions also include providing the query to a supplemental content provision system, wherein the supplemental content provision system identifies a second supplemental content item based on the query. The actions also include providing the second supplemental content item to the generative model. The actions additionally include causing the second supplemental content item to be displayed on the display as part of the conversation between the user and the chatbot.

[0081] (A6) In some embodiments of the method of (A5), the method further includes providing a query to a search engine, wherein the search engine identifies the image based on the query. The method additionally includes inserting the image into the second supplemental content item.

[0082] (A7) In some embodiments of the method of at least one of (A1) to (A6), the prompt word includes instructions for avoiding consideration of information related to the supplemental content item when identifying that text in the output is to be associated with the supplemental content item.

[0083] (A8) In some embodiments of the method of at least one of (A1) to (A6), the prompt word includes user history information, and further, wherein the text in the output is identified as being associated with the supplemental content item based on the user history information.

[0084] (B1) In another aspect, a method performed by a computing system is disclosed, wherein the method includes receiving a prompt word at a generative model. The method also includes generating, by the generative model, an output based on the prompt word, wherein the output includes text. The method also includes identifying, by the generative model, that at least one word in the text is to be associated with supplemental content. The method additionally includes obtaining a hyperlink pointing to the supplemental content. The method also includes assigning, by the generative language model that generates the output, a hyperlink to at least one word in the outputted text. The method also includes causing the output to be displayed on a display, wherein at least one word in the text is displayed as having a hyperlink assigned to it, and further, wherein, in response to hovering over the hyperlink, the supplemental content item is displayed along with the output of the generative model in a graphical user interface (GUI) associated with the generative model.

[0085] (B2) In some embodiments of the method of (B1), the method further includes receiving an indication that the supplemental content item has been selected. The method further includes updating a prompt word of the generative model to include content of the webpage pointed to by the supplemental content item, wherein the prompt word is updated in response to receiving the indication that the supplemental content item has been selected.

[0086] (B3) In some embodiments of the method of at least one of (B1) to (B2), the method further includes receiving user input, wherein the user input is included in the prompt word, and further, wherein the user input is received before the generative model generates the output.

[0087] (B4) In some embodiments of the method of (B3), the method further includes generating, by the generative model, a query based on the user input. The method further includes providing the query to a search engine, wherein the search engine identifies search results based on the query. The method additionally includes including at least a portion of the search results in the prompt word.

[0088] (C1) In yet another aspect, a method disclosed herein includes providing a prompt word to a generative model, wherein the prompt word includes instructions for the generative model. The instructions instruct the generative model to: 1) review output to be generated by the generative model based on the prompt word for text to be associated with a supplemental content item; and 2) assign a hyperlink to the text in the output. The method also includes receiving output from the generative model, wherein the output includes text and a hyperlink assigned to the text. The method additionally includes causing the output to be displayed in a graphical user interface (GUI), wherein the text has a hyperlink assigned to it, and further, wherein upon hovering over the hyperlink, the supplemental content item is displayed concurrently with the output.

[0089] (C2) In some embodiments of the method of (C1), when the hyperlink is selected, a web page related to the supplemental content item is loaded by a web browser and presented on a display of the client computing device.

[0090] (C3) In some embodiments of the method of at least one of (C1) to (C2), the method includes performing several actions before the prompt word is provided to the generative model. The actions include receiving user input and providing the user input to the generative model. The actions also include receiving a query generated by the generative model based on the user input. The actions additionally include providing the query to a search engine, wherein the search engine identifies search results based on the query. The actions also include including at least a portion of the search results in the prompt word.

[0091] (C4) In some embodiments of the method of at least one of (C1) to (C3), the method further includes including the user input and the query generated by the generative model in the prompt word.

[0092] (C5) In some embodiments of the method of at least one of (C1) to (C4), the method further includes providing user input to the generative model after receiving the output from the generative model. The method further includes receiving a query generated by the generative model based on the user input. The method additionally includes providing the query to the supplemental content provision system, wherein the supplemental content provision system identifies a second supplemental content item based on the query. The method further includes receiving the second supplemental content item. The method further includes causing the second supplemental content item to be displayed on the GUI as part of the conversation between the user and the chatbot.

[0093] (C6) In some embodiments of the method of at least one of (C1) to (C5), the supplemental content item is an electronic advertisement.

[0094] (C7) In some embodiments of the method of at least one of (C1) to (C6), the supplemental content item is linked to the webpage via a hyperlink. The method further includes receiving user input after causing the output to be displayed. The method further includes obtaining content from the webpage. The method additionally includes providing a second prompt to the generative model, wherein the second prompt includes the user input and the content from the webpage, and further wherein the generative model generates a second output based on the second prompt.

[0095] (C8) In some embodiments of the method of at least one of (C1) to (C7), the prompt further includes information related to the supplemental content item. The prompt additionally includes instructions for avoiding consideration of the information related to the supplemental content item when generating the output, wherein the generative model assigns a hyperlink to text in the output based on the information related to the supplemental content item.

[0096] (D1) In another aspect, a system described herein includes a processor and a memory, wherein the memory stores instructions that, when executed by the processor, cause the processor to perform any of the methods disclosed herein (e.g., any of (A1) to (A8), (B1) to (B4), or (C1) to (C8)).

[0097] (E1) In another aspect, a computer-readable storage medium includes instructions that, when executed by a processor, cause the processor to perform any of the methods disclosed herein (e.g., any of (A1) to (A8), (B1) to (B4), or (C1) to (C8).

[0098] What has been described above includes examples of one or more embodiments. Of course, for purposes of describing the aforementioned aspects, it is not possible to describe every conceivable modification and alteration of the above apparatus or methods, but one of ordinary skill in the art will recognize that many further modifications and permutations of various aspects are possible. Accordingly, the described aspects are intended to encompass all such alterations, modifications, and variations that fall within the spirit and scope of the appended claims. Furthermore, to the extent that the term "include" is used in a specific embodiment or in a claim, such term is intended to be inclusive in a manner similar to how the term "comprising" is interpreted when "comprising" is employed as a transition word in a claim.

Claims

1. A computing system comprising: processor; as well as a memory storing instructions that, when executed by the processor, cause the processor to perform actions, the actions comprising: Providing a prompt word to a generative model, wherein the generative model is configured to: generating an output based on the prompt word; identifying that text in the output is to be associated with a supplemental content item; and assigning a hyperlink to the text in the output; and The output is caused to be displayed on a display of a client computing device, wherein the text in the displayed output has the hyperlink assigned thereto, and further wherein upon hovering over the hyperlink, the supplemental content item is displayed on the display concurrently with the output. 2 . The computing system of claim 1 , wherein when the hyperlink is selected, a web page related to the supplemental content is loaded by a web browser of the client computing device and presented on the display.

3. The computing system according to at least one of claims 1 to 2, wherein the actions further comprise: Prior to providing the prompt word to the generative model, user input is received, wherein the user input is included in the prompt word.

4. The computing system of claim 3, wherein the generative model generates a query based on the user input, the actions further comprising: providing the query to a search engine, wherein the search engine identifies search results based on the query; as well as At least a portion of the search results is included in the prompt word.

5. The computing system according to at least one of claims 1 to 4, wherein the actions further comprise: After causing the output to be displayed on the display of the client computing device: providing user input to the generative model, wherein the generative model is configured to generate a query based on the user input; providing the query to a supplemental content provision system, wherein the supplemental content provision system identifies a second supplemental content item based on the query; providing the chatbot with the second supplemental content item; as well as The second supplemental content item is caused to be displayed on the display as part of a conversation between a user and the chatbot.

6. The computing system of claim 5, wherein the actions further comprise: providing the query to a search engine, wherein the search engine identifies images based on the query; as well as The image is inserted into the second supplemental content item.

7. The computing system of at least one of claims 1 to 6, wherein the prompt word includes instructions for avoiding consideration of information related to the supplemental content item when identifying that the text in the output is to be associated with the supplemental content item.

8. The computing system of at least one of claims 1 to 7, wherein the prompt word includes user history information, and further wherein text in the output is identified as being associated with the supplemental content item based on the user history information.

9. A method performed by a computing system, the method comprising: At the generative model, receiving a prompt word; generating, by the generative model, an output based on the prompt word, wherein the output comprises text; identifying, by the generative model, at least one word in the text to be associated with supplemental content; obtaining a hyperlink to said Supplemental Content; assigning, by the generative language model that generates the output, the hyperlink to the at least one word in the text of the output; causing the output to be displayed on a display, wherein the at least one word in the text is displayed as having the hyperlink assigned thereto, and further wherein, in response to hovering over the hyperlink, the supplemental content item is displayed along with the output of the generative model in a graphical user interface (GUI) associated with the generative model.

10. The method according to claim 9, further comprising: receiving an indication that the supplemental content item has been selected; as well as In response to receiving the indication that the supplemental content item has been selected, updating the prompt words of the generative model to include content of the webpage pointed to by the supplemental content item.

11. The method according to at least one of claims 9 to 10, further comprising: Before the generative model generates the output, user input is received, wherein the user input is included in the prompt word.

12. The method according to claim 11, further comprising: generating, by the generative model, a query based on the user input; providing the query to a search engine, wherein the search engine identifies search results based on the query; as well as At least a portion of the search results is included in the prompt word.

Images