Generative search engine result document

Abstract

The present disclosure describes utilizing a generative document system to dynamically build and provide generative search result documents. A generative document system utilizes an aggregation framework that utilizes one or more large generative models (LGMs). For example, the aggregation framework includes three stages in which a local process is applied to a generative output from the LGM, each stage being established on a generative input from a previous stage. A generative document system uses an aggregation framework to create a generative search result document based on a search query and its corresponding search result link. These generative search result documents provide interactive, intuitive, comprehensive, and flexible answer selection to process corresponding search queries.

Description

Background Technology

[0001] In recent years, significant progress has been made in both hardware and software, particularly in internet search. Current systems are designed to retrieve and present lists of links and resources in response to user search queries. These systems also provide direct answers to frequently searched topics. However, with the continuous growth in the number of accessible resources, existing systems still have limitations in providing useful and relevant answers without requiring significant user effort. For example, users often have to manually check the results list and view multiple entries before finding a satisfactory answer. Sometimes, users find direct answers to search results, while the complete list of results on a webpage becomes unhelpful. These and other problems exist in current search result systems. Attached Figure Description

[0002] The following detailed description provides specific and detailed embodiments with accompanying drawings. Furthermore, each figure listed below corresponds to one or more embodiments discussed in this disclosure.

[0003] Figure 1 The illustration provides an overview of an example generative document system that uses a large generative model to create generative search results documents for a search query.

[0004] Figure 2 The illustration shows an example computing environment in which a generative document system is implemented.

[0005] Figure 3A The diagram illustrates an example sequence of generating unformatted generative documents using a large generative model, and... Figure 3B The illustration shows an example of generating an unformatted generative document.

[0006] Figure 4A The illustration shows an example sequence diagram of using a large generative model to determine answer card matching, and Figure 4B The illustration shows an example of generating answer card matches for a specific topic section.

[0007] Figure 5A The illustration shows an example sequence of diagrams demonstrating the use of a large generative model to determine layout guidelines, and Figure 5B The illustration shows an example of generating a topic section layout.

[0008] Figure 6 The diagram illustrates an example sequence of formats that generate documents for queries that have been previously searched.

[0009] Figure 7 The illustration shows a sample graphical user interface for a generative search results document.

[0010] Figure 8The illustration shows an example sequence of actions for a computer-implemented method of generating generative search results documents using one or more large generative models.

[0011] Figure 9 The illustration shows example components included within a computer system for implementing a generative document system. Detailed Implementation

[0012] This disclosure describes the use of a generative document system to dynamically construct and deliver generative search result documents. This system utilizes an aggregation framework that leverages one or more large generative models (LGMs). For example, the aggregation framework comprises three stages, where local processes are applied to the generative output from the LGM, with each stage building upon the generative input from the previous stage. The generative document system uses the aggregation framework to create generative search result documents based on search queries and their corresponding search result links. These generative search result documents provide an interactive, intuitive, comprehensive, and flexible selection of answers for the corresponding search query.

[0013] The embodiments of this disclosure provide benefits and address problems in the art by utilizing generative document systems, computer-readable media, and computer-implemented methods to select and construct generative documents from search result links using one or more LGMs. As described below, generative document systems provide an aggregation framework that utilizes one or more LGMs to leverage different output types to construct dynamic and interactive generative documents to be provided along with search result links, wherein the generative documents intelligently select search result information into a streamlined and understandable format.

[0014] As mentioned above, the generative document system creates generative search results documents (generative documents). For illustration, in various embodiments, in response to receiving a search request from a user and obtaining a set of search results links, the generative document system generates a generative document request prompt and search results links and provides them to a first LGM, which generates an unformatted generative document with a topic section containing a textual description response corresponding to the links in the search results links. In response to providing an answer card association prompt, a topic section, and a set of candidate answer cards for the topic section to a second LGM, the generative document system receives a first set or list of answer cards for each topic section.

[0015] In addition, the generative documentation system provides generative layout hints and available layout options to the third-party LGM. In response, the generative documentation system receives generative layout guidelines for each topic section, where the layout visually positions the textual response combined with answer cards for each topic section. Furthermore, the generative documentation system generates a formatted generative document including the layout for the topic sections and provides it to the client device requesting the search query.

[0016] As described in this disclosure, generative document systems offer several significant technical benefits in terms of improved accuracy, efficiency, and flexibility compared to existing search result systems. Furthermore, generative document systems provide numerous practical applications for addressing problems related to providing search results in response to search queries.

[0017] To better understand the technical benefits of generative document systems, consider existing search result systems. For example, in response to a user's web search query, existing search result systems typically display the response as a Search Engine Results Page (SERP), which includes a collection or list of results most relevant to the search query. In its early stages, search result systems simply provided a sequence of hyperlinks associated with the user's search terms in the SERP. However, currently, existing search result systems also provide SERPs that include direct responses to frequently queried topics. Despite these advancements, current systems face technical challenges.

[0018] As mentioned earlier, generative document systems create generative documents that provide streamlined, dynamic, and understandable answers to search queries. In various implementations, the generative document includes an accurate and efficient selection of search result links, which are presented along with the generative document. In various implementations, generative document systems create improved SERPs that include generative documents and corresponding search result links.

[0019] By using LGM to build generative documents from search result links for a search query, generative document systems improve the accuracy and efficiency of delivering search results. For example, a generative document system uses LGM to generate textual narrative responses from search result links in an unformatted generative document, segmented into topics relevant to the search query. The generative document system then matches the textual narrative responses with curated answer cards that provide targeted and interactive answers. Furthermore, the generative document system generates layout guidelines for formatting the generative document into a logical and readable arrangement. In addition to including textual narrative responses paired with relevant answer cards referencing search result links, the formatted generative document can be presented alongside search result links for efficient navigation.

[0020] As mentioned, generative documentation systems provide an aggregation framework built upon the output of multiple invocations or prompts to one or more LGMs, where different prompts produce different output formats. In some instances, the generative documentation system invokes multiple LGMs with different prompts. In some implementations, the generative documentation system prompts the same LGM with different prompts to generate different output formats (e.g., unformatted generative documents, lists of related answer cards, and layout guidelines). Generative documentation systems effectively leverage the different features and functionalities of LGMs to create outputs that can be composited at each stage to improve accuracy and efficiently construct highly dynamic, streamlined, and useful generative documents in response to search queries.

[0021] In various implementations, generative document systems improve efficiency by reusing generative documents to reproduce lists of search results. For example, a generative document is created based on a set of search result links retrieved in response to a search query. When the same or similar search queries are repeated (by the same user or another user) and the same or similar set of search result links is obtained, the generative document system can efficiently and accurately reuse the same generative document without multiple calls to one or more LGMs. When the difference in the set of search results exceeds a threshold, and by using external real-time signals (such as news, stock codes, weather, etc.), the generative document system can update the generative document to ensure accurate results are provided in response to search queries.

[0022] Generative document systems also offer greater flexibility than existing systems. As mentioned earlier, generative document systems utilize different features of one or more LGMs to generate multiple output formats. As part of this flexibility, generative document systems use LGM output from earlier steps as LGM input in later steps to cleverly create the elements needed to build the final formatted generative document. As another part of this flexibility, generative document systems can provide different hints to the same LGM to generate different LGM output types, or provide them to different LGMs.

[0023] As explained in the preceding discussion, this disclosure uses various terms to describe the features and advantages of one or more of the described embodiments. For example, this disclosure describes a generative document system in the context of a cloud computing system.

[0024] As an example, a “large generative model” (LGM) is a large-scale artificial intelligence system that leverages deep learning and a vast number of parameters (e.g., billions or trillions) trained on one or more extensive datasets to produce coherent, context-sensitive, and fluent topic-specific outputs (e.g., text and / or images). In many instances, a generative model refers to a high-level computational system that uses natural language processing, machine learning, and / or image processing to generate coherent and context-sensitive human-like responses.

[0025] Large-scale generative models have applications in natural language understanding, content generation, text summarization, dialogue systems, language translation, creative writing assistance, image generation, and audio generation. A single large-scale generative model typically performs a wide range of tasks by receiving diverse inputs such as cues (e.g., input instructions, rules, example inputs, example outputs, and / or tasks), data, and / or access to that data. In response, large-scale generative models generate a variety of output formats, from single-word answers to long narratives, images and videos, labeled datasets, documents, tables, and presentations.

[0026] Furthermore, large generative models (LGMs) are primarily based on transformer architectures to understand, generate, and manipulate human language. LGMs can also utilize other architecture types, such as recurrent neural network (RNN) architectures, long short-term memory (LSTM) model architectures, convolutional neural network (CNN) architectures, or other architecture types. Examples of LGMs include generative pre-trained transformer (GPT) models (such as GPT-3.5 and GPT-4), bidirectional encoder representation (BERT) models derived from transformers, text-to-text transformer models (such as T5), conditional transformer language (CTRL) models, and Turing-NLG models. Other types of large generative models include sequence-to-sequence (Seq2Seq) models, basic RNNs, and LSTM networks. In some instances, LGMs include large language models (LLMs) that act as text-based versions of LGMs (such as LGMs that receive text prompts and / or generate text outputs). In various implementations, an LGM is a multimodal generative model that receives multiple input formats (e.g., text, images, videos, data structures) and / or generates multiple output formats.

[0027] As an example, the term "LGM hint" or "hint" refers to a request provided to a large generative model based on colloquial language guidance to create generative LGM output. In some instances, generative documentation systems provide additional information with hints. Hints can include user-level hints that include user requests, or system-level or meta-level hints that provide important contextual and / or general framing information to ensure that the LGM understands the correct context, syntax, and underlying information of the data it is processing. Examples of hints include: generative document request hints, which include instructions to the LGM for generating an unformatted generative search results document with topic sections; answer card association hints, which include instructions to the LGM for generating a list of one or more answer cards associated with the topic sections of an unformatted generative document; and generative layout hints, which include instructions to the LGM for generative layout guidelines that guide the generative documentation system to generate a layout that includes a combination of textual narrative responses and associated answer cards.

[0028] As an example, the term "search link results" refers to links (e.g., hyperlinks) and their corresponding resources obtained in response to a search query (e.g., a user-requested search query). In various implementations, providing search link results to the LGM enables the model to access resources, summaries, and / or metadata associated with the links. In various implementations, the set of search link results includes a ranking order of results based on a score assigned to each result according to its relevance to the search query. Search link results may include any number of results corresponding to the search query.

[0029] As an example, the term "generative search results document" (or generative document) refers to a search-based document that includes a textual response corresponding to a search query and a set of corresponding search result links. In many implementations, a generative document includes a topic section corresponding to a search query and a set of search result links based on the search query. For example, the topic section includes a response that answers the search query. A generative document can include an unformatted generative document that includes a textual response body (e.g., a text response written in colloquial language) arranged in one or more topic sections (e.g., without answer cards or grid layout attributes). A generative document can also include a formatted generative document that includes a visual layout for each topic section, typically within a grid-based or list-based structure, visually arranging the textual response using a combination of answer cards. In some implementations, a generative document is a portion of a search engine results page (SERP) along with a corresponding set or list of search result links.

[0030] As another example, the term "answer card" refers to an element that provides a direct answer to a search query or a subquery derived from a search query. Answer cards provide quick, accurate answers to questions without requiring further user searching or interaction. Answer cards can include text, images, audio, video, and / or animations to convey the quick answer. Furthermore, answer cards can include various versions that incorporate information at different granularities and / or have different layout dimensions (e.g., availability dimensions). Additionally, answer cards include metadata and / or other basic information to allow the LGM to understand the context associated with the card.

[0031] As an example, the term "layout" refers to the visual arrangement of text and / or elements (such as answer cards) within a generative document. While an unformatted generative document may include a topic section, it does not include the layout of text and elements. In some instances, the layout is defined by a grid and / or coordinate system that allocates one or more grid spaces (e.g., blocks) for the text or elements. In various implementations, layout templates provide one or more possible layouts for a generative document or a portion of a generative document (e.g., for a topic section). Furthermore, as an example, the term "layout guide" refers to a set of recommendations, rules, tips, and / or suggestions for arranging textual narrative responses within a topic section having one or more relevant answer cards.

[0032] Furthermore, as an example, "network" refers to one or more data links that enable electronic data transmission between computer systems, modules, and / or other electronic devices. A network can include public networks (such as the Internet) as well as private networks. When information is transmitted or provided to a computer via a network or other communication connection (hardwired, wireless, or a combination of hardwired and wireless), the computer considers that connection as a transmission medium. A transmission medium can include networks and / or data links, which can be used to carry the required program code in the form of computer-executable instructions or data structures, and can be accessed by general-purpose or special-purpose computers. The combinations described above are also included within the scope of computer-readable media.

[0033] Examples and details of implementing generative document systems have been discussed with reference to the accompanying drawings, which will now be described. For example, Figure 1 The illustration provides an overview of a generative document system that uses a large generative model to create generative search results documents for a search query, according to some implementation methods. Although Figure 1 A high-level overview of the invention has been provided, but additional details are provided in the following figures.

[0034] Figure 1The diagram illustrates a sequence of actions 100 performed by or in conjunction with a generative document system. As shown, the sequence of actions 100 briefly illustrates how a generative document system uses an aggregation framework with different LGMs to generate output for subsequent LGM suggestions. These LGM outputs are combined to build or create a formatted generative search results document (generative document).

[0035] As shown, action sequence 100 includes action 102 of generating a generative document using a first LGM based on search result links for a search query. For example, in response to receiving a search query 110 from a user, the generative document system uses it to obtain a set of search result links 114. The generative document system provides a generative document request prompt 112, the search query 110, and the set of search result links 114 to a first LGM 116, which generates an unformatted generative search result document 118 having a first topic section 120, a second topic section 122, and / or other topic sections. The topic sections are a synthesis of information from the set of search result links 114 organized by topic. The following is combined with... Figure 3A and Figure 3B Additional details are provided regarding the generation of unformatted generative documents.

[0036] As shown, action 104 includes matching answer cards with topic sections of an unformatted generative document using an answer card system and a second LGM. For example, the generative document system obtains a set 124 of candidate answer cards from the answer card system for each topic section. The generative document system provides the second LGM 128 with answer card association hints 126, an unformatted generative search results document 118, and one or more sets of candidate answer cards for one or more topic sections, the second LGM generating a set or list of answer cards that matches the topic section (or a portion within a topic section). For example, the second LGM 128 generates a first set or list of answer cards 130 that is associated with a first topic section 120 within the unformatted generative search results document 118. The following is combined with… Figure 4A and Figure 4B Additional details are provided regarding generating a list of answer cards related to the topic sections.

[0037] Action 106 includes creating a formatted generative document template, which includes layout options determined for each topic section based on layout guidelines generated by a third LGM. For example, the generative document system provides the third LGM 134 with a first set or list of answer cards 130, generated layout hints 132, and an unformatted generative search results document 118 to generate a first list or set of generative layout guidelines 136. Based on the first list or set of generative layout guidelines 136, the generative document system generates layout options for the first topic section, such as the first layout 138 shown (and repeats for other topic sections). The generative document system then combines the various layout options for each topic section into a formatted generative document template. The following section combines... Figure 5A and Figure 5B Additional details are provided regarding guidelines for generating layouts for specific theme sections.

[0038] As shown, action 108 includes providing a formatted version of the generative document in response to a search query, which includes layout options for each topic section. For example, the generative document system selects the best available layout options for each section from a formatted generative document template to generate a formatted generative search results document 140. The generative document system then provides the formatted generative search results document 140 to the client device that submitted the search query. In some instances, the generative document system stores and / or accesses the formatted generative search results document 140. The following is combined with... Figure 5A , Figure 5B and Figure 6 Additional details are provided regarding the generation of formatted, generative search results documents. Figure 7 A visual example of a formatted, generative search results document 140 is provided.

[0039] Following the general overview, additional details regarding the components, features, and elements of generative document systems are provided. For illustration, Figure 2 An example computing environment implementing a generative document system according to some embodiments is shown. In particular, Figure 2 An example of a computing environment 200 is illustrated, including various computing devices of a cloud computing system 202 associated with a generative document system 206. Although Figure 2 Example arrangements and configurations of computing environment 200, cloud computing system 202, generative document system 206, and associated components are shown, but other arrangements and configurations are also possible.

[0040] As shown, computing environment 200 includes a cloud computing system 202 associated with generative document system 206, a large generative model 230, an answer card system 240, and a client device 250 with client application 252, connected via network 260. Many of these components can be implemented on one or more computing devices, such as one or more server devices. Some of these components can be implemented on personal devices. The following description, in conjunction with... Figure 9 Further details about the computing device are provided, as well as additional details about the network (such as the network 260 shown).

[0041] Before describing the components of the cloud computing system 202, which includes the generative document system 206, other components of the computing environment 200 will be discussed first to provide better context when discussing the generative document system 206. As shown, the computing environment 200 includes a large generative model 230 (LGM), which creates various types and / or formats of generated outputs (e.g., LGM outputs) and prompt inputs (e.g., LGM inputs). For example, the large generative model 230 generates an unformatted generative search results document 118, a first list or set of answer cards 130, and a first list or set of generative layout guidelines 136 in response to different prompts. The large generative model 230 represents a single LGM or a collection of multiple LGMs.

[0042] As shown, computing environment 200 includes answer card system 240, which receives answer card requests and returns a list of candidate answer cards corresponding to a search query, topic, or part of a search query. In various implementations, the list or set of candidate answer cards is ranked based on quality, engagement, or other metrics. In some instances, the answer card system provides attributes (such as display size, dimensions, media type, rendering metadata, and / or grounding data) for one or more answer cards.

[0043] As shown, computing environment 200 includes client device 250. In various embodiments, client device 250 is associated with a user (e.g., a user client device), such as a user requesting a search query. In various examples, client device 250 includes client application 252, such as a web browser, mobile application, or another form of computer application for accessing and interacting with cloud computing system 202 and / or generative document system 206. For example, client device 250 interacts with generated content (e.g., textual description responses and corresponding answer cards) within a formatted generative search results document via client application 252.

[0044] Returning to cloud computing system 202, as shown, cloud computing system 202 includes search result system 204 and search result link system 228. In various embodiments, search result link system 228 returns a set of search result links in response to a search request. In various embodiments, the search result set includes one or more ranked search result links.

[0045] In various implementations, the search results system 204 provides search results in response to a search query. For example, client device 250 submits a search request, and search results system 204 returns a formatted generative search results document generated by generative document system 206 and / or a set of search results links provided by search results link system 228.

[0046] As shown, the search results system 204 implements a generative document system 206. In some embodiments, the generative document system 206 resides on a separate computing device from the search results system 204, within (or outside) the cloud computing system 202. In various embodiments, the search results system 204 operates without the generative document system 206.

[0047] In various implementations, including those described, the generative document system 206 includes various components and elements implemented in hardware and / or software. For example, the generative document system 206 includes a generative document creation manager 210, a generative answer card association manager 212, a generative layout manager 214, a generative document display manager 216, and a storage manager 218. The storage manager 218 includes a search result link set 220, an answer card set 222, a formatted generative document 224, and topic section layout options 226.

[0048] As mentioned above, the generative document system 206 includes a generative document creation manager 210, which communicates with the search result linking system 228 to obtain one or more search result links from a set 220 of search result links for a search query (and, in some cases, subqueries). Furthermore, the generative document creation manager 210 communicates with a large generative model 230 to generate unformatted generative documents.

[0049] The generative document system 206 also includes a generative answer card association manager 212, which communicates with the answer card system 240 to obtain a set of candidate answer cards corresponding to search queries and / or unformatted generative documents. The generative answer card association manager 212 also communicates with a large generative model 230 to obtain a set of answer cards 222 from the set of candidate answer cards for topical sections within the unformatted generative document.

[0050] Generative document system 206 includes a generative layout manager 214 that communicates with a large generative model 230 to generate layout guidelines for topic sections within an unformatted generative document, wherein the layout guidelines associate answer cards with textual narrative responses within the topic sections. In various embodiments, generative layout manager 214 also determines various layout options (e.g., generating topic section layout option 226) for each topic section based on the generative layout guidelines.

[0051] Generative document system 206 includes a generative document display manager 216 that constructs a formatted generative document 224 based on topic section layout options 226 (e.g., based on a formatted generative document template). For example, generative document system 206 determines the available layout options for each section within the formatted generative document template and then provides those layout options to client device 250 in response to a search query. In some instances, generative document display manager 216, in response to receiving a search query, provides client device 250 with a search engine results page (SERP) that includes the formatted generative document and a collection of search result links.

[0052] Moving on to the next set of attached figures, Figures 3A to 3B , Figures 4A to 4B , Figures 5A to 5B and Figure 6 The diagrams illustrate example sequences of interactions between a generative document system 206 used to construct generative documents and one or more LGMs. In part, these diagrams show the different communications between the generative document system 206, the search result linking system 228, the large generative model 230, the answer card system 240, and the network 260.

[0053] first, Figure 3A The illustration shows an example sequence diagram of generating unformatted generative documents using a large generative model according to some implementation methods. As shown, Figure 3A This includes a sequence of actions 300 performed by or in conjunction with the generative document system 206. In some implementations, the sequence of actions 300 may include fewer or different actions (e.g., some actions may be skipped or omitted). Additionally, in some instances, the actions in the sequence of actions 300 are performed in a different order.

[0054] As shown, action sequence 300 begins with action 302, where the generative document system 206 receives a search query from client device 250. For example, based on user interaction, client device 250 sends a request for search results for a topic or question. For instance, a user enters a search question into a search field of a search engine via a web browser.

[0055] Action 304 illustrates that the generative document system 206 requests search query results from the search result linking system 228. In response, the search result linking system 228 provides the generative document system 206 with a set or list of links to websites and / or resources corresponding to the search query. As shown, action 306 includes the generative document system 206 receiving the set of search query results from the search result linking system 228. In some embodiments, the search query is provided directly to the search result linking system 228, which then provides the search result links to the generative document system 206.

[0056] In some implementations, the search result linking system 228 determines and provides additional information corresponding to the search results. For example, for frequently searched queries, the search result linking system 228 provides a general answer. In some instances, the search result linking system 228 also provides answer fragments from search result links and / or other knowledge base sources.

[0057] Action 308 illustrates the generative document system 206 providing search intent cues to a large generative model 230. In various implementations, the search intent cues provide instructions to the large generative model 230 to analyze a search query, generate a set or list of candidate intents behind the search query, rank the list, and return one or more intents from the list. For illustration, action 310 illustrates the generative document system 206 receiving a set or list of search intents from the large generative model 230.

[0058] In some implementations, the generative document system 206 also provides additional information and search queries to the large generative model 230. For example, the generative document system 206 provides some or all links from the set of search result links. As another example, the generative document system 206 also provides previous user search queries, previous user interactions, user preferences and information, and / or session or state information.

[0059] Action 312 illustrates that the generative document system 206 requests search query results based on / for each search intent. In various embodiments, the generative document system 206 obtains a set of search result links for one or more search intents. The generative document system 206 may send individual requests or make combined requests for different intents. The search result link system 228 may generate and return search result links for each request. For illustration, action 314 illustrates that the generative document system 206 receives search result links based on / for each search intent.

[0060] Action sequence 300 includes action 316 of generative document system 206 selecting a search intent. In various embodiments, generative document system 206 also uses a search intent selection model (which can use various methods including LGM) to select one or more search intents. In some embodiments, action sequence 300 omits this step or uses a default selection process to select the search intent.

[0061] Action 318 illustrates the generative documentation system 206 providing a generative documentation request prompt to a large generative model 230. In various embodiments, the generative documentation request prompt includes instructions for generating an unformatted generative document corresponding to a set of search result links associated with a search query provided by client device 250. For example, the generative documentation request prompt instructs the large generative model 230 to synthesize the set of search result links and corresponding resources individually and collectively, determine a set or list of topics corresponding to the search query (e.g., building block information snippets or questions), generate a textual narrative response for each topic, and reference the textual narrative response back to one or more search result links. In some instances, synthesis includes organizing information from one or more search result links into a textual narrative response, a list, or a chart.

[0062] In some implementations, generative document request prompts include system-level prompts such as establishing context (e.g., "You are a data analyst who discovers connections between websites and writes comprehensive summaries") and / or ensuring responsible artificial intelligence (AI). In some instances, generative document request prompts include output types. Generative document request prompts may also include sample outputs of sample search queries and sample unformatted generative documents with sample topic sections.

[0063] In various implementations, the generative document system 206 also provides one or more search intents and their corresponding search request links. In these instances, the generative document request prompt instructs the large generative model 230 to also consider additional links in the list when generating topic sections and textual narrative responses as parts of an unformatted generative document.

[0064] As shown, action 320 illustrates network 260 receiving an unformatted generative document. As provided above, in many embodiments, an unformatted generative document refers to a text-based generative document that includes one or more sections (e.g., topic sections), each with a corresponding textual narrative response that provides a generative composition of information from a set or list of search results. In some instances, an unformatted generative document may include images or other graphics inline within the topic sections. That is, an unformatted generative document does not include answer cards or visual layout between or within sections.

[0065] As previously mentioned, the unformatted generative document is based on one or more lists of search result links and includes references to some of those links. In various implementations, the generative document system 206 reorders or re-ranks the links based on their appearance in the generative document. This can cause initially high-ranking result links to drop in the ranking order, preventing them from initially appearing in the search result links displayed alongside the generative document. In effect, the generative document system 206 can influence which search result links (and their order) are displayed in the SERP.

[0066] Figure 3B The illustration shows an example of generating an unformatted generative document according to some implementation methods. Figure 3B Examples of various elements and outputs corresponding to the generation of unformatted generative documents are provided. As shown, Figure 3B This includes a sequence of actions 350 performed by the generative document system 206 to generate an unformatted generative document using a large generative model 230.

[0067] Action sequence 350 includes a generative document system 206 providing a search query 352, search results 354, and search intent 356. As shown, search query 352 requests the best time to visit Hawaii. Search intent 356 determines various intents for this search query, such as the best time in terms of weather, cost, business, etc. Search results 354, generated by search result linking system 228, include search result links, general answers, result snippets, etc., as described above.

[0068] As also shown, the generative document system 206 provides a generative document request prompt 112 to the large generative model 230, along with instructions for generating an unformatted generative document from the provided input. In response, the large generative model 230 generates an unformatted generative search results document 360. As shown, the unformatted generative search results document 360 includes topic sections, each comprising a textual narrative response and references to links used to generate the narrative. In particular, the topic sections illustrate different aspects of Hawaiian weather, which may correspond to a search intent to find the best time to visit Hawaii based on the weather.

[0069] Figure 4A The illustration shows example sequence diagrams of using a large generative model to determine answer card matching according to some implementation methods. As shown, Figure 4AThis includes a sequence of actions 400 performed by or in conjunction with the generative document system 206. In some implementations, the sequence of actions 400 may include fewer or different actions (e.g., some actions may be skipped or omitted). Additionally, in some instances, the actions in the sequence of actions 400 are performed in a different order.

[0070] As shown, action sequence 400 begins with action 402 of generative document system 206 maintaining unformatted generative documents. (As previously combined...) Figure 3A As described, the generative document system 206 uses a large generative model 230 to generate unformatted generative documents. In some instances, action sequence 400 omits action 402.

[0071] Action 404 illustrates that the generative document system 206 parses an unformatted generative document into topic sections. In various implementations, because the generative document is unformatted, the generative document system 206 divides the generative document into topic sections. In some instances, the generative document system 206 generates sub-generative documents for each topic section.

[0072] Action 406 illustrates that the generative document system 206 generates an answer card query for each topic section. For example, when parsing or separating a generative document into topic sections, the generative document system 206 generates an answer card query (e.g., an answer recall query) for each section. In some instances, the answer card query requests the answer card system 240 to identify potential answer cards for the content included in the request (e.g., topic sections). Therefore, the generative document system 206 may send multiple answer card queries to the answer card system 240 for different topic sections.

[0073] In some implementations, the search result linking system 228 performs a fan-out search to collect all potential answer cards that match the generative document. For example, consider a search query, a topic section, a result link, a textual response, or a section providing the input "best snorkeling in Maui in July". In this example, along with the query "snorkeling in Maui in July", the search result linking system 228 can break down the input into a first subquery "Maui", a second subquery "July in Maui", and a third subquery "snorkeling in Maui". The search result linking system 228 searches for candidate answer cards based on each fan-out query.

[0074] In various implementations, the search result linking system 228 also provides metadata and answer cards. For example, the search result linking system 228 indicates whether the answer card has multiple versions. For instance, the answer card may include multiple layouts, grid layout dimensions, granularity levels, and / or resolutions. Additionally, the answer card includes grounding data (e.g., the card is self-grounded). In some cases, candidate answer cards are ranked and / or scored based on their quality and / or relevance to the topic section.

[0075] As shown in action 408, generative document system 206 receives candidate answer cards for each topic section. For example, answer card system 240 determines a set or list of possible answer cards for each topic and provides one or more answer cards from the list as candidates. Additionally, in some embodiments, generative document system 206 removes duplicate and / or similar answer cards for topic sections and / or unformatted generative documents.

[0076] Action 410 illustrates the generative documentation system 206 providing answer card association hints to a large generative model 230 for a given topic section. In various implementations, the generative documentation system 206 provides answer card association hints and candidate answer cards for a given topic section. The generative documentation system 206 may provide separate answer card association hints for each topic section (e.g., in parallel with multiple LGM instances), or it may provide combined hints for multiple topic sections. If separate hints are used, the generative documentation system 206 may send a request to the large generative model 230 to combine the final hints from the separate LGM outputs.

[0077] Answer card association prompts can provide the large generative model 230 with instructions for selecting a set of answer cards for a topic section from a corresponding set of candidate answer cards. In some instances, answer card association prompts instruct the large generative model 230 to match one or more answer cards with portions of the textual narrative response of the topic section. In various implementations, answer card association prompts include system-level prompts. In one or more implementations, answer card association prompts provide scoring criteria for evaluating or scoring the matching strength between answer cards and some or all of the textual narrative responses.

[0078] In some implementations, the generative documentation system 206 provides additional input to the large generative model 230. For example, the generative documentation system 206 also provides unformatted generative documents, which are the outputs of previous calls to the large generative model (LGM). In this way, the large generative model 230 is built on previous LGM calls to achieve improved efficiency and accuracy gains compared to existing systems. In some implementations, the scoring criteria are provided as a separate input document.

[0079] In various implementations, the large generative model 230 generates a ranked set or list of answer card matches for each part from the corresponding set of answer cards. The large generative model 230 may determine one or more matches for the overall or partial thematic parts that are responses to textual narratives within a part.

[0080] In some implementations, the large generative model 230 uses a scoring metric to score each match to indicate the strength of relevance for the answer card match. For example, for the section on the best time to visit Hawaii in February, the large generative model 230 assigns a first score to answer cards mentioning Hawaiian weather in February, a second score to answer cards mentioning booking flights to Hawaii, and a third score to answer cards mentioning booking flights to Hawaii in February. In this example, the first answer card would have the lowest score, while the third answer card would have the highest score.

[0081] Action 412 illustrates that the generative document system 206 receives a list of answer cards for a topic section. For example, for a topic section, the large generative model 230 provides a list of answer cards selected from the corresponding candidate answer cards, along with the matching position of each answer card within the topic section. In various implementations, the large generative model 230 does not return any layout, layout suggestions, or visual arrangement when providing the list of answer cards for each topic section.

[0082] Action 414 illustrates that generative document system 206 updates the topic sections in an unformatted generative document with a list of answer cards. For example, generative document system 206 begins creating a generative document template that links answer cards to relevant topic sections. In various implementations, the answer cards include visual information (e.g., available grid layout sizes and dimensions) for later use by generative document system 206.

[0083] Figure 4B The illustration shows an example of generating answer card matching for a specific topic section according to some implementation methods. Figure 4B Examples of various elements and outputs corresponding to the generation of unformatted generative documents are provided. As shown, Figure 4B This includes action sequences 450 performed by the generative document system 206 to generate answer card matches using a large generative model 230.

[0084] Figure 4BThe action sequence 450 includes the generative document system 206 providing various inputs to the large generative model 230. As shown and described above, the generative document system 206 provides candidate answer cards 452 obtained from an answer card system. Inputs also include general search query answers 454, such as answers to frequently searched topics. Additionally, inputs include an answer card ranking 456, which scores, sorts, and ranks the candidate answer cards. In some embodiments, the answer card ranking 456 includes criteria for determining the ranking of matching answer cards. In various embodiments, each topic has its own set of inputs.

[0085] In addition, the input includes supplementary queries 458. In various implementations, supplementary queries are queries related to a given query. For example, for the search query "What is the best time to visit Hawaii?", supplementary queries could include "What is the worst time to visit?", "The most popular islands in Hawaii", "Humpback whales in Hawaii", and "Surfing in Hawaii". In some implementations, supplementary queries are associated with a topic section rather than with the entire generative document.

[0086] As shown, the generative document system 206 also provides the large generative model 230 with answer card association hints 126 and unformatted generative search results documents 118. As described above, the answer card association hints 126 instruct the large generative model 230 to generate answer card matches 460 for the topic section.

[0087] Figure 4B Includes a visual example of answer card matching 460 for each topic section. As shown, the topic sections are tagged (e.g., document_topic_section_). N (Doc_Topic_Section_) N The generative document system 206 adds corresponding answer card matches to each section. For example, the first topic includes two answer cards: a BestTimeToVisitCard and an ImageAnswer card. It is worth noting that in the illustrated implementation, the large generative model 230 matches different answer cards with different topics to prevent the same answer card from appearing in different sections. In some implementations, answer cards are included in different sections.

[0088] In various implementations, the large generative model 230 can indicate a specific text string corresponding to a particular answer card. In these implementations, the large generative model 230 can indicate the string to which the answer card belongs.

[0089] Figure 5AThe illustration shows an example sequence of diagrams illustrating the use of a large generative model to determine layout guidelines according to some implementation methods. As shown, Figure 5A This includes a sequence of actions 500 performed by or in conjunction with the generative document system 206. In some implementations, the sequence of actions 500 may include fewer or different actions (e.g., some actions may be skipped or omitted). Additionally, in some instances, the actions in the sequence of actions 500 are performed in a different order.

[0090] As shown, action sequence 500 begins with action 502 of generative document system 206 maintaining an unformatted generative document with a topic section and a list of answer cards. (As previously combined...) Figure 4A As described, the generative document system 206 uses a large generative model 230 to generate an unformatted generative document with a topic section and a list of answer cards. In some instances, action sequence 500 omits action 502.

[0091] Action 504 includes the generative document system 206 providing layout cues for topic sections to the large generative model 230. For example, the generative document system 206 instructs the large generative model 230 in the generated layout cues to generate layout guidelines for each topic section, which provide suggestions for visually pairing textual narrative responses with matching answer cards. In some embodiments, the layout guidelines indicate which answer cards should be associated with text portions in the textual narrative response of the topic section. In various embodiments, the layout guidelines provide logical ordering, grouping types, and positioning for topic sections. Additionally, in some embodiments, the layout guidelines rank based on priority and / or importance.

[0092] In some implementations, the generative document system 206 provides a set or list of layout templates and generated layout hints to a large generative model 230. The generated layout hints instruct the large generative model 230 to generate layout guidelines that conform to the layout templates for each topic section. In some implementations, the layout guidelines suggest which types of visual arrangements are best suited for the content. For example, if the topic section is comparing items, the layout guidelines recommend a column-based arrangement for placing different parts of the textual narrative response and their corresponding answer cards (e.g., images of the items) in their own columns. As another example, if the topic section is providing information, the layout guidelines suggest the types of answer cards that should be used to optimize flow and readability.

[0093] Action 506 illustrates that the generative document system 206 receives layout suggestions for subject sections from the large generative model 230. In some instances, layout guidelines are provided separately based on individual calls to the large generative model 230. In some implementations, layout guidelines for all subject sections in the generative document are provided together.

[0094] In various implementations, the layout guide does not include visual arrangement instructions. More specifically, while the layout guide recommends which answer cards are best suited for each section of the theme, it does not provide grid placement or coordinate suggestions for the precise placement of answer cards within a theme section. However, in some implementations, the layout guide may also include suggested template layouts for one or more theme sections.

[0095] In implementations where the layout guidelines do not provide visual arrangements, generative document system 206 generates a list of layouts for visual arrangements of thematic sections, as shown in action 508. For example, generative document system 206 utilizes a layout output generator that determines the visual layout for each thematic section by pairing textual responses with matching answer cards according to the layout guidelines. By following the layout guidelines, generative document system 206 combines the content of the thematic sections with one or more corresponding answer cards in the most logical manner.

[0096] In some implementations, the answer cards have multiple available grid sizes or dimensions. In these implementations, the generative document system 206 takes each available size into account when generating the template layout. Along with the dimensions, the generative document system 206 also considers the different content and / or answer granularity within different versions of the answer cards. In various implementations, the generative document system 206 generates multiple template layouts in parallel.

[0097] Generative document system 206 can generate multiple layout templates for each topic section. For example, generative document system 206 generates different layout options that include different combinations of answer cards (or versions of answer cards) selected from answer cards previously matched to that section by the large generative model 230. In various implementations, each layout option follows the same grid width, but the height can vary based on the combination of answer cards included in the layout option. In some implementations, generative document system 206 generates template layouts for multiple topic sections in parallel.

[0098] In various implementations, the generative document system 206 ranks or sorts the layout options for each topic section. Additionally, the generative document system 206 may store a limited number of layout options for each section. In some implementations, the generative document system 206 also stores fail-safe or alternate layout options that include a textual response without any answer cards.

[0099] In many implementations, the generative document system 206 stores answer card placeholders in each layout option, rather than the answer cards themselves. The placeholders define the dimensions and position of the answer cards within the layout option. However, the generative document system 206 does not need to store the contents of the answer cards, thus saving memory and storage. Furthermore, when a layout option is selected while creating a new generative document for a client device, the placeholders allow the generative document system 206 to dynamically populate the answer cards with the latest information.

[0100] Action 510 includes generative document system 206 generating a formatted generative document template with thematic sections and their visual layout. For example, generative document system 206 aggregates or constructs one or more layout options from the layout options for each thematic section into a formatted generative document template, where each layout option provides a visual arrangement of content including textual narrative responses and answer cards. Using this formatted template for a search query, generative document system 206 can quickly create formatted generative documents.

[0101] To illustrate, action 512 includes the generative document system 206 providing a formatted generative document to the client device 250 by selecting a visual layout for each topic section. For example, the generative document system 206 selects the highest-ranking layout option for each topic section, populates the layout option with the corresponding answer card, and provides a formatted generative document to the client device 250 in response to a search query.

[0102] In some implementations, one or more answer cards may not be available in the visual layout. For example, the visual layout references an image that returns an error message when accessed. In these implementations, the generative document system 206 selects another layout option for the topic section, such as the next highest-ranking layout. In this way, the generative document system 206 quickly and dynamically creates generative documents.

[0103] Figure 5B The illustration shows an example of a layout for generating a theme section according to some implementation methods. Figure 5B Examples of various elements and outputs corresponding to the generation of unformatted generative documents are provided. As shown, Figure 5BThis includes a sequence of actions 550 performed by a generative document system 206 for answer card matching using a large generative model 230.

[0104] Figure 5B The action sequence 550 includes the generative document system 206 providing various versions of grid layouts 552 and template options 554 to the large generative model 230. In various implementations, the grid layout 552 includes available sizes and / or dimensions for the entire formatted generative document. The template options 554 provide various arrangement options for subject sections. For example, the template options 554 provide different content arrangements on rows and columns within a subject section.

[0105] Additionally, the generative documentation system 206 provides the large generative model 230 with generated layout hints 132, unformatted generative search results documents 118, and answer card lists 556 for each topic section. The generated layout hints 132 instruct the large generative model 230 to generate topic guides for each topic section based on the input, as described above.

[0106] As previously mentioned, in some implementations, the large generative model 230 generates layout guidelines for each topic section, and the generative document system 206 uses these layout guidelines to generate layout options. In some implementations, the large generative model 230 may directly generate layout options. In either case, the large generative model 230 uses the previous LGM output in an aggregated manner to generate layout guidelines (e.g., the aggregation framework includes providing the unformatted generative search results document 118 and topic section 556 back to the large generative model 230).

[0107] As shown, action sequence 550 causes generative document system 206 to generate layout options 558 for the topic section. All the generative layout options shown are for the same topic section; however, each version has a different combination of answer cards. Specifically, the first three versions include common answer cards, but the others are different. Furthermore, the last version is an alternative option that does not include any answer cards, to ensure that the topic section is displayed even if answer cards are unavailable in the other layout options.

[0108] Figure 6 The illustration shows an example sequence diagram of providing formatted generative documents for a previously searched query, according to some implementation methods. As shown, Figure 6This includes a sequence of actions 600 performed by or in conjunction with the generative document system 206. In some implementations, the sequence of actions 600 may include fewer or different actions (e.g., some actions may be skipped or omitted). Additionally, in some instances, the actions in the sequence of actions 600 are performed in a different order.

[0109] Action 602 illustrates that the generative document system 206 stores formatted generative document templates with subject sections and their visual layouts. For example, in conjunction with providing a formatted generative document to a client device 250, the generative document system 206 stores or caches the formatted generative document templates in a data repository for future retrieval. In particular, the generative document system 206 stores formatted generative document templates for search queries, which include one or more visual layout options for each subject section.

[0110] In some instances, generative document system 206 generates and stores formatted generative document templates based on previous or historical searches. For example, generative document system 206 searches query logs to identify frequently requested search queries. For the identified queries, generative document system 206 generates and stores formatted generative document templates.

[0111] Action 604 illustrates that the generative document system 206 receives a search query associated with a stored formatted generative document template from the client device 250. For example, a user request associated with the client device 250 corresponds to the same or similar search query for the stored formatted generative document template. The search query may be re-requested by a previous user or a new request by a new user.

[0112] Action 606 illustrates the generative document system 206 requesting and receiving search result links. For example, as previously described, the generative document system 206 obtains a set of result links from the search result link system 228 in response to a search request. As described above... Figure 3A As described in Actions 302 and 304, the set of search result links can be sorted or ranked based on search result scores.

[0113] Action 608 illustrates that generative document system 206 retrieves a stored formatted generative document template associated with a search query. For example, when a search query is received, generative document system 206 checks a data repository for a cached version of the formatted generative document template for the search results. Action 608 can be performed before, after, or simultaneously with action 606.

[0114] Action 610 illustrates that the generative document system 206 determines that the formatted generative document template is up-to-date for the search query. In various implementations, the generative document system 206 verifies the freshness of the formatted generative document template before determining or deciding whether to use the template in response to the search results. If the formatted generative document template is not up-to-date, the generative document system 206 may generate and store a new template for the search query.

[0115] In various implementations, generative document system 206 determines whether a formatted generative document template is up-to-date based on age or creation date (or last modification date). For example, generative document system 206 verifies whether the template is less than a threshold age. In some implementations, the threshold age depends on the topic of the search results, allowing more historical topics to have a longer age before being considered outdated.

[0116] In some implementations, generative document system 206 determines whether a formatted generative document template is up-to-date by comparing search result links associated with a stored formatted generative document template with newly acquired search result links. For example, generative document system 206 determines whether threshold amounts (e.g., numbers or percentages) of links and / or external real-time signals (e.g., news, stock tickers, weather) between a stored set and a newly received set match.

[0117] For illustration, in one or more embodiments, generative document system 206 compares the first few in each set. n The results are used to determine freshness matches. For example, generative document system 206 determines the first 3 (or 5, 10, 25, etc.) results between two sets to match. In one instance, generative document system 206 requires an exact match. In some instances, generative document system 206 also determines number or percentage matches, such as 3 out of the first 5 links (or another set of numbers) to match.

[0118] In some implementations, generative document system 206 also determines whether the order of search result links has changed between a stored set of search result links and a new set of search result links. For example, if the new search result links have a new link order or ranking, generative document system 206 determines that the stored formatted generative document template is outdated.

[0119] In various implementations, generative document system 206 uses a freshness algorithm to compare whether search result links have changed between a stored set of search result links and a new set of search result links. For example, generative document system 206 selects the top results from each set of search result links... nThe results (e.g., the top 3, 5, 8, or 10) are hashed to generate a freshness value based on which links are included in the set and their ranking or order. The generative document system 206 then compares the freshness values ​​between the stored set of search results links and the new set of search results links to determine a match. In some instances, the freshness algorithm is a fuzzy algorithm that allows for minor changes between the two sets (e.g., the same links in a different order).

[0120] By using freshness values, generative document system 206 can store formatted generative document templates instead of storing sets of search result links, thus saving computational resources. For example, when storing a formatted generative document template for a search query, a freshness value for the formatted generative document template is associated with the search query. Then, when a client device searches for the same search query again, generative document system 206 only needs to compare the current freshness value of the search result with the stored freshness value to determine a match. In some implementations, generative document system 206 verifies the freshness of the stored formatted generative document template before retrieving it from a data repository, saving bandwidth.

[0121] If the formatted generative document template is up-to-date or new, the generative document system 206 uses it to provide a formatted generative document in response to a search query. Otherwise, the generative document system 206 generates a new formatted generative document template. For illustration, action 612 shows the generative document system 206 selecting a valid visual layout for each topic section from its layout list.

[0122] Specifically, action 612 includes generative documentation system 206 generating a valid formatted generative document from a stored formatted generative document template. For example, for the topic section, generative documentation system 206 identifies the first listed or highest-ranked visual layout option. For answer card placeholders within a visual layout option, generative documentation system 206 verifies that the resource associated with the placeholder is still available (e.g., image placeholders are linked to valid image files). If so, generative documentation system 206 selects that visual layout option for the topic section.

[0123] If an answer card is unavailable within the visual layout options, the generative document system 206 moves to the second listed (or the next highest-ranked) visual layout option and repeats the above validation process until a valid visual layout option is selected for the topic section. If necessary, the generative document system 206 selects a failover or backup visual layout option without an answer card. By moving to the next visual layout option when validation at a higher ranking fails, the generative document system 206 quickly and efficiently determines the visual layout option to select for each topic section.

[0124] After selecting a visual layout option for each topic section, generative document system 206 renders each section to dynamically generate a formatted generative document for the search query. Generative document system 206 then provides it to client device 250 in response to the search query. For illustration, action 614 shows generative document system 206 providing a formatted generative document to client device 250, which renders the selected visual layout for each topic section.

[0125] In various implementations, generative document system 206 provides a newly retrieved set of search result links to a newly created formatted generative document. For example, the generative document references one or more search result links in the search result links. If the order of the search result links has changed, generative document system 206 may cause the set of search result links to be reordered to match the generative document and / or the numbering in the topic sections to match the new search result link order.

[0126] Figure 7 The illustration depicts an example graphical user interface according to some implementations, which provides generative documents and a collection of search result links based on a search query. As shown, Figure 7 The system includes a client device 700 with a graphical user interface 702 and a client application 704. The client application 704 allows the user to interact with the image generation system to receive generative documents for a search query. For example, the client application 704 could be a web browser that enables the user to submit search queries.

[0127] As shown, client application 704 displays the results of the search request, which include a generative document 710 corresponding to the search results link set 708. In this example, the generative document 710 and the search results link set 708 are included within the SERP. In some implementations, the SERP includes only the generative document 710. For example, the search results link set 708 may be omitted, located on a separate page, or selectively displayed based on user interaction with corresponding elements of the generative document.

[0128] Generative document 710 includes a heading section 712 at the top. In some embodiments, the heading section includes an interactive link to a main topic section 714 provided in the generative document 710. In various embodiments, the generative document system 206 considers the heading section 712 as the first main topic section of the generative document.

[0129] As shown, the topic section 714 includes a textual description response that provides a colloquial description of the set of search result links categorized by topic. Specifically, the textual description response includes citation numbers that link to corresponding numbers shown within the search result link set 708. In some implementations, the topic section provides first-level details and options for further exploration of the topic. If this option is selected, the topic section expands to display additional content.

[0130] The topic section 714 also includes answer cards 716. Answer cards may include images, graphics, text, and / or links. In some instances, answer cards include additional queries. In some instances, answer cards also link to one or more search result links in the search result link set 708.

[0131] As described above, generative document 710 serves as a curated document based on search result link set 708, which is constructed using a large generative model. Generative document 710 facilitates rapid understanding of the search result link set 708 in a narrative and interactive format. As the search result link set 708 changes over time, generative document system 206 also updates generative document 710 to maintain accuracy and up-to-dateness.

[0132] Now go to Figure 8 The figure illustrates an example sequence of actions in a computer-implemented method for generating search results documents using a large generative model (LGM) according to some implementations. Although Figure 8 The illustrations depict actions according to one or more embodiments, but alternative embodiments may omit, add, reorder, and / or modify any actions shown.

[0133] Figure 8 The actions described can be performed as part of a method (e.g., a computer-implemented method). Alternatively, the computer-readable medium can include instructions that, when executed by a processing system having a processor, cause a computing device to perform... Figure 8 The actions performed within. In some implementations, the system (e.g., a processing system including a processor) can perform... Figure 8 The system includes, for example, a processing system and a computer memory containing instructions that, when executed by the processing system, cause the system to perform various actions or steps.

[0134] As shown, action sequence 800 includes action 810 of providing a generative document request prompt and a set of search result links to a first LGM. For example, in an example implementation, action 810 involves providing a generative document request prompt and a set of search result links to the first LGM, wherein the set of search result links is based on a search query. In some implementations, action 810 includes receiving a search query from a client device associated with a user, providing the search query to a search query system, and receiving the set of search result links from the search query system. In some implementations, action 810 includes providing the set of search result links and a search intent prompt to the LGM, receiving a set of potential search intents from the LGM, selecting a search intent from the set of potential search intents, and providing the search intent, the generative document request prompt, and the set of search result links to the first LGM.

[0135] As further illustrated, action sequence 800 includes action 820 of receiving an unformatted generative search results document from a first LGM. For example, in an example implementation, action 820 involves receiving an unformatted generative search results document from a first LGM having a first topic section and a second topic section, wherein each of the first and second topic sections associates a specific topic textual response with search results links from a set of search results links. In some implementations, the determination of the first and second topic sections (e.g., the first LGM determines the first and second topic sections) is based on analysis or processing of the set of search results links; the first topic section includes a first textual response corresponding to a first subset of the search results links; and / or the second topic section includes a second textual response corresponding to a second subset of the search results links, the first subset being different from the second subset.

[0136] As further illustrated, action sequence 800 includes action 830 of receiving a first set or list of answer cards from the second LGM based on providing answer card association hints to the second LGM. For example, in an example implementation, action 830 involves receiving a first set or list of answer cards from the second LGM based on providing answer card association hints to the second LGM, a first topic section, and a set of candidate answer cards associated with the first topic section, wherein the first set or list of answer cards associates a first textual description response for the first topic section with answer cards within the candidate answer card set. In some implementations, action 830 includes providing a first answer card query for the first topic section to the answer card system and receiving a set of candidate answer cards associated with the first topic section from the answer card system. In some implementations, the first set or list of answer cards from the second LGM includes available dimensions for each answer card within the first set or list of answer cards.

[0137] As further illustrated, action sequence 800 includes action 840 of generating a first layout for the first topic section using a first set or list of generative layout guidelines generated by a third LGM for the first topic section. For example, in an example implementation, action 840 involves generating a first layout for the first topic section using a first set or list of generative layout guidelines generated by a third LGM for the first topic section based on providing generative layout prompts to the third LGM, wherein the first layout visually positions a first textual narrative response with a first combination from a first set or list of answer cards.

[0138] In some embodiments, action 840 includes generating an additional layout for the first topic section using a first set or list of generative layout guidelines, wherein the additional layout visually positions the first textual response in combination with a second set or list of answer cards, the second answer card combination being different from the first answer card combination. In some embodiments, action 840 also includes generating an alternative layout for the first topic section that displays the first textual response in the absence of any answer cards from the first set or list of answer cards. In some embodiments, at least two of the first LGM, second LGM, and third LGM are identical.

[0139] In some instances, in conjunction with action 840, a first generative layout guide set or list provides a ranked layout order between the portions of the textual narrative response in the first topic section and the combination of answer cards from the first answer card set or list. In some implementations, the first topic section is divided into a grid with adjustable height, and / or the first layout is generated based on the available dimensions of the answer cards within the first answer card set or list.

[0140] As further illustrated, action sequence 800 includes action 850 of generating a formatted generated search results document. For example, in an example implementation, action 850 relates to generating a formatted generated search results document including a first layout for a first topic section and a second layout generated for a second topic section. In some implementations, action 850 includes, in response to a search query, providing a formatted generated search results document adjacent to the set of search results links to a client device, wherein the formatted generated search results document is generated based on the set of search results links. In some implementations, action 850 also includes modifying the ranking order of the set of search results links to prioritize the search results links referenced in the formatted generated search results document.

[0141] In some implementations, action 850 includes storing a formatted, generated search results document; receiving an additional instance of a search query; comparing a set of additional search results links associated with the additional instance of the search query with a set of search results links associated with the search query; and / or, in response to the additional instance of the search query, providing a formatted, generated search results document from a storage device based on the additional search results link set and the search results link set meeting a relevance threshold. In some implementations, action 850 also includes comparing the additional search results link set with the search results link set by selecting multiple search links as a relevance threshold and matching multiple search links between the additional search results link set and the search results link set in ranking order.

[0142] In some implementations, action 850 includes storing a formatted generative search results document; receiving an additional instance of a search query; determining a mismatch between the ranking order of one or more links from a set of additional search results links associated with the additional instance of the search query and the search results links; and generating an updated formatted generative search results document based on the set of additional search results links. In some implementations, action 850 also includes storing the formatted generative search results document, including storing multiple layouts for a first topic section. In some implementations, action 850 includes retrieving a stored version of the formatted generative search results document in response to receiving an additional instance of the search query; determining that an answer card in a first layout of the first topic section is unavailable or problematic; and providing additional layouts for the first topic section to the formatted generative search results document in response to an additional instance of the search query.

[0143] In some instances, action sequence 800 includes (or is replaced by) receiving an unformatted generative search results document from a first LGM, the unformatted generative search results document having a topic section having a specific topic textual narrative response determined from a set of search results links corresponding to a search query; receiving a set or list of answer cards for the topic section from the topic section in response to providing an answer card association hint, the textual narrative response from the topic section, and a set of candidate answer cards associated with the topic section to a second LGM; generating multiple layouts for the topic section using a set or list of generative layout guidelines generated by the third LGM for the topic section based on providing a generative layout hint to a third LGM, wherein the multiple layouts visually position different combinations of the textual narrative response with answer cards from the set or list of answer cards; and generating a formatted generative search results document including the layouts for the topic section from the multiple layouts.

[0144] In some instances, action sequence 800 includes (or is replaced by) providing a generative document request prompt and a set of search result links to an LGM (e.g., a fourth LGM), wherein the set of search result links is based on a search query; receiving an unformatted generative search result document from the LGM, the unformatted generative search result document having a first topic section and a second topic section, each topic section associating a specific topic textual description response with search result links from the set of search result links; and receiving a first set of answer cards from the LGM based on providing the LGM with answer card association prompts, the first topic section, and a set of candidate answer cards associated with the first topic section. The first set or list of answer cards associates a first textual response for a first topic section with answer cards within that set of candidate answer cards; based on providing generative layout hints to the LGM, a first layout for the first topic section is generated using a first set or list of generative layout guidelines generated by the LGM for the first topic section, wherein the first layout visually positions the first textual response with first answer cards from the first set or list of answer cards; and a formatted generative search results document is generated, which includes the first layout for the first topic section and a second layout generated for the second topic section.

[0145] Figure 9 The illustration shows certain components that may be included within a computer system 900. The computer system 900 can be used to implement various computing devices, components, and systems described herein (e.g., by executing computer-implemented instructions). As used herein, a “computing device” means an electronic component that performs a set of operations based on a set of programmed instructions. Computing devices include groups of electronic components, client devices, server devices, etc.

[0146] In various implementations, computer system 900 refers to one or more of the client devices, server devices, or other computing devices described above. For example, computer system 900 can refer to various types of network devices capable of accessing data on a network, cloud computing system, or another system. For example, client device can refer to mobile devices such as mobile phones, smartphones, personal digital assistants (PDAs), tablets, laptops, or wearable computing devices (e.g., headphones or smartwatches). Client device can also refer to non-mobile devices such as desktop computers, server nodes (e.g., from another cloud computing system), or other non-portable devices.

[0147] Computer system 900 includes a processing system, which includes a processor 901. Processor 901 may be a general-purpose single-chip or multi-chip microprocessor (e.g., an Advanced Reduced Instruction Set Computer (RISC) machine (ARM)), a special-purpose microprocessor (e.g., a Digital Signal Processor (DSP)), a microcontroller, a programmable gate array, etc. Processor 901 may be referred to as a Central Processing Unit (CPU) and enables the execution of computer-implemented instructions. Although the processor 901 shown is only a... Figure 9 The computer system 900 uses a single processor, but in alternative configurations, a combination of processors (e.g., ARM and DSP) can be used.

[0148] The computer system 900 also includes a memory 903 that is in electronic communication with the processor 901. The memory 903 can be any electronic component capable of storing electronic information. For example, the memory 903 can be embodied as random access memory (RAM), read-only memory (ROM), magnetic disk storage media, optical storage media, flash memory devices in RAM, on-board memory included in the processor, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, etc., including combinations thereof.

[0149] Instruction 905 and data 907 may be stored in memory 903. Instruction 905 may be executed by processor 901 to implement some or all of the functions disclosed herein. Execution of instruction 905 may involve using data 907 stored in memory 903. Any example of the various examples of modules and components described herein may be implemented in part or in whole as instruction 905 stored in memory 903 and executed by processor 901. Any example of the various examples of data described herein may be data in data 907 stored in memory 903 and used by processor 901 during execution of instruction 905.

[0150] The computer system 900 may also include one or more communication interfaces 909 for communicating with other electronic devices. The one or more communication interfaces 909 may be based on wired communication technology, wireless communication technology, or both. Some examples of the one or more communication interfaces 909 include Universal Serial Bus (USB), Ethernet adapters, wireless adapters operating according to the Institute of Electrical and Electronics Engineers (IEEE) 902.11 wireless communication protocol, Bluetooth® wireless communication adapters, and infrared (IR) communication ports.

[0151] Computer system 900 may also include one or more input devices 911 and one or more output devices 913. Some examples of one or more input devices 911 include a keyboard, mouse, microphone, remote control device, button, joystick, trackball, touchpad, and light pen. Some examples of one or more output devices 913 include speakers and printers. A particular type of output device typically included in computer system 900 is a display device 915. The display device 915 used with the embodiments disclosed herein can utilize any suitable image projection technology, such as liquid crystal display (LCD), light-emitting diode (LED), gas plasma, electroluminescence, etc. A display controller 917 may also be provided for converting data 907 stored in memory 903 into text, graphics, and / or moving images (if appropriate) displayed on display device 915.

[0152] Various components of the computer system 900 can be coupled together via one or more buses, which may include power buses, control signal buses, status signal buses, data buses, etc. For clarity, the various buses are... Figure 9 The diagram shows the 919 bus system.

[0153] This disclosure describes a subjective data application system within a network framework. In this disclosure, "network" refers to one or more data links that enable electronic data transmission between computer systems, modules, and other electronic devices. A network can include public networks such as the Internet as well as private networks. When information is transmitted or provided through a network or another communication connection (hardwired, wireless, or both), the computer correctly regards that connection as a transmission medium. The transmission medium can include networks and / or data links carrying desired program code in the form of computer-executable instructions or data structures, which can be accessed by general-purpose or special-purpose computers.

[0154] Furthermore, the network described herein can refer to a network or combination of networks (such as the Internet, corporate intranet, virtual private network (VPN), local area network (LAN), wireless local area network (WLAN), cellular network, wide area network (WAN), metropolitan area network (MAN), or a combination of two or more such networks) through which one or more computing devices can access the various systems described in this disclosure. In practice, the network described herein can include one or more networks that use one or more communication platforms or technologies to transmit data. For example, a network can include the Internet or other data links that enable the transmission of electronic data between corresponding client devices and components of a cloud computing system (e.g., server devices and / or virtual machines on them).

[0155] Furthermore, upon arrival at various computer system components, program code in the form of computer-executable instructions or data structures can be automatically transferred from the transmission medium to a non-transitory computer-readable storage medium (device) and vice versa. For example, computer-executable instructions or data structures received via a network or data link can be buffered in random access memory (RAM) within a network interface module (NIC), and then ultimately transferred to the computer system RAM and / or a non-volatile computer storage medium (device) at the computer system. Therefore, it should be understood that computer-readable storage media (devices) can be included in computer system components that also (or even primarily) utilize the transmission medium.

[0156] Computer-executable instructions include instructions and data that, when executed by a processor, cause a general-purpose computer, a special-purpose computer, or a special-purpose processing device to perform a specific function or group of functions. In some embodiments, a general-purpose computer executes computer-executable and / or computer-implemented instructions to turn the general-purpose computer into a special-purpose computer that implements the elements of this disclosure. Computer-executable instructions may include, for example, binary code, intermediate format instructions such as assembly language, or even source code. Although the subject matter has been described in language specific to structural features and / or methodological actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the features or actions described above. Rather, the described features and actions are disclosed as exemplary forms of implementing the claims.

[0157] Those skilled in the art will understand that this disclosure can be practiced in networked computing environments with many types of computer system configurations, including personal computers, desktop computers, laptop computers, message processors, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframes, mobile phones, PDAs, tablets, pagers, routers, switches, etc. This disclosure can also be practiced in distributed system environments, where local and remote computer systems linked via a network (via a hardwired data link, a wireless data link, or a combination of hardwired and wireless data links) perform tasks. In a distributed system environment, program modules can reside in local and remote memory storage devices.

[0158] The techniques described herein can be implemented in hardware, software, firmware, or any combination thereof, unless specifically described as being implemented in a particular manner. Any features described as modules, components, etc., can also be implemented together in an integrated logic device or separately as discrete but interoperable logic devices. If implemented in software, the techniques can be implemented at least in part by a non-transitory processor-readable storage medium, including instructions that, when executed by at least one processor, perform one or more methods (including computer-implemented methods) described herein. These instructions can be organized into routines, programs, objects, components, data structures, etc., which can perform specific tasks and / or implement specific data types, and can be combined or distributed in various implementations as needed.

[0159] Computer-readable media can be any available medium accessible by a general-purpose or special-purpose computer system. A computer-readable medium storing computer-executable instructions is a non-transitory computer-readable storage medium (device). A computer-readable medium carrying computer-executable instructions is a transmission medium. Therefore, by way of example, embodiments of this disclosure may include at least two distinct types of computer-readable media: non-transitory computer-readable storage media (devices) and transmission media.

[0160] As used herein, a computer-readable storage medium (device) may include RAM, ROM, EEPROM, CD-ROM, solid-state drive (SSD) (e.g., RAM-based), flash memory, phase-change memory (PCM), other types of memory, other optical disc storage, disk storage or other magnetic storage devices, or any other medium that may be used to store desired program code in the form of computer-executable instructions or data structures and that may be accessed by a general-purpose or special-purpose computer.

[0161] Without departing from the scope of the claims, the steps and / or actions of the methods described herein may be interchanged with each other. In other words, unless the correct operation of the described methods requires a specific order of steps or actions, the order and / or use of specific steps and / or actions may be modified without departing from the scope of the claims.

[0162] The term "determine" encompasses a variety of actions; therefore, "determine" can include calculation, operation, processing, derivation, investigation, searching (e.g., looking in a table, database, or other data structure), ascertainment, etc. Furthermore, "determine" can include receiving (e.g., receiving information), accessing (e.g., accessing data in memory), etc. Additionally, "determine" can include parsing, selecting, choosing, building, etc.

[0163] The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that other elements besides those listed may be present. Additionally, it should be understood that references to “one embodiment” or “implementation” in this disclosure are not intended to exclude the existence of additional embodiments that also incorporate the described features. For example, where compatible, any element or feature described with respect to an embodiment herein may be combined with any element or feature of any other embodiment described herein.

[0164] This disclosure may be embodied in other specific forms without departing from the spirit or characteristics thereof. The described embodiments are to be considered illustrative rather than restrictive. The scope of this disclosure is indicated by the appended claims rather than the foregoing description. Changes within the meaning and scope of equivalents of the claims will be included within their scope.

Claims

1. A computer-implemented method for generating search result documents using a large generative model (LGM), comprising: Provide the first LGM (116) with a generative document request prompt (112) and a set of search results links (114), wherein the set of search results links (114) is based on the search query (110). An unformatted generative search results document (118) is received from the first LGM (116), the unformatted generative search results document having a first topic section (120) and a second topic section (122), each of the first topic section (120) and the second topic section (122) associating a particular topic textual narrative response with search results links from the search results link set (114); Based on providing the second LGM (128) with answer card association prompts (126), the first topic section (120), and a set of candidate answer cards (124) associated with the first topic section (120), a first answer card set (130) is received from the second LGM (128), wherein the first answer card set (130) associates a first textual narrative response for the first topic section (120) with answer cards within the candidate answer card set (124); Based on providing generative layout hints to the third LGM (134), a first layout (138) for the first theme section (120) is generated using a first set of generative layout guidelines (136) generated by the third LGM (134) for the first theme section (120), wherein the first layout (138) visually positions the first textual narrative response in conjunction with a first answer card from the first set of answer cards (130); as well as A formatted generative search results document (140) is generated, the formatted generative search results document including a first layout (138) for the first topic section (120) and a second layout for the second topic section (122).

2. The computer-implemented method according to claim 1, wherein: The first topic section and the second topic section are determined based on the analysis of the set of search results links; The first topic section includes the first textual narrative response corresponding to a first subset of the search results links; and The second topic section includes a second textual narrative response corresponding to a second subset linked to the search results, the first subset being different from the second subset.

3. The computer-implemented method according to claim 1, further comprising: Provide the answer card system with a first answer card query for the first topic section; as well as Receive the set of candidate answer cards associated with the first topic section from the answer card system.

4. The computer-implemented method of claim 1, wherein the first generative layout guide set provides a ranked layout order between portions of the textual narrative response in the first topic section and combinations of answer cards from the first answer card set.

5. The computer-implemented method of claim 1, wherein the first set of answer cards from the second LGM includes available sizes of answer cards for each answer card within the first set of answer cards.

6. The computer-implemented method according to claim 5, wherein: The first subject area is divided into a grid with adjustable height; and The first layout is generated based on the available sizes of the answer cards within the first set of answer cards.

7. The computer-implemented method according to claim 1, further comprising: In response to the search query, a formatted generative search results document adjacent to the set of search results links is provided to the client device, wherein the formatted generative search results document is generated based on the set of search results links.

8. The computer-implemented method according to claim 6, further comprising: Modify the ranking order of the search result link set to prioritize the search result links referenced in the formatted generative search result document.

9. The computer-implemented method according to claim 8, further comprising: Store the formatted generative search results document; Receive additional instances of the search query; The set of additional search result links associated with the additional instance of the search query is compared with the set of search result links associated with the search query; and Based on the fact that the set of additional search results links and the set of search results links meet a relevance threshold, in response to the additional instance of the search query, the formatted generative search results document is provided from the storage device.

10. The computer-implemented method of claim 9, further comprising comparing the additional search result link set with the search result link set in the following manner: Select multiple search links as the relevance threshold; and The plurality of search links are matched in ranking order between the additional search result link set and the search result link set.

11. The computer-implemented method according to claim 8, further comprising: Store the formatted generative search results document; Receive additional instances of the search query; Determine that the ranking order of one or more links from the set of additional search result links associated with the additional instance of the search query does not match the search result link; and Based on the set of additional search result links, an updated, formatted, generative search result document is generated.

12. The computer-implemented method according to claim 11, further comprising: The formatted, generative search results document is stored, including multiple layouts for the first topic section.

13. The computer-implemented method of claim 1, wherein at least two of the first LGM, the second LGM, and the third LGM are the same LGM.

14. A system for generating search result documents using a large generative model (LGM), comprising: Processing system (900); as well as Computer memory (903), the computer memory including instructions (905), which, when executed by a processing system, cause the system to perform the following operations: Receive an unformatted generative search results document (118) from the first LGM (116), the unformatted generative search results document having a topic section having a specific topic textual narrative response determined from a set of search results links (114) corresponding to the search query (110); In response to providing the second LGM (128) with answer card association prompts (126), a textual narrative response from the topic section, and a set of candidate answer cards associated with the topic section (124), the set of answer cards for the topic section from the topic section is received (130). Based on providing generative layout hints to the third LGM (134), a plurality of layouts for the topic section are generated using a set of generative layout guidelines (136) generated by the third LGM (134) for the topic section, wherein the plurality of layouts visually position the textual narrative response in combination with different answer cards from the set of answer cards (130). as well as A formatted generative search results document (140) is generated, the formatted generative search results document including layouts from the plurality of layouts for the topic section.

15. The system of claim 14, wherein the operation further comprises: The search query is received from the client device associated with the user; Provide the search query to the search query system; as well as Receive the set of search results links from the search query system.

16. The system of claim 15, wherein the operation further comprises: The set of search results links is provided to a fourth LGM with search intent indications; Receive a set of potential search intents from the fourth LGM; Select a search intent from the set of potential search intents; as well as The search intent, generative document request prompts, and the set of search result links are provided to the first LGM.

17. A computer-implemented method for generating search result documents using a large generative model (LGM), comprising: Provide LGM with a generative document request prompt (112) and a set of search results links (114), wherein the set of search results links (114) is based on the search query (110). Receive an unformatted generative search results document (118) from the LGM, the unformatted generative search results document having a first topic section (120) and a second topic section (122), the first topic section and the second topic section each associating a specific topic textual narrative response with a search results link from the search results link set (114); Based on providing the LGM with answer card association prompts (126), the first topic section (120), and a set of candidate answer cards (124) associated with the first topic section (120), a first answer card set (130) is received from the LGM, wherein the first answer card set (130) associates the first textual narrative response of the first topic section (120) with the answer cards in the set of candidate answer cards (124); Based on providing generative layout hints to the LGM, a first layout (138) for the first theme section (120) is generated using a first set of generative layout guidelines (136) generated by the LGM for the first theme section (120), wherein the first layout (138) visually positions the first textual narrative response in combination with a first answer card from the first set of answer cards (130). as well as A formatted generative search results document (140) is generated, the formatted generative search results document including the first layout (138) for the first topic section (120) and the second layout generated for the second topic section (122).

18. The computer-implemented method according to claim 17, further comprising: The first set of generative layout guidelines is used to generate an additional layout for the first topic section, wherein the additional layout visually positions the first textual narrative response together with a second set of answer cards from the first set of answer cards, the second set of answer cards being different from the first set of answer cards.

19. The computer-implemented method according to claim 18, further comprising: In response to receiving an additional instance of the search query, retrieve a stored version of the formatted generative search results document; It was determined that the answer cards in the first layout of the first topic section were unavailable; and In response to the additional instance of the search query, the additional layout for the first topic section is provided to the formatted generative search results document.

20. The computer-implemented method according to claim 18, further comprising: Generate an alternative layout for the first topic section, which displays the first textual narrative response in the absence of any answer cards from the first set of answer cards.

Images