Streaming Generative Language Model Output in Search Results
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional search engines and generative language models (GLMs) are inadequate in providing accurate and relevant information that requires reasoning over multiple webpages or information sources, leading to incorrect outputs.
Innovation Solution
The integration of a GLM with a search engine to generate streaming content within a Search Engine Results Page (SERP), where the prompt includes user input, search results, and previous dialog turns, enabling the GLM to reason and provide accurate outputs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a conventional search engine returns knowledge cards and links to webpages, then the user can access information, but the user must manually compute answers and the information accuracy cannot be verified
Solution Approach 1:
The patent introduces an intermediary verification mechanism that cross-references GLM-generated answers against multiple search result sources. This intermediary layer validates the accuracy of information before presenting it to the user, resolving the contradiction between ease of information access and information reliability.
2Speed
If a GLM generates output based on training data, then the output is generated in near real-time, but the output may be incorrect when reasoning over multiple information sources
Solution Approach 1:
The patent implements continuous verification where the GLM's real-time generation is continuously cross-checked against search results throughout the generation process, not just after completion. This maintains both the speed advantage of real-time generation and the reliability of accurate output by performing validation continuously during the generation action.
Solution Approach 2:
The system incorporates feedback loops where search results are used to verify and potentially correct GLM output in real-time. The verification feedback mechanism ensures that while the GLM generates quickly, any inaccuracies are detected and corrected, maintaining both speed and reliability.
3Adaptability or versatility
If conventional search engines return multiple links and knowledge cards, then various information is provided, but the engine cannot reason over content to produce synthesized answers
Solution Approach 1:
The patent merges the strengths of conventional search engines (providing diverse information types through knowledge cards and links) with the reasoning capabilities of GLMs. The system combines search result data with GLM generation to produce synthesized answers that leverage both information retrieval versatility and analytical productivity.
4Reliability
If the GLM processes complete search results before generating output, then the output is more accurate, but the response time increases
Solution Approach 1:
The patent performs preliminary actions by pre-processing and indexing search results before they are needed for generation. This allows the verification process to occur more efficiently during output generation, maintaining accuracy while reducing the time penalty associated with processing complete search results.
Data Source
AI summary
A computing system is disclosed that includes a processor and memory. The memory stores instructions that, when executed by the processor, cause the processor to perform several acts. The acts include generating a prompt that is to be input to a generative language model. The prompt includes conversational input set forth by a user. The acts further comprise providing the prompt as input to the generative language model, and receiving conversational output from the generative language model, where the generative language model generated the conversational output based upon the prompt. Additionally, the acts comprise streaming the conversational output on one of a SERP or webpage to which the user has navigated from the SERP.