Active query and content suggestion for question answering generated with a generative model

Abstract

Systems and methods for active query and content suggestion can include obtaining web data, determining an occurring change event, and generating a query and content suggestion. Generating a query and content suggestion can include processing data describing a change event using a generative model to generate one or more model-generated query suggestions. One or more web resources can be obtained and then processed to generate a change event summary. The one or more query suggestions and the change event summary can then be provided for display.

Description

[0001] Related Applications

[0002] This application claims priority to and the benefit of U.S. Provisional Patent Application No. 63 / 584,015, filed September 20, 2023. U.S. Provisional Patent Application No. 63 / 584,015 is hereby incorporated by reference in its entirety. TECHNICAL FIELD

[0003] The present disclosure relates generally to proactive query and content suggestions. More specifically, the present disclosure relates to determining that an event has occurred, generating suggested queries using a generative language model, determining suggested content, and providing query and content suggestions. BACKGROUND

[0004] It can be difficult to keep up with changes and general news related to global and / or interest-specific changes. Furthermore, as new information is continually added to the web, it can become non-intuitive to understand when to seek information and what information to seek. Additionally, content requested by a user can not be readily available for use by the user based on where to search, based on the storage location of the content, and / or based on the content being difficult to understand.

[0005] Existing query suggestion techniques can be outdated as the suggestions can simply be a repetition of past queries or associated with content last viewed. Techniques that provide trend-based suggestions can not provide customized suggestions and the suggested queries can result in a large number of results to be perused and reviewed to determine what is popular. SUMMARY

[0006] Aspects and advantages of embodiments of the present disclosure will be set forth in part in the following description, or can be learned from the description, or can be learned through practice of the embodiments.

[0007] One example aspect of the present disclosure relates to a computer-implemented method for proactive query and content suggestions. The method can include obtaining, by a computing system comprising one or more processors, web data. The web data can describe one or more topics. The method can include processing, by the computing system, the web data to determine an event of change that occurred. The event of change can include an update associated with the one or more topics. The method can include processing, by the computing system, data describing the event of change using a generative model to generate one or more query suggestions. The method can include obtaining, by the computing system, one or more web resources associated with the event of change. The method can include processing, by the computing system, the one or more web resources using the generative model to generate a summary of the event of change and providing, by the computing system, the one or more query suggestions and the summary of the event of change for display.

[0008] In some implementations, the method can include obtaining, by the computing system, user data associated with a user and determining, by the computing system, that the user is interested in one or more particular topics based on the user data. Processing, by the computing system, data describing a change event using a generative model to generate one or more query suggestions can be performed in response to determining, by the computing system, that the change event is associated with the one or more particular topics.

[0009] In some implementations, the method can include determining, by the computing system, a second change event that occurred. The second change event can include an update associated with one or more second topics. The method can include processing, by the computing system, data describing the second change event using a generative model to generate one or more second query suggestions. The method can include obtaining, by the computing system, one or more second web resources associated with the second change event and processing, by the computing system, the one or more second web resources using the generative model to generate a second change event summary. The method can include providing, by the computing system, the one or more second query suggestions and the second change event summary for display. The one or more query suggestions and the change event summary can be provided as a first pair. In some implementations, the one or more second query suggestions and the second change event summary can be provided as a second pair. The first pair and the second pair can be provided in a news feed interface.

[0010] In some implementations, the method can include receiving, by the computing system, a selection of at least one of the one or more query suggestions or the change event summary and providing, by the computing system, a search results interface for display. The search results interface can include a plurality of search results responsive to a search query associated with the one or more query suggestions. The plurality of search results can include one or more web resources. In some implementations, the method can include receiving, by the computing system, a selection of at least one of the one or more query suggestions or the change event summary and providing, by the computing system, a model-generated content item for display. The model-generated content item can be generated using a generative model. The model-generated content item can include information associated with the change event and the information can include details obtained from one or more databases formatted as a natural language article by the generative model. In some implementations, the web data can be obtained prior to receiving the search query. The one or more query suggestions and the change event summary can be provided for display prior to receiving the search query.

[0011] Another example aspect of the disclosure relates to a computing system for proactive querying and content suggestions. The system can include one or more processors and one or more non-transitory computer-readable media collectively storing instructions that, when executed by the one or more processors, cause the computing system to perform operations. The operations can include obtaining a plurality of queries. The plurality of queries can be obtained from a plurality of users. The operations can include processing the plurality of queries to determine a query trend. In some implementations, the query trend can describe an increase in a number of queries associated with a particular topic. The operations can include processing data describing the query trend using a language model to generate one or more query suggestions. The operations can include obtaining one or more web resources responsive to the one or more query suggestions and processing the one or more web resources using the language model to generate a content summary. The content summary can describe content of the one or more web resources. The operations can include providing the one or more query suggestions and the content summary for display.

[0012] In some implementations, the one or more query suggestions can be generated in a question format. The content summary can be generated in an answer format, and the content summary can be generated as a response to the question of the one or more query suggestions. The content summary can include a natural language summary including one or more details obtained from the one or more web resources. In some implementations, processing data describing the query trend using a language model to generate the one or more query suggestions can be performed in response to determining that the particular topic is associated with one or more content items previously viewed by a particular user. The query trend can be a global query trend.

[0013] Another example aspect of the disclosure relates to one or more non-transitory computer-readable media collectively storing instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations. The operations can include obtaining web data. The web data can describe content provided by a collection of web resources. The operations can include processing the web data to determine a plurality of change events that occur. The plurality of change events can include a plurality of updates associated with a plurality of topics. The operations can include processing data describing the plurality of change events using a language model to generate a plurality of query suggestions. The language model can include a generative model. In some implementations, each query suggestion can be associated with a respective change event of the plurality of change events. The operations can include obtaining a plurality of respective web resources associated with the plurality of change events. The plurality of respective web resources can include one or more respective web resources for each of the plurality of change events. The operations can include associating each of the plurality of query suggestions with a respective web resource of the plurality of respective web resources to generate a query suggestion-web resource pair for each of the plurality of change events and providing a news feed interface for display. The news feed interface can include a set of query suggestion-web resource pairs associated with at least a subset of the plurality of change events.

[0014] In some implementations, each of the plurality of change events can be determined based on determining an inflow amount of articles associated with the respective topic. Each of the plurality of change events can be determined based on determining an inflow amount of social media posts associated with the respective topic. The language model can include a machine-learned generation model trained to generate natural language data that is tailored to mimic the vocabulary, tone, and style of a particular user.

[0015] Other aspects of the disclosure relate to various systems, devices, non-transitory computer-readable media, user interfaces, and electronic devices.

[0016] These and other features, aspects, and advantages of various embodiments of the present disclosure will be better understood when the following description is read together with the accompanying drawings and appended claims. The accompanying drawings illustrate example embodiments of the present disclosure and, together with the description, serve to explain the related principles. BRIEF DESCRIPTION OF DRAWINGS

[0017] With reference to the drawings, a detailed discussion of embodiments oriented to those of ordinary skill in the art follows, in which:

[0018] Figure 1 A block diagram of an example query suggestion system according to example embodiments of the present disclosure is depicted.

[0019] Figure 2 A block diagram of an example active query suggestion system according to example embodiments of the present disclosure is depicted.

[0020] Figure 3 A flow diagram of an example method for performing query and content suggestion according to example embodiments of the present disclosure is depicted.

[0021] Figure 4 A block diagram of an example query trend determination system according to example embodiments of the present disclosure is depicted.

[0022] Figure 5 A block diagram of an example query news push system according to example embodiments of the present disclosure is depicted.

[0023] Figure 6 A depiction of an example news push interface according to example embodiments of the present disclosure is depicted.

[0024] Figure 7 A flow diagram of an example method for performing query trend determination and query suggestion according to example embodiments of the present disclosure is depicted.

[0025] Figure 8A flow diagram depicting an example method for performing query news push generation according to example embodiments of the present disclosure is depicted.

[0026] Figure 9 A diagram depicting an example query suggestion interface according to example embodiments of the present disclosure is depicted.

[0027] Figure 10A A block diagram of an example computing system that performs query and content suggestion according to example embodiments of the present disclosure is depicted.

[0028] Figure 10B A block diagram of an example computing system that performs query and content suggestion according to example embodiments of the present disclosure is depicted.

[0029] Reference numerals repeated between the several figures designate the same features in various implementations. DETAILED DESCRIPTION

[0030] Generally, the present disclosure relates to systems and methods for proactive query and content suggestion. Specifically, the systems and methods disclosed herein can utilize change event determination and one or more generation models to generate timely, topical, and / or easy-to-understand language query suggestions and / or content suggestions. Web data (e.g., data describing search query logs, web traffic, web page updates, domain / web page generation, download rates, etc.) can be obtained and processed to determine that one or more topics have changed (e.g., a company has merged, a new comic movie has been announced, a particular player has been traded, etc.). Data describing the change event can be processed using a generation model (e.g., a large language model) to generate a query that can be suggested to a user. The query can be generated to be directed to obtaining search results associated with a particular aspect of the topic that has changed. The generation model can be trained to generate query suggestions formatted in natural language such that a user can easily understand the topic and direction of the query. The query and / or data describing the change event can then be utilized to obtain web resources (e.g., web pages, videos, images, audio clips, articles, books, and / or other data) associated with the change event (e.g., a particular aspect of the changed event). In some implementations, the web resources can be processed using a generation model to generate a change event summary. The change event summary can summarize what was changed as outlined by the web resources. The change event summary can include a topic overview, an indicator of what was changed, and / or an indicator of what the change has impacted. The query, web resources, and / or change event summary can be provided to a user as a query suggestion along with a preview of an answer to a question posed by the query.

[0031] Existing query suggestion techniques can be outdated because the suggestions can simply be a repetition of past queries (e.g., suggesting a previously entered query) or associated with previously viewed content (e.g., suggesting a subtopic associated with a previously viewed web page). Techniques that provide trend-based suggestions can fail to provide customized suggestions, and the suggested queries can result in a large number of results to be reviewed to determine why the topic is popular. For example, not every trend can be relevant to the user, and / or the search result list can not be easily understandable, which can result in the user performing a tedious and time-consuming review of the search result list to understand the reason for the trend.

[0032] The systems and methods disclosed herein can proactively determine updates to topics, which can then be processed to generate natural language query suggestions that can be provided with one or more exemplary results and / or a model-generated summary of the update. Global and / or user-specific trends and changes can be tracked based on query trends and / or based on recursively searching one or more databases. Once a trend and / or change is detected, a generation model can generate a suggested query (e.g., a suggested question), and a plurality of search results for the query can be processed to generate a summary that can be provided with the query suggestion.

[0033] The proactive determination can be performed prior to providing a search input, which can direct the user to one or more specific topic changes without the user having to be previously aware of the changes. Further, the query and content suggestions can be user-customized based on user preferences, determined user interests, user associations, and / or user styles. For example, user data associated with the user can be processed to determine user interests, user groups (e.g., groupings of one or more other users with the user based on social networks, business networks, professions, and / or other attributes), and / or historical interactions. The query and content suggestions can then involve specific attempts to identify change events (and / or query trends) associated with topics relevant to the user based on the determined user interests, user groups, and / or historical interactions. Additionally and / or alternatively, parameters and / or soft cues of the generation model can be trained to generate queries and / or summaries in a vocabulary, tone, and / or style.

[0034] The query and content suggestions can include generating natural language question and answer pairs. For example, the query suggestions can be formatted as understandable language questions, and can be comprehensive. The content summaries can be formatted as answers to the questions posed by the query suggestions. The proactive determination of the question and answer suggestions can be generated by a generation model that can generate natural language question and answer summaries for the determined change events.

[0035] Query suggestions in the form of natural language questions can be proactively performed using tracking trends and / or topic changes, which can be implemented by search engines, news platforms, virtual assistants, and / or other applications and platforms to provide timely and easily understood information to users. The generated suggestions can be provided as query suggestions and / or in the form of a discover feed with a plurality of other generated questions and answers associated with a plurality of other trends and / or changes, such that users are pushed timely and easily understood language queries and content summaries to better understand changes in global topics and / or topics of interest to the user.

[0036] The generation model can process a plurality of queries to generate a query that encompasses a query trend associated with a subset of the plurality of different queries. Alternatively and / or additionally, the generation model can process a query log of a particular user and / or a particular set of users to generate a query that predicts a search interest of the user based on a learning sequence and prediction data of the generation model. The generation model can process headlines, uniform resource locators (URLs), and / or content items of articles or other web resources to determine a change event and generate a suggested query and / or a change event summary.

[0037] The prompt of the generation model can include rewording a logged query to be more comprehensive, interesting, detailed, nuanced, fun, objective, informal or formal, serious or funny, and / or suitable for one or more other preferences. The generation model can determine a query trend and / or a change event, process a plurality of queries and a plurality of web resources to generate a query that includes a direction of a subset of queries associated with the trend and / or change, and supplement the direction of the query with details from the web resources so as to have a detailed query suggestion that specifically identifies a point of interest of the suggested query and can include aspects of what changed and / or why there is a trend.

[0038] The systems and methods of the present disclosure provide a variety of technical effects and benefits. As one example, the systems and methods can provide proactive query and content suggestions. The systems and methods can utilize event determination and generation models to generate timely and easily understood query and content suggestions. In particular, a change associated with one or more topics can be determined. Data describing the change can be processed using a generation model to generate one or more query suggestions. One or more web resources associated with the change event can be identified and obtained using the one or more query suggestions and / or the data describing the change. In some implementations, the one or more web resources can be processed using a generation model to generate a change event summary. The one or more query suggestions and the change event summary can then be provided to a user.

[0039] Another technical benefit of the systems and methods of the present disclosure is the ability to utilize a generative language model to generate natural language question and answer formats. The generative language model can be utilized to generate understandable information in a traditional conversational format. In some implementations, the generative language model and / or one or more soft prompts (e.g., a set of machine learning parameters that can be used by the generative language model to process input) can be trained to mimic the tone, style, and / or vocabulary of a particular user and / or set of users to provide queries and / or summaries in terms, tone, style, and / or dialect traditionally used by the user.

[0040] Another example of technical effects and benefits relates to improved computational efficiency of computing systems and improvements in functionality thereof. For example, the systems and methods disclosed herein can utilize a generative model to provide natural language suggestions and summaries that can reduce the number of web pages accessed by a user in order to gain knowledge about content and / or timing of updates occurring to a topic. Additionally, the systems and methods can reduce the number of searches performed by a user for each inquiry of knowledge. Furthermore, in some implementations, the systems and methods can perform event determination and / or model generation of queries and content generation via a server computing system, which can limit data transmitted and / or received by a user computing system. By reducing web page access, search instances, and system computations at a user computing system, the systems and methods can be utilized to provide targeted and informative information to user computing systems that are resource limited (e.g., limited in computing resources (e.g., smart devices and / or mobile devices) and / or limited in network access (e.g., in poor reception areas)).

[0041] Example embodiments of the present disclosure will now be discussed in further detail with reference to the accompanying drawings.

[0042] Figure 1 A block diagram of an example query suggestion system 10 in accordance with example embodiments of the present disclosure is depicted. In some implementations, the query suggestion system 10 is configured to receive and / or obtain a set of web data 12 describing one or more topics and, as a result of receiving the web data 12, generate, determine, and / or provide one or more suggestions 20 that can include one or more query suggestions and / or change event summaries. Accordingly, in some implementations, the query suggestion system 10 can include a generative model 16 that is operable to process data describing change events to generate query suggestions expressed in understandable language terms and syntax.

[0043] For example, the web data 12 can be obtained from one or more computing systems (e.g., one or more server computing systems). The web data 12 can include web page edits, web page uploads, web traffic data, query usage by a plurality of users, and / or other web data. The web data can be obtained periodically, can be obtained in response to user app usage instances, and / or can be obtained continuously.

[0044] The web data 12 can be processed using the event determination block 14 to determine an occurrence event. The occurrence event can be an update of one or more topics. The occurrence event can be determined based on a query trend, a threshold number of resource updates associated with a topic, an increase in web traffic that exceeds a threshold, and / or based on a threshold number of new content items associated with a topic. The threshold can vary based on time, location, topic, generality of the topic, and / or one or more other contexts. The occurrence event can be associated with an individual, an entity, a group of entities, a location, an object, and / or another topic.

[0045] The data describing the occurrence event can be processed using the generation model 16 to generate one or more query suggestions that involve surfacing search results related to the occurrence event. The generation model 16 can include a machine learning language model trained to generate natural language output. The one or more query suggestions can be formatted as a question. In some implementations, the one or more query suggestions can be in the tone, vocabulary, and / or style of the user by training the generation model (and / or soft prompts) against the user’s conversation.

[0046] One or more web resources 18 can be obtained based on the data describing the occurrence event and / or the one or more query suggestions. The one or more web resources 18 can be top search results and / or trusted content items from a review database. The one or more web resources 18 can be resources that were processed to determine the occurrence event and / or web resources that were determined to evidence the existence of the update and / or details of the occurrence event.

[0047] In some implementations, the one or more web resources 18 can be processed using the generation model 16 to generate an occurrence event summary. The occurrence event summary can describe details of the occurrence event based on information in the one or more web resources 18. The occurrence event summary can be a summary in plain language and can be formatted as an answer to a question posed by the one or more query suggestions.

[0048] The one or more query suggestions and the one or more web resources can be presented to the user as search suggestions 20. Alternatively and / or additionally, the one or more query suggestions and an occurrence event summary can be provided as search suggestions 20. The one or more suggestions 20 can be generated and provided prior to receiving a search input and can be updated based on any input received from the user.

[0049] Figure 2 A block diagram of an example proactive query suggestion system 200 is depicted in accordance with example embodiments of the present disclosure. The proactive query suggestion system 200 is similar to the proactive query suggestion system 100 Figure 1The proactive query suggestion system 200 differs from the query suggestion system 10 in that the proactive query suggestion system 200 further includes an interest determination block 232 and / or a news push interface 228.

[0050] For example, the web data 212 can be obtained from one or more web platforms and / or one or more databases (e.g., one or more domains). The web data 212 can be obtained based on a particular user and / or can be general web data associated with global change tracking. The web data 212 can include web page edits, web page uploads, web traffic data, query usage by a plurality of users, and / or other web data. The web data 212 can be obtained periodically, can be obtained in response to a user app usage instance, and / or can be obtained continuously.

[0051] The web data 212 can be processed using an event determination block 214 to determine an occurring change event 222. Determining an occurring change event can be performed by one or more machine learning models that can be trained to identify topic updates, web anomalies, and / or features that describe a change event 222. The change event 222 can be an update to one or more topics. The one or more topics can be associated with sports, politics, economics, entertainment, and / or other topics. The change event 222 can be determined based on query trends, a threshold number of resource updates associated with a topic, an increase in web traffic that exceeds a threshold, and / or based on a threshold number of new content items associated with a topic. The threshold can vary based on time, location, topic, generality of the topic, and / or one or more other contexts. The change event 222 can be associated with an individual, an entity, a group of entities, a location, an object, and / or another topic.

[0052] In some implementations, the determination of change events 222 can be customized based on a particular user. For example, user data 230 associated with a particular user can be obtained. The user data 230 can include user preferences, user profile data, user relationships, a search history of the user, a browsing history of the user, a purchase history of the user, and / or other interaction data associated with the user. The user data 230 can be processed using an interest determination block 232 to determine a set of topics associated with the user. The set of topics can include topics determined to be of interest to the user, topics associated with a profession of the user, topics of interest to an individual associated with the user, topics associated with an entity of which the user is a part, and / or other topics identified as relevant to the user. The interest determination block 232 can include one or more machine learning models and / or can include one or more deterministic functions and / or thresholds. In some implementations, the change events 222 can be determined based on an output of the interest determination block, which can include adjusting a threshold for one or more topics and / or limiting topics under review for change event determination.

[0053] The data describing the change event 222 can be processed using the generative model 216 (e.g., a generative language model) to generate one or more query suggestions 224 (e.g., one or more example queries for selection) that involve surfacing search results related to the change event 222. The one or more query suggestions 224 can include detailed language that relates to specific aspects of the subject that has changed. The generative model 216 can include a machine-learned language model trained to generate natural language output that can include one or more complete sentences that can be read as conversational language and tone. The one or more query suggestions 224 can be formatted as questions. In some implementations, the one or more query suggestions 224 can be in the tone, vocabulary, and / or style of the user by training the generative model 216 (and / or soft prompts) on the user’s conversations.

[0054] The one or more web resources 218 can be obtained based on the data describing the change event 222 and / or the one or more query suggestions 224. The one or more web resources 218 can include web pages, videos, images, audio files, documents, and / or other content items. The one or more web resources 218 can be top-level search results (e.g., top-level search results for the queries of the query suggestions 224) and / or trusted content items from a review database (e.g., scholarly articles from a scholarly publication database). The one or more web resources 218 can be resources that were processed to determine the change event 222 and / or web resources that were determined to confirm the presence of an update and / or details of the change event 222.

[0055] In some implementations, the one or more web resources 218 can be processed using the generative model 216 to generate a change event summary 226 (e.g., one or more sentences that briefly identify one or more details associated with the subject update). The change event summary 226 can describe details of the change event 222 based on information in the one or more web resources 218. The change event summary 226 can be a summary in plain language and can be formatted as an answer to the question posed by the one or more query suggestions 224.

[0056] The one or more query suggestions 224 and the one or more web resources 218 can be presented to the user as search suggestions 220. Alternatively and / or additionally, the one or more query suggestions 224 and the change event summary 226 can be provided as search suggestions 220. The one or more suggestions 220 can be generated and provided prior to receiving a search input, and can be updated based on any input received from the user.

[0057] In some implementations, a plurality of query suggestions, a plurality of web resources, and / or a plurality of change event summaries can be generated and / or obtained to generate a plurality of query and content suggestions that can be provided in a news feed interface 228. The news feed interface 228 can display each query suggestion 224 with a respective web resource 218 and / or a respective change event summary 226. The displayed content can then be selected to perform a search on the query suggestion 224 to generate a search results interface, redirect to the respective web resource 218, and / or generate an article using the generative model 216 that describes the change event 222 in a longer and more comprehensive format than the change event summary 226.

[0058] In some implementations, the generative model 216 can perform event determination. For example, the generative model 216 can process web data 212 and generate predicted query suggestions 224. The web data 212 can include data describing popular and / or recent articles, social media posts, and / or other web data. The generative model 216 can process titles, URLs, and / or content items of recent and / or popular articles to predict change events 222 (e.g., updates about topics and / or trends). The generative model 216 can then generate query suggestions 224 based on the determined change events 222. In some implementations, the web data 212 can include data describing previously viewed articles, social media profiles, domains, and / or web platforms associated with a user. The generative model 216 can process the data describing previously viewed articles, social media profiles, domains, and / or web platforms associated with a user to generate query suggestions 224 based on recent changes to the previously viewed resources, based on predicted queries based on the previously viewed resources, and / or based on trends determined based on the previously viewed resources. In some implementations, the generative model 216 can perform the generation of predicted query suggestions 224 based on a viewing order of the previously viewed resources.

[0059] Figure 3 A flow diagram depicting an example method performed in accordance with example embodiments of the present disclosure is depicted. Although the method 300 is depicted as a series of acts, it is contemplated that the method 300 can be performed in a different order, or that some acts can be omitted, combined, or reordered. Figure 3 The steps depicted for purposes of illustration and discussion are performed in a particular order, but the methods of the present disclosure are not limited to the particular order or arrangement specifically illustrated. Various steps of the method 300 can be omitted, rearranged, combined, and / or adjusted in various ways without departing from the scope of the present disclosure.

[0060] At 302, the computing system can obtain web data. The web data can describe one or more topics. In some implementations, the web data can be obtained prior to receiving a search query. The web data can be generated by obtaining and processing a plurality of content items. In some implementations, the web data can describe a plurality of queries obtained from a plurality of users. The web data can include data describing changes in web traffic for one or more web pages, platforms, and / or domains.

[0061] At 304, the computing system can process the web data to determine an occurrence of a change event. The change event can include an update associated with one or more topics. The change event can describe an increase in a count of web traffic associated with one or more topics. Alternatively and / or additionally, the change event can describe an increase in a count of global and / or localized search queries associated with one or more topics. The change event can be determined based on satisfying one or more thresholds. The one or more thresholds can vary based on a degree of generality of a respective topic (e.g., the thresholds can be lower for more specific topics). Additionally and / or alternatively, the one or more thresholds can be determined based on a determined degree of interest of a particular user in a respective topic (e.g., a higher determined degree of interest can result in lower one or more thresholds, which can result in a change event being more likely to be determined for a topic determined to be highly interesting to a user). The degree of interest can be determined based on user data associated with the particular user. The user data can be associated with user preferences, user search history, user browsing history, social media accounts of the user, and / or other user-specific data. The one or more topics can be associated with one or more particular entities, one or more particular locations, one or more particular products, one or more particular individuals, and / or one or more other topics.

[0062] At 306, the computing system can process the data describing the change event using a generative model to generate one or more query suggestions. The generative model can include a large language model, a visual language model, an image-to-text model, and / or an image generation model. In some implementations, the data describing the change event can be generated (and / or processed) using one or more semantic understanding models, one or more detection models, one or more classification models, one or more segmentation models, one or more embedding models, and / or one or more augmentation models. The data describing the change event can describe a topic classification and / or an identification of what has changed.

[0063] At 308, the computing system can obtain one or more web resources associated with the change event. The one or more web resources can be obtained by processing the one or more query suggestions using a search engine to determine one or more web resources responsive to one or more queries associated with the one or more query suggestions. Alternatively and / or additionally, the one or more web resources can be obtained based on a determined topic associated with the change event.

[0064] At 310, the computing system can process the one or more web resources using a generative model to generate a change event summary. The change event summary can include an updated natural language summary based on processing the one or more web resources. The update can be determined based on topic classification, semantic understanding, and / or other natural language processing techniques. In some implementations, the change event summary can include a model-generated summary provided with one or more references associated with the one or more web resources.

[0065] At 312, the computing system can provide the one or more query suggestions and the change event summary for display. The one or more query suggestions and the change event summary can be provided for display prior to receiving a search query. The one or more query suggestions and the change event summary can be provided for display in a search query suggestion interface that provides the pair in a panel adjacent to a search query input box.

[0066] In some implementations, the computing system can obtain user data associated with a user and determine that the user is interested in one or more particular topics based on the user data. Processing data describing a change event using a generative model to generate one or more query suggestions can be performed in response to determining that the change event is associated with the one or more particular topics.

[0067] In some implementations, the computing system can determine a second change event that occurred. The second change event can include updates associated with one or more second topics. The computing system can process data describing the second change event using a generative model to generate one or more second query suggestions, obtain one or more second web resources associated with the second change event, and process the one or more second web resources using a generative model to generate a second change event summary. In some implementations, the computing system can provide the one or more second query suggestions and the second change event summary for display. The one or more query suggestions and the change event summary can be provided as a first pair. The one or more second query suggestions and the second change event summary can be provided as a second pair. The first pair and the second pair can be provided in a news feed interface.

[0068] In some implementations, the computing system can receive a selection of at least one of the one or more query suggestions or the change event summary and provide a search results interface for display. The search results interface can include a plurality of search results responsive to a search query associated with the one or more query suggestions. The plurality of search results can include one or more web resources.

[0069] Alternatively and / or additionally, the computing system can receive a selection of at least one of the one or more query suggestions or the change event summary and provide a model-generated content item for display. The model-generated content item can be generated using a generative model. In some implementations, the model-generated content item can include information associated with the change event. The information can include details obtained from one or more databases formatted as a natural language article by the generative model.

[0070] Figure 4 A block diagram of an example query trend determination system 400 is depicted in accordance with example embodiments of the present disclosure. Specifically, the query trend determination system 400 can perform proactive query and content suggestion based on processing a plurality of queries to determine query trends, which can then result in a generative model 416 generating one or more query suggestions that can be provided with one or more web resources 418 and / or a content summary.

[0071] For example, a plurality of queries 412 can be obtained. The plurality of queries can be associated with a plurality of different users. The plurality of queries 412 can be recent queries within the past hour, day, week, and / or month. The plurality of different users can be global users, users local to a particular user, and / or other users determined to be associated with a particular user based on social media connections, interests, and / or other similarities.

[0072] The plurality of queries 412 can be processed using a trend determination block 414 to determine one or more query trends. The one or more query trends can be determined based on an inflow of queries related to a particular topic and / or a particular group of topics. The determination can be based on one or more thresholds and / or based on one or more machine learning models.

[0073] Data describing the one or more query trends can be processed using a generative model 416 to generate one or more query suggestions associated with a topic of the query trend. The generative model 416 can include a language model trained to generate natural language queries. The one or more query suggestions can be formatted as a question, and can relate to a particular aspect of a topic that is a focal feature of the query trend.

[0074] Additionally or alternatively, one or more web resources 418 can be obtained based on data describing query trends and / or one or more query suggestions. The one or more web resources 418 can be associated with content items selected and / or viewed by a plurality of users when entering queries that trend. Alternatively and / or additionally, the one or more web resources 418 can be search results obtained proactively for query suggestions. In some implementations, the one or more web resources 418 can be obtained based on determined query trends and then processed using the generative model 416 to generate one or more query suggestions.

[0075] The one or more web resources 418 can be processed using the generative model 416 to generate a content summary describing information described in the one or more web resources 418. The content summary can include a natural language summary describing one or more shared details of content items responsive to the query suggestion. The content summary can be formatted as a conversation responsive to a prompt of the query suggestion.

[0076] The one or more query suggestions and the content summary can be provided for display to the particular user as search what suggestions 420. Alternatively and / or additionally, the one or more query suggestions and the one or more web resources can be provided for display to the particular user as search what suggestions 420.

[0077] In some implementations, the generative model 416 (e.g., a large language model) can process the plurality of queries 412 to generate one or more query suggestions. For example, the generative model 416 can determine query trends (e.g., an influx of trends associated with a particular topic) and can generate query suggestions based on the determined query trends. Query suggestions can be generated that are more detailed, nuanced, and / or more intensive than queries in the plurality of queries. In some implementations, query suggestions can be generated to mimic user vocabulary, user style, user tone, and / or user query length. Query suggestions can be generated to satisfy one or more query preferences (e.g., length preferences, tone preferences, style preferences, etc.).

[0078] In some implementations, the plurality of queries 412 can include a plurality of previously used queries of a particular user. The generative model 416 can process the plurality of queries 412 to generate interesting, nuanced, fun, serious, objective, and / or other types of query suggestions. In particular, the predictive capabilities of the generative model can be used to predict queries that can be of interest to the particular user and can generate interesting and nuanced query suggestions based on the prediction.

[0079] In some implementations, the generation model 416 can identify one or more representative queries of a topic that can describe a query trend in the plurality of queries. The generation model 416 can then reword the one or more representative queries to make them more detailed and more indicative of the topic of the trend. Alternatively and / or additionally, the one or more representative queries can be reworded to personalize the query suggestions for a particular user.

[0080] Figure 5 A block diagram of an example query newsfeed system 500 is depicted in accordance with example embodiments of the present disclosure. In particular, the query newsfeed system 500 can identify query suggestions 504, web resources 506, and / or change event summaries 508 for each change event 502, which can then be provided in a newsfeed format with other query suggestions.

[0081] For example, a change event 502 associated with one or more topics can be determined. The change event 502 can be associated with one or more changes to one or more topics. Data describing the change event 502 can be processed to generate query suggestions 504. The query suggestions 504 can be processed using a search engine to determine one or more web resources 506 associated with the query suggestions 504. The one or more web resources 506 can include content items that include information about aspects of the one or more topics that have changed.

[0082] The one or more web resources 506 can be processed to generate change event summaries 508. The change event summaries 508 can be based on one content item and / or multiple content items. In some implementations, multiple content items associated with the change event can be obtained and processed. Overlaps between the multiple content items can be determined, and a change event summary can be generated based on the determined overlaps and / or determined shared features.

[0083] The change event summaries 508 and / or the one or more web resources 506 can be associated with the query suggestions 504 to generate query content pairs 510. The query content pairs 510 can be processed to generate suggestion interface elements 512, which can include the query suggestions 504 and the change event summaries 508 in a conversational format (e.g., two individuals talking (e.g., question and answer)). The suggestion interface elements 512 can be selected to perform one or more operations.

[0084] For example, the suggestion interface element 512 can be selected to search 514 the query suggestion 504, generate a model-generated article 516, and / or navigate to a resource 518. The search 514 can be performed to obtain and display a plurality of search results for the query by processing the query suggestion 504 using a search engine. The model-generated article 516 can include processing a plurality of web resources associated with the change event 502 using a generative model to generate an article describing the change event 502 with details about what changed, an overview of the topic, and / or potential impacts of the change. In some implementations, the suggestion interface element 512 can be selected to navigate to one or more web resources 506 and / or provide the one or more web resources 506 in a viewing window.

[0085] Figure 6 A diagram depicting an example newsfeed interface 600 in accordance with example embodiments of the present disclosure is illustrated. Specifically, the newsfeed interface 600 can include a query input box 602, a plurality of query suggestions (e.g., 604, 608, and 612), and a plurality of corresponding change event summaries (e.g., 606, 610, and 614). The query input box 602 can be configured to receive a search input (e.g., a textual query, an image query, an audio query, and / or a multi-modal query). The plurality of query suggestions (e.g., 604, 608, and 612) and the plurality of corresponding change event summaries (e.g., 606, 610, and 614) can be provided in a newsfeed format that displays the queries and summaries in pairs in a question-and-answer format. The plurality of query suggestions (e.g., 604, 608, and 612) and the plurality of corresponding change event summaries (e.g., 606, 610, and 614) can be generated in a tone, dialect, style, grammar, and / or terminology that is offered to the user. The plurality of query suggestions (e.g., 604, 608, and 612) and the plurality of corresponding change event summaries (e.g., 606, 610, and 614) can be selected to perform a search, obtain additional data, and / or generate a content item.

[0086] Figure 7 A flow diagram depicting an example method for performance in accordance with example embodiments of the present disclosure is illustrated. Although the method 700 is described below with reference to the components of the example systems illustrated in FIGS. 1-6, the method 700 can be implemented utilizing any suitable system. Figure 7 The steps depicted for performance in a particular order for purposes of illustration and discussion, the methods of the present disclosure are not limited to the order or arrangement specifically illustrated. Various steps of the method 700 can be omitted, rearranged, combined, and / or adjusted in various ways without departing from the scope of the present disclosure.

[0087] At 702, the computing system can obtain a plurality of queries. The plurality of queries can be obtained from a plurality of users. The plurality of users can be global users and / or can be from a particular region. In some implementations, the computing system can determine that the plurality of users are associated with a particular user for which the computing system is currently determining suggestions. For example, the plurality of users can be determined that have one or more shared attributes (e.g., shared interests, shared connections, shared schedules, shared activities, etc.) with the particular user. Search query data associated with each of the plurality of users can be obtained and processed on a regular basis.

[0088] At 704, the computing system can process the plurality of queries to determine a query trend. The query trend can describe an increase in the number of queries associated with a particular topic. In some implementations, the query trend can be a global query trend. Alternatively and / or additionally, the query trend can be a regional trend, a localized trend, an age group based trend, a profession based trend, and / or another sub-group trend.

[0089] At 706, the computing system can process data describing the query trend using a language model to generate one or more query suggestions. The one or more query suggestions can be generated in a question format. For example, the computing system can determine a query trend associated with potential trade assets for a rumored trade between two football teams, and the language model can process data describing the query trend to generate a query suggestion that includes "Who are the potential trade assets for the rumored trade between football team A and football team B?"

[0090] In some implementations, processing data describing the query trend using a language model to generate one or more query suggestions can be performed in response to determining that the particular topic is associated with one or more content items previously viewed by the particular user.

[0091] At 708, the computing system can obtain one or more web resources in response to the one or more query suggestions. The one or more web resources can be obtained by processing the one or more query suggestions using a search engine. In some implementations, the one or more web resources can be one or more top search results for a given query associated with the query suggestion. Alternatively and / or additionally, the one or more web resources can include one or more content items obtained from one or more databases. The one or more databases can include one or more curated databases determined to include trusted and authentic information. The one or more databases can be general knowledge databases and / or one or more specialized databases.

[0092] At 710, the computing system can process the one or more web resources using a language model to generate a content summary. The content summary can describe content of the one or more web resources. In some implementations, the content summary can be generated in an answer format. The content summary can be generated as a question suggested in response to the one or more queries. Additionally or alternatively, the content summary can include a natural language summary including one or more details obtained from the one or more web resources.

[0093] At 712, the computing system can provide the one or more query suggestions and the content summary for display. The one or more query suggestions and the content summary can be provided in a question and answer format. In some implementations, additional query suggestions and additional content summaries can be provided adjacent to the one or more query suggestions and the one or more content summaries. The one or more query suggestions can be selected to navigate to a search results interface. Additionally and / or alternatively, the content summary can be selected to obtain a long format model generated article including additional details regarding a particular topic associated with the query trend.

[0094] Figure 8 A flow diagram depicting an example method performed in accordance with example embodiments of the present disclosure is depicted. Although the method 800 is depicted as a series of steps, it is understood that the method 800 can include additional steps, omit some of the steps, and / or combine two or more of the steps. Figure 8 The steps depicted for purposes of illustration and discussion are performed in a particular order, but the methods of the present disclosure are not limited to the particular order or arrangement specifically illustrated. Various steps of the method 800 can be omitted, rearranged, combined, and / or adjusted in various ways without departing from the scope of the present disclosure.

[0095] At 802, the computing system can obtain web data. The web data can describe content provided by a collection of web resources. The web data can include data from one or more particular databases and / or can be obtained from a plurality of databases. The web data can be generated by crawling a plurality of web pages.

[0096] At 804, the computing system can process the web data to determine a plurality of change events that occurred. The plurality of change events can include a plurality of updates associated with a plurality of topics. Each change event of the plurality of change events can be determined based on determining an inflow of articles associated with a respective topic. Alternatively and / or additionally, each change event of the plurality of change events can be determined based on determining an inflow of social media posts associated with a respective topic.

[0097] At 806, the computing system can process the data describing the plurality of change events using a language model to generate a plurality of query suggestions. The language model can include a generative model. Each query suggestion can be associated with a respective change event of the plurality of change events. The language model can include a machine-learned generative model trained to generate natural language data that is tailored to mimic a vocabulary, tone, and / or style of a particular user.

[0098] At 808, the computing system can obtain a plurality of respective web resources associated with the plurality of change events. The plurality of respective web resources can include one or more respective web resources for each of the plurality of change events. The plurality of respective web resources can be obtained by processing the plurality of query suggestions using a search engine. Alternatively and / or additionally, the plurality of respective web resources can be obtained by identifying content items associated with respective change events based on data describing the respective change events and / or web data.

[0099] At 810, the computing system can associate each of the plurality of query suggestions with a respective web resource of the plurality of respective web resources to generate a query suggestion web resource pair for each of the plurality of change events. Associating the plurality of query suggestions with respective web resources of the plurality of respective web resources to generate a query suggestion web resource pair for each of the plurality of change events can include generating a graphical user interface element for each of the query suggestion web resource pairs.

[0100] At 812, the computing system can provide a newsfeed interface for display. The newsfeed interface can include a set of query suggestion web resource pairs associated with at least a subset of the plurality of change events. The newsfeed interface can include one or more graphical user interface elements for each of the query suggestion web resource pairs. The graphical user interface elements can include a selectable query element, a selectable summary element, and / or a selectable web resource element. The selectable query element can be selected to navigate to a search results interface displaying search results for the respective query suggestion. The selectable summary element can be selected to generate and / or open a summary interface providing a change event summary associated with the respective change event. The selectable resource element can be selected to open a web page viewing interface displaying the respective web resource.

[0101] Figure 9 An illustration of an example query suggestion interface 900 is depicted in accordance with example embodiments of the present disclosure. Specifically, Figure 9 An example query recommendation interface 910, an example generated query anchor interface 920, and an example card-inspired journey interface 930 are depicted.

[0102] The query recommendation interface 910 can include one or more generated query recommendations 912 that are displayed with discovery push cards 914 that can be associated with the respective query recommendations 912. For example, the discovery push cards 914 can include graphical cards that describe web resources associated with the query recommendations 912. Alternatively and / or additionally, the discovery push cards 914 can include model-generated summaries that are generated based on processing one or more web resources in response to a query. In some implementations, the query recommendations 912 and discovery push cards 914 can be provided with one or more additional graphical cards 916 that are associated with one or more additional web resources that are associated with the query recommendations 912. The query recommendation interface 910 can include multiple pairs of query recommendations 912 and discovery push cards 914.

[0103] The generated query anchor interface 920 can provide model-generated query suggestions 922 for display with discovery push cards 924 that are associated with the model-generated query suggestions. In some implementations, one or more web resources can be determined to be relevant to a user and / or a set of users. A generation model can then process the one or more web resources to generate the model-generated query suggestions 922. Additionally, the discovery push cards 924 can be generated to describe the one or more web resources and can include a title, a URL, and / or one or more content items from the one or more web resources. The query recommendation interface 910 can include multiple pairs of model-generated query suggestions 922 and discovery push cards 924.

[0104] The card-inspired journey interface 930 can include a discovery push card 932 that is provided for display with one or more action elements 934. For example, the discovery push card 932 can be associated with a query suggestion and / or can be generated based on a web resource that is identified without an initial query suggestion. The web resource associated with the discovery push card 932 can be processed to determine a topic, a content item type, and / or one or more other attributes of the web resource that are associated with the web resource. Based on the topic, the content type, and / or the other attributes, one or more candidate actions can be determined to be associated with the web resource. The candidate actions can include a follow-up query, a question-and-answer knowledge retrieval, an application redirection, a shopping, a subscription, a mapping, etc. For example, the web resource associated with the discovery push card 932 can be processed to generate a query that involves obtaining other web resources about a topic associated with the web resource and / or obtaining additional information associated with a subtopic mentioned in the web resource. In some implementations, the web resource associated with the discovery push card 932 can be processed using a generation model to generate a model-generated question-and-answer set that describes information that can be learned from the web resource.

[0105] The candidate action can then be utilized to generate one or more action elements 934. For example, a query suggestion to obtain additional information can be provided with the discovery push card 932, and can be selected to perform a search. Model-generated question and answers can be provided in a pull-down format, such that the question is displayed first, and the answer can be viewed by selecting the associated question. Each action element 934 can be selected to perform the respective action (e.g., search, application redirection, model inference, information display, etc.).

[0106] Figure 10A A block diagram of an example computing system 100 that performs query and content suggestions in accordance with example embodiments of the disclosure is depicted. The system 100 includes a user computing system 102, a server computing system 130, and / or a third-party computing system 150 that are communicatively coupled over a network 180.

[0107] The user computing system 102 can include any type of computing device, such as, for example, a personal computing device (e.g., a laptop or desktop computer), a mobile computing device (e.g., a smartphone or tablet computer), a gaming console or controller, a wearable computing device, an embedded computing device, or any other type of computing device.

[0108] The user computing system 102 includes one or more processors 112 and a memory 114. The one or more processors 112 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, a FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. The memory 114 can include one or more non-transitory computer-readable storage media, such as

[0109] In some implementations, the user computing system 102 can store or include one or more machine learning models 120. For example, the machine learning models 120 can be or can otherwise include various machine learning models, such as neural networks (e.g., deep neural networks) or other types of machine learning models, including nonlinear models and / or linear models. The neural networks can include feed-forward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks, or other forms of neural networks. The machine learning models 120 can include one or more generative models (e.g., large language models, text-to-image models, image plus caption models, etc.). The one or more generative models can include one or more transformer models, and can include autoregressive language models and / or image diffusion models.

[0110] In some implementations, the one or more machine learning models 120 can be received from the server computing system 130 over the network 180, stored in the user computing device memory 114, and then used or otherwise implemented by the one or more processors 112. In some implementations, the user computing system 102 can implement multiple parallel instances of a single machine learning model 120 (e.g., to perform parallel machine learning model processing across multiple instances of input data and / or detected features).

[0111] More specifically, the one or more machine learning models 120 can include one or more detection models, one or more classification models, one or more segmentation models, one or more enhancement models, one or more generative models, one or more natural language processing models, one or more optical character recognition models, and / or one or more other machine learning models. The one or more machine learning models 120 can include one or more transformer models. The one or more machine learning models 120 can include one or more neural radiance fields models, one or more diffusion models, and / or one or more autoregressive language models.

[0112] The one or more machine learning models 120 can be used to detect one or more object features. The detected object features can be classified and / or embedded. The classifications and / or embeddings can then be utilized to perform a search to determine one or more search results. Alternatively and / or additionally, the one or more detected features can be utilized to determine that an indicator (e.g., a user interface element indicating the detected feature) is to be provided to indicate that a feature has been detected. The user can then select the indicator to perform the feature classification, embedding, and / or search. In some implementations, the classification, embedding, and / or search can be performed prior to selection of the indicator.

[0113] In some implementations, the one or more machine learning models 120 can process image data, text data, audio data, and / or latent encoded data to generate output data that can include image data, text data, audio data, and / or latent encoded data. The one or more machine learning models 120 can perform optical character recognition, natural language processing, image classification, object classification, text classification, audio classification, context determination, action prediction, image correction, image enhancement, text enhancement, sentiment analysis, object detection, error detection, repair, video stabilization, audio correction, audio enhancement, and / or data segmentation (e.g., mask-based segmentation).

[0114] Additionally or alternatively, one or more machine-learned models 140 can be included in or otherwise stored and implemented by a server computing system 130 that communicates with the user computing system 102 according to a client-server relationship. For example, the machine-learned models 140 can be implemented by the server computing system 130 as part of a web service (e.g., a viewfinder service, a visual search service, an image processing service, an environment computing service, and / or an overlay application service). Thus, one or more models 120 can be stored and implemented at the user computing system 102, and / or one or more models 140 can be stored and implemented at the server computing system 130.

[0115] In some implementations, the computing system 100 can utilize one or more soft prompts to adjust one or more machine-learned models (120 and / or 140) for downstream tasks. The one or more soft prompts can include a set of tunable parameters that can be trained (or tuned) when the parameters of the one or more machine-learned models (120 and / or 140) are fixed. The one or more soft prompts can be trained for a particular task and / or a particular set of tasks. Alternatively and / or additionally, the one or more soft prompts can be trained to adjust the one or more machine-learned models (120 and / or 140) to perform inferences for a particular individual and / or one or more entities such that the output is customized for the particular individual and / or the particular entities. The one or more soft prompts can be obtained and processed by the one or more machine-learned models (120 and / or 140) using one or more inputs.

[0116] The user computing system 102 can also include one or more user input components 122 that receive user input. For example, the user input components 122 can be touch-sensitive components (e.g., a touch-sensitive display screen or a touchpad) that are sensitive to touch by a user input object (e.g., a finger or a stylus). The touch-sensitive components can be used to implement a virtual keyboard. Other example user input components include a microphone, a conventional keyboard, or other mechanisms by which a user can provide user input.

[0117] In some implementations, the user computing system can store and / or provide one or more user interfaces 124 that can be associated with one or more applications. The one or more user interfaces 124 can be configured to receive input and / or provide data for display (e.g., image data, textual data, audio data, one or more user interface elements, an augmented reality experience, a virtual reality experience, and / or other data for display). The user interfaces 124 can be associated with one or more other computing systems (e.g., the server computing system 130 and / or the third-party computing system 150). The user interfaces 124 can include a viewfinder interface, a search interface, a generative model interface, a social media interface, and / or a media content gallery interface.

[0118] The user computing system 102 can include and / or receive data from one or more sensors 126. The one or more sensors 126 can be housed in a housing assembly that houses the one or more processors 112, the memory 114, and / or one or more hardware components that can store and / or cause execution of one or more software packages. The one or more sensors 126 can include one or more image sensors (e.g., cameras), one or more lidar sensors, one or more audio sensors (e.g., microphones), one or more inertial sensors (e.g., inertial measurement units), one or more biological sensors (e.g., heart rate sensors, pulse sensors, retinal sensors, and / or fingerprint sensors), one or more infrared sensors, one or more location sensors (e.g., GPS), one or more touch sensors (e.g., conductive touch sensors and / or mechanical touch sensors), and / or one or more other sensors. The one or more sensors can be utilized to obtain data associated with a user’s environment (e.g., images of a user’s environment, recordings of an environment, and / or a user’s location).

[0119] The user computing system 102 can include and / or be part of a user computing device 104. The user computing device 104 can include a mobile computing device (e.g., a smartphone or a tablet computer), a desktop computer, a laptop computer, a smart wearable, and / or a smart appliance. Additionally and / or alternatively, the user computing system can obtain data from and / or generate data using one or more user computing devices 104. For example, a camera of a smartphone can be utilized to capture image data that describes an environment, and / or an overlay application of a user computing device 104 can be utilized to track and / or process data provided to a user. Similarly, one or more sensors associated with a smart wearable can be utilized to obtain data about a user and / or about a user’s environment (e.g., image data can be obtained using a camera housed in a user’s smart glasses). Additionally and / or alternatively, data can be obtained and uploaded from other user devices that can be dedicated to data obtaining or generation.

[0120] The server computing system 130 includes one or more processors 132 and memory 134. The one or more processors 132 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, a FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. The memory 134 can include one or more non-transitory computer-readable storage media, such as

[0121] In some implementations, the server computing system 130 includes or is otherwise implemented by one or more server computing devices. In instances in which the server computing system 130 includes multiple server computing devices, such server computing devices can operate according to a sequential computing architecture, a parallel computing architecture, or some combination thereof.

[0122] As noted above, the server computing system 130 can store or otherwise include one or more machine-learned models 140. For example, the models 140 can be or otherwise include various machine-learned models. Example machine-learned models include neural networks or other multi-layer non-linear models. Example neural networks include feed-forward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks. See, e.g., Goodfellow, et al., Deep Learning, MIT Press, 2016, which is incorporated herein by reference in its entirety. Figure 10B Example models 140 are discussed.

[0123] Additionally and / or alternatively, the server computing system 130 can include and / or be communicatively connected with a search engine 142, which can be used to crawl one or more databases (and / or resources). The search engine 142 can process data from the user computing system 102, the server computing system 130, and / or the third-party computing system 150 to determine one or more search results associated with input data. The search engine 142 can perform term-based searching, tag-based searching, Boolean-based searching, image searching, embedding-based searching (e.g., nearest neighbor searching), multi-modal searching, and / or one or more other searching techniques.

[0124] The server computing system 130 can store and / or provide one or more user interfaces 144 for obtaining input data and / or providing output data to one or more users. The one or more user interfaces 144 can include one or more user interface elements, which can include input fields, navigational tools, content tiles, selectable tiles, widgets, data display carousels, dynamic animations, informational popups, image enhancements, text-to-speech, speech-to-text, augmented reality, virtual reality, feedback loops, and / or other interface elements.

[0125] The user computing system 102 and / or the server computing system 130 can train the models 120 and / or 140 via interaction with a third-party computing system 150 communicatively coupled by the network 180. The third-party computing system 150 can be separate from the server computing system 130 or be a part of the server computing system 130. Alternatively and / or additionally, the third-party computing system 150 can be associated with one or more web resources, one or more web platforms, one or more other users, and / or one or more contexts.

[0126] The third-party computing system 150 can include one or more processors 152 and memory 154. The one or more processors 152 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, a FPGA, a controller, a microcontroller, etc.) and can be one processor or a plurality of processors that are operatively connected. The memory 154 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc. and combinations thereof. The memory 154 can store data 156 and instructions 158 that are executed by the processor 152 to cause the third-party computing system 150 to perform operations. In some implementations, the third-party computing system 150 includes or is otherwise implemented by one or more server computing devices.

[0127] The network 180 can be any type of communications network, such as a local area network (e.g., an intranet), a wide area network (e.g., the Internet), or some combination thereof, and can include any number of wired or wireless links. In general, communications over the network 180 can be carried out using a variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or protection schemes (e.g., VPN, secure HTTP, SSL), via any type of wired and / or wireless connection.

[0128] The server computing system 130 (and / or the user computing system 102, via one or more entered preferences) can prompt the generation model for particular query and content recommendation tasks (e.g., via few shot prompting, zero shot prompting, and / or via soft prompting). The prompt can include specifying a set amount of query, article data, post data, and / or historical data to obtain and process to perform model inference. In some implementations, the prompt can include specifying “can you recommend some new questions that the user will find interesting?” The prompt can include requesting the generation model to generate “why.” The prompt can include requesting obtaining and processing of user data to determine and / or query generation specific to the user.

[0129] The machine learning models described in this specification can be used for various tasks, applications, and / or use cases.

[0130] In some implementations, an input to a machine learning model of the present disclosure can be image data. The machine learning model can process the image data to generate an output. As an example, the machine learning model can process the image data to generate an image recognition output (e.g., a recognition of the image data, a latent embedding of the image data, an encoded representation of the image data, a hash of the image data, etc.). As another example, the machine learning model can process the image data to generate an image segmentation output. As another example, the machine learning model can process the image data to generate an image classification output. As another example, the machine learning model can process the image data to generate an image data modification output (e.g., an alteration of the image data, etc.). As another example, the machine learning model can process the image data to generate an encoded image data output (e.g., an encoded and / or compressed representation of the image data, etc.). As another example, the machine learning model can process the image data to generate an upscaled image data output. As another example, the machine learning model can process the image data to generate a prediction output.

[0131] In some implementations, input to a machine learning model of the present disclosure can be text or natural language data. The machine learning model can process the text or natural language data to generate an output. As an example, the machine learning model can process natural language data to generate a language encoding output. As another example, the machine learning model can process the text or natural language data to generate a latent text embedding output. As another example, the machine learning model can process the text or natural language data to generate a translation output. As another example, the machine learning model can process the text or natural language data to generate a classification output. As another example, the machine learning model can process the text or natural language data to generate a text segmentation output. As another example, the machine learning model can process the text or natural language data to generate a semantic intent output. As another example, the machine learning model can process the text or natural language data to generate augmented text or natural language output (e.g., text or natural language data of higher quality than the input text or natural language, etc.). As another example, the machine learning model can process the text or natural language data to generate a prediction output.

[0132] In some implementations, input to a machine learning model of the present disclosure can be speech data. The machine learning model can process the speech data to generate an output. As an example, the machine learning model can process the speech data to generate a speech recognition output. As another example, the machine learning model can process the speech data to generate a speech translation output. As another example, the machine learning model can process the speech data to generate a latent embedding output. As another example, the machine learning model can process the speech data to generate an encoded speech output (e.g., an encoded and / or compressed representation of the speech data, etc.). As another example, the machine learning model can process the speech data to generate augmented speech output (e.g., speech data of higher quality than the input speech data, etc.). As another example, the machine learning model can process the speech data to generate a text representation output (e.g., a text representation of the input speech data, etc.). As another example, the machine learning model can process the speech data to generate a prediction output.

[0133] In some implementations, an input to a machine learning model of the present disclosure can be sensor data. The machine learning model can process the sensor data to generate an output. As an example, the machine learning model can process the sensor data to generate a recognition output. As another example, the machine learning model can process the sensor data to generate a prediction output. As another example, the machine learning model can process the sensor data to generate a classification output. As another example, the machine learning model can process the sensor data to generate a segmentation output. As another example, the machine learning model can process the sensor data to generate a segmentation output. As another example, the machine learning model can process the sensor data to generate a visualization output. As another example, the machine learning model can process the sensor data to generate a diagnosis output. As another example, the machine learning model can process the sensor data to generate a detection output.

[0134] In some cases, the input includes visual data, and the task is a computer vision task. In some cases, the input includes pixel data for one or more images, and the task is an image processing task. For example, the image processing task can be image classification, in which the output is a set of scores, each score corresponding to a different object class and representing a likelihood that the one or more images depict an object belonging to the object class. The image processing task can be object detection, in which the image processing output identifies one or more regions in the one or more images, and for each region, identifies a likelihood that the region depicts an object of interest. As another example, the image processing task can be image segmentation, in which the image processing output defines, for each pixel in the one or more images, a respective likelihood for each of a predetermined set of classes. For example, the set of classes can be foreground and background. As another example, the set of classes can be object classes. As another example, the image processing task can be depth estimation, in which the image processing output defines, for each pixel in the one or more images, a respective depth value. As another example, the image processing task can be motion estimation, in which the network input includes multiple images, and the image processing output defines, for each pixel of one of the input images, a motion of a scene depicted at the pixel between the images in the network input.

[0135] A user computing system can include multiple applications (e.g., applications 1 through N). Each application can include its own respective machine learning library and machine learning models. For example, each application can include a machine learning model. Example applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, and the like.

[0136] Each application can communicate with a number of other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, each application can communicate with each device component using an API (e.g., a public API). In some implementations, the API used by each application is specific to that application.

[0137] The user computing system 102 can include a number of applications (e.g., applications 1 through N). Each application communicates with a central intelligence layer. Example applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, and the like. In some implementations, each application can use an API (e.g., a public API across all applications) to communicate with the central intelligence layer (and the models stored therein).

[0138] The central intelligence layer can include a number of machine learning models. For example, a respective machine learning model (e.g., model) can be provided for each application, and the machine learning model is managed by the central intelligence layer. In other implementations, two or more applications can share a single machine learning model. For example, in some implementations, the central intelligence layer can provide a single model (e.g., a single model) for all applications. In some implementations, the central intelligence layer is included within or otherwise implemented by the operating system of the computing system 100.

[0139] The central intelligence layer can communicate with a central device data layer. The central device data layer can be a centralized data repository for the computing system 100. The central device data layer can communicate with a number of other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, the central device data layer can communicate with each device component using an API (e.g., a private API).

[0140] Figure 10BA block diagram of an example computing system 50 that performs query and content suggestions in accordance with example embodiments of the present disclosure is depicted. Specifically, the example computing system 50 can include one or more computing devices 52 that can be used to obtain and / or generate one or more data sets that can be processed by a sensor processing system 60 and / or an output determination system 80 to feedback to a user so that information about features in the one or more obtained data sets can be provided. The one or more data sets can include image data, textual data, audio data, multi-modal data, latent encoded data, and the like. The one or more data sets can be obtained via one or more sensors associated with the one or more computing devices 52 (e.g., one or more sensors in the computing devices 52). Additionally and / or alternatively, the one or more data sets can be stored data and / or retrieved data (e.g., data retrieved from a web resource). For example, a user can interact with an image, text, and / or other content item. The interaction with the content item can then be utilized to generate one or more determinations.

[0141] The one or more computing devices 52 can obtain and / or generate the one or more data sets based on image capture, sensor tracking, data store retrieval, content download (e.g., download of an image or other content item from a web resource via the internet), and / or via one or more other techniques. The one or more data sets can be processed using the sensor processing system 60. The sensor processing system 60 can use one or more machine learning models, one or more search engines, and / or one or more other processing techniques to perform one or more processing techniques. The one or more processing techniques can be performed in any combination and / or individually. The one or more processing techniques can be performed serially and / or in parallel. Specifically, the one or more data sets can be processed using a context determination block 62 that can determine a context associated with the one or more content items. The context determination block 62 can identify and / or process metadata, user profile data (e.g., preferences, user search history, user browsing history, user purchase history, and / or user input data), previous interaction data, global trend data, location data, time data, and / or other data to determine a particular context associated with the user. The context can be associated with an event, a determined trend, a particular action, a particular type of data, a particular environment, and / or another context associated with the user and / or the retrieved or obtained data.

[0142] The sensor processing system 60 can include an image pre-processing block 64. The image pre-processing block 64 can be used to adjust one or more values of an obtained and / or received image in preparation for processing of the image by one or more machine learning models and / or one or more search engines 74. The image pre-processing block 64 can adjust image size, adjust saturation values, adjust resolution, strip and / or add metadata, and / or perform one or more other operations.

[0143] In some implementations, the sensor processing system 60 can include one or more machine learning models that can include a detection model 66, a segmentation model 68, a classification model 70, an embedding model 72, and / or one or more other machine learning models. For example, the sensor processing system 60 can include one or more detection models 66 that can be used to detect particular features in a processed dataset. Specifically, one or more images can be processed using one or more detection models 66 to generate one or more bounding boxes associated with detected features in the one or more images.

[0144] Additionally and / or alternatively, one or more segmentation models 68 can be utilized to segment one or more portions of a dataset from one or more datasets. For example, one or more segmentation models 68 can utilize one or more segmentation masks (e.g., one or more segmentation masks generated manually and / or based on one or more bounding boxes) to segment a portion of an image, a portion of an audio file, and / or a portion of text. Segmentation can include isolating one or more detected objects and / or removing one or more detected objects from an image.

[0145] One or more classification models 70 can be used to process image data, text data, audio data, latent encoded data, multi-modal data, and / or other data to generate one or more classifications. The one or more classification models 70 can include one or more image classification models, one or more object classification models, one or more text classification models, one or more audio classification models, and / or one or more other classification models. The one or more classification models 70 can process data to determine one or more classifications.

[0146] In some implementations, one or more embedding models 72 can be used to process data to generate one or more embeddings. For example, one or more embedding models 72 can be used to process one or more images to generate one or more image embeddings in an embedding space. The one or more image embeddings can be associated with one or more image features of the one or more images. In some implementations, the one or more embedding models 72 can be configured to process multi-modal data to generate multi-modal embeddings. The one or more embeddings can be used for classification, search, and / or learning embedding space distributions.

[0147] The sensor processing system 60 can include one or more search engines 74 that can be used to perform one or more searches. The one or more search engines 74 can crawl one or more databases (e.g., one or more local databases, one or more global databases, one or more private databases, one or more public databases, one or more specialized databases, and / or one or more general-purpose databases) to determine one or more search results. The one or more search engines 74 can perform feature matching, text-based searching, embedding-based searching (e.g., k-nearest neighbor searching), metadata-based searching, multi-modal searching, web resource searching, image searching, text searching, and / or application searching.

[0148] Additionally and / or alternatively, the sensor processing system 60 can include one or more multi-modal processing blocks 76 that can be used to assist in processing multi-modal data. The one or more multi-modal processing blocks 76 can include generating multi-modal queries and / or multi-modal embeddings for processing by one or more machine learning models and / or one or more search engines 74.

[0149] The output of the sensor processing system 60 can then be processed using an output determination system 80 to determine one or more outputs to provide to a user. The output determination system 80 can include heuristic-based determinations, machine learning model-based determinations, user selection-based determinations, and / or context-based determinations.

[0150] The output determination system 80 can determine how and / or where to provide one or more search results (and / or query suggestions) in a search results interface 82. Additionally and / or alternatively, the output determination system 80 can determine how and / or where to provide one or more machine learning model outputs in a machine learning model output interface 84. In some implementations, one or more search results and / or one or more machine learning model outputs can be provided for display via one or more user interface elements. The one or more user interface elements can be overlaid on displayed data. For example, one or more detection indicators can be overlaid on a detected object in a viewfinder. The one or more user interface elements can be selectable to perform one or more additional searches and / or one or more additional machine learning model processes. In some implementations, the user interface elements can be provided as specialized user interface elements for a particular application and / or can be provided uniformly across different applications. The one or more user interface elements can include pop-up displays, interface overlays, interface tiles and / or widgets, carousel interfaces, audio feedback, animations, interactive widgets, and / or other user interface elements.

[0151] Additionally and / or alternatively, the data associated with the output of the sensor processing system 60 can be utilized to generate and / or provide an augmented reality experience and / or a virtual reality experience 86. For example, one or more obtained data sets can be processed to generate one or more augmented reality rendering assets and / or one or more virtual reality rendering assets, which can then be utilized to provide an augmented reality experience and / or a virtual reality experience 86 to a user. An augmented reality experience can render information associated with an environment into the respective environment. Alternatively and / or additionally, objects related to the processed data sets can be rendered into a user environment and / or a virtual environment. Rendering data set generation can include training one or more neural radiance field models to learn a three-dimensional representation of one or more objects.

[0152] In some implementations, one or more action prompts 88 can be determined based on the output of the sensor processing system 60. For example, a search prompt, a purchase prompt, a generation prompt, a reservation prompt, a call prompt, a redirection prompt, and / or one or more other prompts can be determined to be associated with the output of the sensor processing system 60. The one or more action prompts 88 can then be provided to a user via one or more selectable user interface elements. In response to a selection of the one or more selectable user interface elements, a respective action of the respective action prompt can be performed (e.g., a search can be performed, a purchase application programming interface can be utilized, and / or another application can be opened).

[0153] In some implementations, one or more generative models 90 can be used to process one or more data sets and / or the output of the sensor processing system 60 to generate model-generated content items, which can then be provided to a user. The generation can be prompted based on a user selection and / or can be performed automatically (e.g., automatically based on one or more conditions, which can be associated with a threshold amount of unidentified search results).

[0154] The output determination system 80 can use a data augmentation block 92 to process one or more data sets and / or the output of the sensor processing system 60 to generate augmented data. For example, one or more images can be processed using the data augmentation block 92 to generate one or more augmented images. Data augmentation can include data correction, data cropping, removal of one or more features, addition of one or more features, resolution adjustment, lighting adjustment, saturation adjustment, and / or other augmentations.

[0155] In some implementations, one or more data sets and / or the output of the sensor processing system 60 can be stored based on a determination of a data storage block 94.

[0156] The output of the output determination system 80 can then be provided to a user via one or more output components of the user computing device 52. For example, one or more user interface elements associated with the one or more outputs can be provided for display via a visual display of the user computing device 52.

[0157] The process can be performed iteratively and / or continuously. One or more user inputs to the provided user interface elements can adjust and / or influence the continuous processing loop.

[0158] The technology discussed herein makes reference to servers, databases, software applications, and other computer-based systems and actions taken by and information sent to and from such systems. The inherent flexibility of computer-based systems allows for a great variety of possible configurations, combinations, and divisions of tasks and

[0159] While the subject matter has been described in detail with respect to various specific embodiments of the subject matter, it should be understood that the various specific embodiments are meant to be illustrative only and not limiting of the disclosure. Certain modifications, changes, and equivalents will be apparent to those skilled in the art upon reading this disclosure. Therefore, it is intended that this disclosure not be limited to the examples described herein, but instead has wide applicability and scope. For example, features shown or described as part of one embodiment can be used with another embodiment to create yet another embodiment. It is therefore intended that this disclosure cover all such changes and modifications that come within the scope of the disclosure.

Claims

1. A computer-implemented method for proactive querying and content suggestion, the method comprising: Web data is obtained by a computing system including one or more processors, wherein the web data describes one or more topics; The computing system processes the web data to determine change events that have occurred, wherein the change events include updates associated with the one or more topics; The computing system uses a generative model to process the data describing the change event to generate one or more query suggestions; The computing system obtains one or more web resources associated with the change event; The computing system uses the generative model to process the one or more web resources to generate a change event summary; The computing system provides one or more query suggestions and a summary of the change events for display. The computing system receives a selection of at least one of the one or more query suggestions or the change event summary; as well as The computing system provides model-generated content items for display, wherein the model-generated content items are generated by the generating model, wherein the model-generated content items include information associated with the change event, and wherein the information includes details obtained from one or more databases and formatted into natural language articles by the generating model.

2. The method of claim 1, further comprising: The computing system obtains user data associated with the user; The computing system determines, based on the user data, whether the user is interested in one or more specific topics; as well as The processing of the data describing the change event by the computing system using the generative model to generate the one or more query suggestions is performed in response to the following: The computing system determines that the change event is associated with one or more specific topics.

3. The method of claim 1, further comprising: A second change event determined by the computing system, wherein the second change event includes an update associated with one or more second topics; The computing system uses the generative model to process the data describing the second change event to generate one or more second query suggestions; The computing system obtains one or more second web resources associated with the second change event; The computing system uses the generative model to process the one or more second web resources to generate a second change event summary; as well as The computing system provides one or more second query suggestions and a summary of the second change event for display.

4. The method of claim 3, wherein one or more query suggestions and the change event summary are provided as a first pair, wherein one or more second query suggestions and the second change event summary are provided as a second pair, and wherein the first pair and the second pair are provided in the news push interface.

5. The method of claim 1, further comprising: In response to the selection, the computing system further provides a search results interface for display, wherein the search results interface includes multiple search results in response to a search query associated with the one or more query suggestions.

6. The method of claim 5, wherein the plurality of search results includes the one or more web resources.

7. The method of claim 1, wherein the web data is obtained prior to receiving a search query.

8. The method of claim 7, wherein, prior to receiving a search query, the one or more query suggestions and the change event summary are provided for display.

9. A computing system for proactive querying and content suggestion, the system comprising: One or more processors; as well as One or more non-transitory computer-readable media storing instructions that, when executed by the one or more processors, cause the computing system to perform operations, including: Obtain multiple queries, wherein the multiple queries are obtained from multiple users; The multiple queries are processed to determine query trends, wherein the query trends describe an increase in the number of queries associated with a particular topic; The language model is used to process the data describing the query trends to generate one or more query suggestions; Obtain one or more web resources in response to the one or more query suggestions; The language model is used to process the one or more web resources to generate a content summary, wherein the content summary describes the content of the one or more web resources; Provide one or more query suggestions and the content summary for display; Receive selection of at least one of the one or more query suggestions or the content summary; and Provide model-generated content items for display, wherein the model-generated content items are generated by a generative model, wherein the model-generated content items include information associated with a specific topic related to the query trend, and wherein the information includes details obtained from one or more databases and formatted into natural language articles by the generative model.

10. The system of claim 9, wherein the one or more query suggestions are generated in a question format.

11. The system of claim 10, wherein the content summary is generated in an answer format, and wherein the content summary is generated as a question in response to the one or more query suggestions.

12. The system of claim 9, wherein the content summary includes a natural language summary, the natural language summary including one or more details obtained from the one or more web resources.

13. The system of claim 9, wherein processing the data describing the query trend using the language model to generate the one or more query suggestions is performed in response to: Determine that the specific topic is associated with one or more content items previously viewed by a specific user.

14. The system of claim 9, wherein the query trend is a global query trend.

15. A non-transitory computer-readable medium comprising one or more non-transitory computer-readable media that collectively store instructions, the instructions causing the one or more computing devices to perform operations when executed by the one or more computing devices, the operations including: Obtain web data, wherein the web data describes content provided by an aggregation of web resources; The web data is processed to determine multiple change events that have occurred, wherein the multiple change events include multiple updates associated with multiple topics; The data describing the plurality of change events is processed using a language model to generate a plurality of query suggestions, wherein the language model includes a generative model, and wherein each query suggestion is associated with a corresponding change event among the plurality of change events; Obtain multiple corresponding web resources associated with the multiple change events, wherein the multiple corresponding web resources include one or more corresponding web resources for each of the multiple change events; Associate each of the plurality of query suggestions with a corresponding web resource among the plurality of corresponding web resources to generate a query suggestion web resource pair for each of the plurality of change events; and A news feed interface is provided for display, wherein the news feed interface includes a set of query-suggested web resources associated with at least a subset of the plurality of change events.

16. The non-transitory computer-readable medium of claim 15, wherein each of the plurality of change events is determined based on: Determine the inflow of articles related to the relevant topic.

17. The non-transitory computer-readable medium of claim 15, wherein each of the plurality of change events is determined based on: Determine the inflow of social media posts related to the relevant topic.

18. The non-transitory computer-readable medium of claim 15, wherein the language model comprises a machine learning generative model trained to generate natural language data tailored to mimic the vocabulary, intonation, and style of a particular user.

Images