Proactive queries and content suggestions for questions and answers generated using generative models

Abstract

Systems and methods for proactive query and content suggestions may include obtaining web data, determining a change event occurring, and generating query and content suggestions. Generating queries and content suggestions may include processing data describing a change event using a generative model to generate one or more model-generated query suggestions. One or more web resources may be obtained and then processed to generate a change event summary. One or more query suggestions and change event summary may then be provided for display.

Description

[0001] Related Applications

[0002] This application claims priority to and the benefit of U.S. Provisional Patent Application No. 63 / 584,015, filed on September 20, 2023. U.S. Provisional Patent Application No. 63 / 584,015 is hereby incorporated by reference in its entirety. Technical Field

[0003] The present disclosure generally relates to proactive query and content suggestions. More specifically, the present disclosure relates to determining that an event has occurred, generating suggested queries using a generative language model, determining suggested content, and providing query and content suggestions. Background Art

[0004] It may be difficult to stay abreast of changes and general news related to global and / or interest-specific changes. Furthermore, because new information is constantly being added to the web, it may become non-intuitive to understand when to look for information and what to look for. Additionally, content requested by a user may not be readily available to the user based on the user not knowing where to search, based on where the content is stored, and / or based on the content being difficult to understand.

[0005] Existing query suggestion technologies may be outdated because the suggestions may simply be a repeat of past queries or associated with the last viewed content. Technologies that provide trend-based suggestions may not provide customized suggestions, and the suggested queries may result in a large number of results to be navigated and reviewed to determine what topics are popular. Summary of the invention

[0006] Aspects and advantages of embodiments of the present disclosure will be set forth in part in the following description, or may be learned from the description, or may be learned through practice of the embodiments.

[0007] An example aspect of the present disclosure relates to a computer-implemented method for active query and content suggestion. The method may include obtaining web data by a computing system including one or more processors. The web data may describe one or more topics. The method may include processing the web data by a computing system to determine a change event that occurred. The change event may include an update associated with one or more topics. The method may include processing data describing the change event by a computing system using a generative model to generate one or more query suggestions. The method may include obtaining one or more web resources associated with the change event by a computing system. The method may include processing one or more web resources by a computing system using a generative model to generate a change event summary and providing one or more query suggestions and the change event summary by the computing system for display.

[0008] In some implementations, the method may include obtaining, by the computing system, user data associated with the user and determining, by the computing system based on the user data, that the user is interested in one or more specific topics. Processing, by the computing system, data describing the change event using the generative model to generate one or more query suggestions may be performed in response to determining, by the computing system, that the change event is associated with the one or more specific topics.

[0009] In some implementations, the method may include determining, by a computing system, that a second change event has occurred. The second change event may include an update associated with one or more second topics. The method may include processing, by a computing system, data describing the second change event using a generative model to generate one or more second query suggestions. The method may include obtaining, by a computing system, one or more second web resources associated with the second change event and processing, by a computing system, one or more second web resources using a generative model to generate a second change event summary. The method may include providing, by a computing system, one or more second query suggestions and a second change event summary for display. One or more query suggestions and a change event summary may be provided as a first pair. In some implementations, one or more second query suggestions and a second change event summary may be provided as a second pair. The first pair and the second pair may be provided in a news push interface.

[0010] In some implementations, the method may include receiving, by a computing system, a selection of at least one of one or more query suggestions or change event summaries and providing, by the computing system, a search results interface for display. The search results interface may include a plurality of search results in response to a search query associated with the one or more query suggestions. The plurality of search results may include one or more web resources. In some implementations, the method may include receiving, by a computing system, a selection of at least one of one or more query suggestions or change event summaries and providing, by the computing system, a model-generated content item for display. The model-generated content item may be generated using a generation model. The model-generated content item may include information associated with the change event, and the information may include details obtained from one or more databases and formatted as a natural language article by the generation model. In some implementations, the web data may be obtained before receiving the search query. One or more query suggestions and a change event summary may be provided for display before receiving the search query.

[0011] Another example aspect of the present disclosure relates to a computing system for active query and content suggestion. The system may include one or more processors and one or more non-transitory computer-readable media storing instructions together, which instructions, when executed by one or more processors, cause the computing system to perform operations. The operation may include obtaining multiple queries. The multiple queries may be obtained from multiple users. The operation may include processing multiple queries to determine query trends. In some implementations, the query trend may describe an increase in the number of queries associated with a particular topic. The operation may include processing data describing the query trend using a language model to generate one or more query suggestions. The operation may include obtaining one or more web resources in response to one or more query suggestions and processing one or more web resources using a language model to generate a content summary. The content summary may describe the content of one or more web resources. The operation may include providing one or more query suggestions and a content summary for display.

[0012] In some implementations, one or more query suggestions may be generated in a question format. A content summary may be generated in an answer format, and the content summary may be generated in response to the question of the one or more query suggestions. The content summary may include a natural language summary that includes one or more details obtained from one or more web resources. In some implementations, processing data describing query trends using a language model to generate one or more query suggestions may be performed in response to determining that a particular topic is associated with one or more content items previously viewed by a particular user. The query trend may be a global query trend.

[0013] Another example aspect of the present disclosure relates to one or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more computing devices, cause one or more computing devices to perform operations. The operation may include obtaining web data. The web data may describe content provided by a collection of web resources. The operation may include processing the web data to determine multiple change events that occurred. The multiple change events may include multiple updates associated with multiple topics. The operation may include processing data describing multiple change events using a language model to generate multiple query suggestions. The language model may include a generation model. In some implementations, each query suggestion may be associated with a corresponding change event in the multiple change events. The operation may include obtaining multiple corresponding web resources associated with the multiple change events. The multiple corresponding web resources may include one or more corresponding web resources for each of the multiple change events. The operation may include associating each of the multiple query suggestions with a corresponding web resource in the multiple corresponding web resources to generate a query suggestion web resource pair for each of the multiple change events and providing a news push interface for display. The news push interface may include a set of query suggestion web resource pairs associated with at least a subset of the multiple change events.

[0014] In some implementations, each of the plurality of change events may be determined based on determining an influx of articles associated with the respective topic. Each of the plurality of change events may be determined based on determining an influx of social media posts associated with the respective topic. The language model may include a machine learning generative model trained to generate natural language data customized to simulate the vocabulary, tone, and style of a particular user.

[0015] Other aspects of the disclosure relate to various systems, apparatuses, non-transitory computer-readable media, user interfaces, and electronic devices.

[0016] These and other features, aspects and advantages of various embodiments of the present disclosure will be better understood with reference to the following description and appended claims.The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate example embodiments of the present disclosure and, together with the description, serve to explain the relevant principles. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] A detailed discussion of the embodiments for those skilled in the art is set forth in this specification with reference to the accompanying drawings, in which:

[0018] Figure 1 Depicted is a block diagram of an example query suggestion system according to an example embodiment of the present disclosure.

[0019] Figure 2 Depicted is a block diagram of an example proactive query suggestion system according to an example embodiment of the present disclosure.

[0020] Figure 3 Depicted is a flow diagram of an example method for performing query and content suggestions according to an example embodiment of the present disclosure.

[0021] Figure 4 Depicted is a block diagram of an example query trend determination system according to an example embodiment of the present disclosure.

[0022] Figure 5 Depicted is a block diagram of an example query news push system according to an example embodiment of the present disclosure.

[0023] Figure 6 Depicted is a diagram of an example news feed interface according to an example embodiment of the present disclosure.

[0024] Figure 7 Depicted is a flow diagram of an example method for performing query trend determination and query suggestion according to an example embodiment of the present disclosure.

[0025] Figure 8Depicted is a flow chart of an example method for performing query news feed generation according to an example embodiment of the present disclosure.

[0026] Fig. 9 Depicted is a diagram of an example query suggestion interface according to an example embodiment of the present disclosure.

[0027] Fig. 10A Depicted is a block diagram of an example computing system that performs query and content suggestions according to an example embodiment of the present disclosure.

[0028] Fig. 10B Depicted is a block diagram of an example computing system that performs query and content suggestions according to an example embodiment of the present disclosure.

[0029] Reference numerals repeated among the various drawings are intended to identify like features in the various implementations. DETAILED DESCRIPTION

[0030] In general, the present disclosure relates to systems and methods for active query and content suggestions. Specifically, the systems and methods disclosed herein can utilize change event determination and one or more generative models to generate timely, topical and / or easy-to-understand language query suggestions and / or content suggestions. Web data (e.g., data describing search query logs, web traffic, web page updates, domain / web page generation, download rates, etc.) can be obtained and processed to determine that one or more topics have changed (e.g., company mergers, announcements of new comic book movies, trades of specific players, etc.). Generative models (e.g., large language models) can be used to process data describing change events to generate queries that can be suggested to users. Queries can be generated to involve obtaining search results associated with specific aspects of the changed topics. Generative models can be trained to generate query suggestions formatted in natural language so that users can easily understand the topic and direction of the query. Then, queries and / or data describing change events can be used to obtain web resources (e.g., web pages, videos, images, audio clips, articles, books, and / or other data) associated with change events (e.g., specific aspects of the changed change events). In some implementations, a web resource may be processed using a generative model to generate a change event summary. The change event summary may summarize what has changed as outlined by the web resource. The change event summary may include a topic overview, an indicator of what has changed, and / or what impact the change has. The query and web resource and / or change event summary may be provided to the user as a query suggestion along with a preview of the answer to the question posed by the query.

[0031] Existing query suggestion techniques may be outdated because the suggestions may simply be a repetition of past queries (e.g., suggesting a previously entered query) or associated with the last viewed content (e.g., suggesting a subtopic associated with a previously viewed web page). Techniques that provide trend-based suggestions may not provide customized suggestions, and the suggested queries may result in a large number of results to be navigated to determine why a topic is popular. For example, not every trend may be relevant to the user, and / or the search result list may not be easy to understand, which may result in the user performing a tedious and time-consuming review of the search result list to understand the cause of the trend.

[0032] The systems and methods disclosed herein can proactively determine updates to a topic, which can then be processed to generate natural language query suggestions that can be provided along with one or more exemplary results and / or a model-generated summary of the update. Global and / or user-specific trends and changes can be tracked based on query trends and / or based on recursive searches of one or more databases. Once trends and / or changes are detected, the generative model can generate suggested queries (e.g., suggested questions), and multiple search results for the query can be processed to generate a summary that can be provided along with the query suggestions.

[0033] Active determination may be performed before providing search input, which may guide the user to changes in one or more specific topics without the user having to be aware of the change in advance. In addition, query and content suggestions may be customized based on user preferences, determined user interests, user associations, and / or user styles. For example, user data associated with a user may be processed to determine user interests, user groups (e.g., grouping of a user with one or more other users based on a social network, business network, occupation, and / or other attribute), and / or historical interactions. Query and content suggestions may then involve specific attempts to identify change events (and / or query trends) associated with topics related to the user based on determined user interests, user groups, and / or historical interactions. Additionally and / or alternatively, parameters and / or soft prompts for generating models may be trained to generate queries and / or summaries in vocabulary, intonation, and / or style.

[0034] The query and content suggestions may include generating natural language question-answer pairs. For example, the query suggestion may be formatted as a plain language question and may be comprehensive. The content summary may be formatted as an answer responsive to the question posed by the query suggestion. The active determination of the question-answer suggestions may be generated by a generative model that may generate a natural language question-answer summary for the determined change event.

[0035] Tracking trends and / or topic changes can be used to proactively perform query suggestions in the form of natural language questions and answers, which can be implemented by search engines, news platforms, virtual assistants, and / or other applications and platforms to provide users with timely and easy-to-understand information. The generated suggestions can be provided as query suggestions and / or in the form of a discovery feed with multiple other generated questions and answers associated with multiple other trends and / or changes, so that users are pushed timely and easy-to-understand language queries and content summaries, thereby becoming more aware of changes in global topics and / or topics of interest to users.

[0036] The generative model may process multiple queries to generate queries that encompass query trends associated with a subset of multiple different queries. Alternatively and / or in addition, the generative model may process query logs of a particular user and / or a particular set of users to generate queries that predict the user's search interests based on the learned sequences and prediction data of the generative model. The generative model may process titles, uniform resource locators (URLs), and / or content items of articles or other web resources to determine change events and generate suggested queries and / or change event summaries.

[0037] Prompts for the generation model may include rewriting the logged query to make it more comprehensive, interesting, detailed, subtle, entertaining, objective, informal or formal, serious or funny, and / or appropriate for one or more other preferences. The generation model may determine query trends and / or change events, process multiple queries and multiple web resources to generate queries that include directions for subsets of queries associated with trends and / or changes, and supplement the directions of the queries with details from the web resources to have detailed query suggestions that specifically identify points of interest for the suggested query and may include aspects of what has changed and / or why the trend exists.

[0038] The system and method of the present disclosure provide a variety of technical effects and benefits. As an example, the system and method can provide active query and content suggestions. The system and method can use event determination and generation models to generate timely and easy-to-understand query and content suggestions. Specifically, changes associated with one or more topics can be determined. The generation model can be used to process data describing the changes to generate one or more query suggestions. One or more query suggestions and / or data describing the changes can be used to identify and obtain one or more web resources associated with the change event. In some implementations, the generation model can be used to process one or more web resources to generate a change event summary. Then one or more query suggestions and a change event summary can be provided to the user.

[0039] Another technical benefit of the systems and methods of the present disclosure is the ability to generate natural language question-answer formats using a generative language model. The generative language model can be used to generate understandable information in the format of a traditional conversation. In some implementations, the generative language model and / or one or more soft prompts (e.g., a set of machine learning parameters that can be processed by the generative language model using input) can be trained to simulate the tone, style, and / or vocabulary of a particular user and / or set of users to provide queries and / or summaries in the terms, tone, style, and / or dialect traditionally used by the user.

[0040] Another example of technical effects and benefits relates to the improved computing efficiency of computing systems and improvements in their functionality. For example, the systems and methods disclosed herein may utilize generative models to provide natural language suggestions and summaries, which may reduce the number of web pages accessed by users in order to obtain knowledge about the content and / or time of updates to a topic. In addition, the systems and methods may reduce the number of searches performed by users for each knowledge query. In addition, in some implementations, the systems and methods may perform query and content generation for event determination and / or model generation via a server computing system, which may limit the data sent and / or received by a user computing system. By reducing web page accesses, search instances, and system calculations at a user computing system, the systems and methods may be utilized to provide targeted and information-rich information to user computing systems with limited resources (e.g., limited computing resources (e.g., smart devices and / or mobile devices) and / or limited network access (e.g., in a poor reception area)).

[0041] Referring now to the drawings, example embodiments of the present disclosure will be discussed in further detail.

[0042] Figure 1 A block diagram of an example query suggestion system 10 is depicted in accordance with an example embodiment of the present disclosure. In some implementations, the query suggestion system 10 is configured to receive and / or obtain a set of web data 12 describing one or more topics, and generate, determine, and / or provide one or more suggestions 20 as a result of receiving the web data 12, the one or more suggestions may include one or more query suggestions and / or change event summaries. Thus, in some implementations, the query suggestion system 10 may include a generation model 16 operable to process data describing change events to generate query suggestions expressed in understandable language terms and grammar.

[0043] For example, web data 12 may be obtained from one or more computing systems (e.g., one or more server computing systems). Web data 12 may include web page edits, web page uploads, web traffic data, query usage by multiple users, and / or other web data. Web data may be obtained periodically, may be obtained in response to user app usage instances, and / or may be obtained continuously.

[0044] The event determination block 14 may be used to process the web data 12 to determine the change events that occurred. The change events may be updates to one or more topics. Change events may be determined based on query trends, a threshold number of resource updates associated with a topic, an increase in web traffic that exceeds a threshold, and / or a threshold number of new content items associated with a topic. Thresholds may vary based on time, location, topic, generality of a topic, and / or one or more other contexts. A change event may be associated with an individual, an entity, a group of entities, a location, an object, and / or another topic.

[0045] The data describing the change event may be processed using a generative model 16 to generate one or more query suggestions that involve surfacing search results related to the change event. The generative model 16 may include a machine learning language model trained to generate natural language output. The one or more query suggestions may be formatted as questions. In some implementations, by training the generative model (and / or soft prompts) for a user's conversation, the one or more query suggestions may be in the user's tone, vocabulary, and / or style.

[0046] One or more web resources 18 may be obtained based on data describing the change event and / or one or more query suggestions. One or more web resources 18 may be top search results and / or trusted content items from an audit database. One or more web resources 18 may be resources processed to determine the change event and / or web resources determined to confirm the existence of an update and / or details of the change event.

[0047] In some implementations, one or more web resources 18 may be processed using the generation model 16 to generate a change event summary. The change event summary may describe details of the change event based on information in the one or more web resources 18. The change event summary may be a plain language summary and may be formatted as answers to questions suggested by one or more query suggestions.

[0048] The one or more query suggestions and the one or more web resources may be presented to the user as search suggestions 20. Alternatively and / or in addition, the one or more query suggestions and the change event summary may be provided as search suggestions 20. The one or more suggestions 20 may be generated and provided prior to receiving a search input, and may be updated based on any input received from the user.

[0049] Figure 2 A block diagram of an example proactive query suggestion system 200 is depicted according to an example embodiment of the present disclosure. The proactive query suggestion system 200 is similar to Figure 1The query suggestion system 10 of FIG. 1 is different in that the active query suggestion system 200 further includes an interest determination block 232 and / or a news push interface 228 .

[0050] For example, web data 212 may be obtained from one or more web platforms and / or one or more databases (e.g., one or more domains). Web data 212 may be obtained based on a specific user and / or may be general web data associated with global change tracking. Web data 212 may include web page edits, web page uploads, web traffic data, query usage by multiple users, and / or other web data. Web data 212 may be obtained periodically, may be obtained in response to user app usage instances, and / or may be obtained continuously.

[0051] The web data 212 may be processed using an event determination block 214 to determine a change event 222 that occurred. Determining the change event that occurred may be performed by one or more machine learning models that may be trained to identify topic updates, web anomalies, and / or features that describe the change event 222. The change event 222 may be an update to one or more topics. One or more topics may be associated with sports, politics, economics, entertainment, and / or other topics. The change event 222 may be determined based on query trends, a threshold number of resource updates associated with a topic, an increase in web traffic that exceeds a threshold, and / or a threshold number of new content items associated with a topic. The threshold may vary based on time, location, topic, generality of a topic, and / or one or more other contexts. The change event 222 may be associated with an individual, an entity, a group of entities, a location, an object, and / or another topic.

[0052] In some implementations, the determination of the change event 222 may be customized based on a specific user. For example, user data 230 associated with a specific user may be obtained. The user data 230 may include user preferences, user profile data, user relationships, the user's search history, the user's browsing history, the user's purchase history, and / or other interaction data associated with the user. The user data 230 may be processed using an interest determination block 232 to determine a set of topics associated with the user. The set of topics may include topics determined to be of interest to the user, topics associated with the user's career, topics of interest to individuals associated with the user, topics associated with entities of which the user is a part, and / or other topics identified as relevant to the user. The interest determination block 232 may include one or more machine learning models and / or may include one or more deterministic functions and / or thresholds. In some implementations, the change event 222 may be determined based on the output of the interest determination block, which may include adjusting thresholds for one or more topics and / or limiting the topics being reviewed to facilitate change event determination.

[0053] The data describing the change event 222 may be processed using a generative model 216 (e.g., a generative language model) to generate one or more query suggestions 224 (e.g., one or more example queries to select from) that relate to surfacing search results related to the change event 222. The one or more query suggestions 224 may include detailed language related to specific aspects of the subject matter that has changed. The generative model 216 may include a machine learning language model trained to generate natural language output that may include one or more complete sentences that may be read as conversational language and intonation. The one or more query suggestions 224 may be formatted as questions. In some implementations, by training the generative model 216 (and / or soft prompts) for a user's conversation, the one or more query suggestions 224 may be in the user's intonation, vocabulary, and / or style.

[0054] One or more web resources 218 may be obtained based on data describing change event 222 and / or one or more query suggestions 224. One or more web resources 218 may include web pages, videos, images, audio files, documents, and / or other content items. One or more web resources 218 may be top search results (e.g., top search results for a query of query suggestion 224) and / or trusted content items from a review database (e.g., scholarly articles from a scholarly publication database). One or more web resources 218 may be resources that are processed to determine change event 222 and / or web resources that are determined to confirm the existence of an update and / or details of change event 222.

[0055] In some implementations, the one or more web resources 218 can be processed using the generation model 216 to generate a change event summary 226 (e.g., one or more sentences that briefly identify one or more details associated with a subject update). The change event summary 226 can describe the details of the change event 222 based on the information in the one or more web resources 218. The change event summary 226 can be a plain language summary and can be formatted as an answer to the question posed by the one or more query suggestions 224.

[0056] One or more query suggestions 224 and one or more web resources 218 may be presented to the user as search suggestions 220. Alternatively and / or in addition, one or more query suggestions 224 and change event summary 226 may be provided as search suggestions 220. One or more suggestions 220 may be generated and provided prior to receiving search input, and may be updated based on any input received from the user.

[0057] In some implementations, multiple query suggestions, multiple web resources, and / or multiple change event summaries may be generated and / or obtained to generate multiple query and content suggestions that may be provided in the news feed interface 228. The news feed interface 228 may display each query suggestion 224 along with a corresponding web resource 218 and / or a corresponding change event summary 226. The displayed content may then be selected to perform a search on the query suggestion 224 to generate a search results interface, redirect to a corresponding web resource 218, and / or generate a model-generated article using the generative model 216 that describes the change event 222 in a longer and more comprehensive format than the change event summary 226.

[0058] In some implementations, the generation model 216 may perform event determination. For example, the generation model 216 may process the web data 212 and generate predicted query suggestions 224. The web data 212 may include data describing popular and / or recent articles, social media posts, and / or other web data. The generation model 216 may process titles, URLs, and / or content items of recent and / or popular articles to predict change events 222 (e.g., updates on topics and / or trends). The generation model 216 may then generate query suggestions 224 based on the determined change events 222. In some implementations, the web data 212 may include data describing previously viewed articles, social media profiles, domains, and / or web platforms associated with a user. The generation model 216 may process data describing previously viewed articles, social media profiles, domains, and / or web platforms associated with a user to generate query suggestions 224 based on recent changes to previously viewed resources, based on predicting queries based on previously viewed resources, and / or based on trends determined based on previously viewed resources. In some implementations, the generation model 216 can perform the generation of the predictive query suggestions 224 based on a viewing order of previously viewed resources.

[0059] Figure 3 Depicted is a flowchart of an example method performed in accordance with an example embodiment of the present disclosure. Figure 3 The steps performed in a specific order are depicted for the purpose of illustration and discussion, but the method of the present disclosure is not limited to the specific illustrated order or arrangement. The various steps of the method 300 may be omitted, rearranged, combined and / or adjusted in various ways without departing from the scope of the present disclosure.

[0060] At 302, a computing system may obtain web data. The web data may describe one or more topics. In some implementations, the web data may be obtained before receiving a search query. The web data may be generated by obtaining and processing a plurality of content items. In some implementations, the web data may describe a plurality of queries obtained from a plurality of users. The web data may include data describing changes in web traffic for one or more web pages, platforms, and / or domains.

[0061] At 304, the computing system may process web data to determine the change event that occurred. The change event may include an update associated with one or more topics. The change event may describe an increase in the count of web traffic associated with one or more topics. Alternatively and / or in addition, the change event may describe an increase in the count of global and / or localized search queries associated with one or more topics. The change event may be determined based on meeting one or more thresholds. One or more thresholds may vary based on the degree of generality of the corresponding topic (e.g., for more specific topics, the threshold may be lower). Additionally and / or alternatively, one or more thresholds may be determined based on the determined interest level of a specific user to the corresponding topic (e.g., the higher the determined interest level may result in one or more thresholds being lower, which may result in a more likely determination of a change event for a topic that is determined to be highly interesting to the user). The interest level may be determined based on user data associated with a specific user. The user data may be associated with user preferences, user search history, user browsing history, user social media accounts, and / or other user-specific data. One or more topics may be associated with one or more specific entities, one or more specific locations, one or more specific products, one or more specific individuals, and / or one or more other topics.

[0062] At 306, the computing system may process the data describing the change event using a generative model to generate one or more query suggestions. The generative model may include a large language model, a visual language model, an image-to-text model, and / or an image generation model. In some implementations, the data describing the change event may be generated (and / or processed) using one or more semantic understanding models, one or more detection models, one or more classification models, one or more segmentation models, one or more embedding models, and / or one or more enhancement models. The data describing the change event may describe a topic classification and / or an identification of what has changed.

[0063] At 308, the computing system may obtain one or more web resources associated with the change event. The one or more web resources may be obtained by processing the one or more query suggestions using a search engine to determine one or more web resources responsive to one or more queries associated with the one or more query suggestions. Alternatively and / or in addition, the one or more web resources may be obtained based on a determined topic associated with the change event.

[0064] At 310, the computing system may process one or more web resources using the generative model to generate a change event summary. The change event summary may include a natural language summary of updates based on processing the one or more web resources. Updates may be determined based on topic classification, semantic understanding, and / or other natural language processing techniques. In some implementations, the change event summary may include a model-generated summary provided with one or more references associated with the one or more web resources.

[0065] At 312, the computing system may provide one or more query suggestions and a summary of change events for display. Prior to receiving the search query, one or more query suggestions and a summary of change events may be provided for display. The one or more query suggestions and the summary of change events may be provided for display in a search query suggestion interface that provides the pair in a panel adjacent to a search query input box.

[0066] In some implementations, the computing system may obtain user data associated with the user and determine, based on the user data, that the user is interested in one or more specific topics. Processing the data describing the change event using the generative model to generate one or more query suggestions may be performed in response to determining that the change event is associated with the one or more specific topics.

[0067] In some implementations, the computing system may determine a second change event that occurred. The second change event may include updates associated with one or more second topics. The computing system may use a generative model to process data describing the second change event to generate one or more second query suggestions, obtain one or more second web resources associated with the second change event, and use the generative model to process the one or more second web resources to generate a second change event summary. In some implementations, the computing system may provide one or more second query suggestions and a second change event summary for display. One or more query suggestions and a change event summary may be provided as a first pair. One or more second query suggestions and a second change event summary may be provided as a second pair. The first pair and the second pair may be provided in a news push interface.

[0068] In some implementations, the computing system may receive a selection of at least one of the one or more query suggestions or the change event summary and provide a search results interface for display. The search results interface may include a plurality of search results responsive to a search query associated with the one or more query suggestions. The plurality of search results may include one or more web resources.

[0069] Alternatively and / or in addition, the computing system may receive a selection of at least one of the one or more query suggestions or the change event summary and provide the model-generated content item for display. The model-generated content item may be generated using the generative model. In some implementations, the model-generated content item may include information associated with the change event. The information may include details obtained from one or more databases formatted as a natural language article by the generative model.

[0070] Figure 4 A block diagram of an example query trend determination system 400 is depicted in accordance with an example embodiment of the present disclosure. Specifically, the query trend determination system 400 can perform proactive query and content suggestions based on processing multiple queries to determine query trends, which can then cause a generation model 416 to generate one or more query suggestions that can be provided along with one or more web resources 418 and / or content summaries.

[0071] For example, multiple queries 412 may be obtained. Multiple queries may be associated with multiple different users. Multiple queries 412 may be recent queries within the past hour, day, week, and / or month. Multiple different users may be global users, users local to a particular user, and / or other users associated with a particular user based on social media connections, interests, and / or other similarities.

[0072] A trend determination block 414 may be used to process multiple queries 412 to determine one or more query trends. One or more query trends may be determined based on an influx of queries related to a particular topic and / or a particular group of topics. The determination may be based on one or more thresholds and / or based on one or more machine learning models.

[0073] The data describing one or more query trends may be processed using a generation model 416 to generate one or more query suggestions associated with the subject matter of the query trends. The generation model 416 may include a language model trained to generate natural language queries. The one or more query suggestions may be formatted as questions and may relate to specific aspects of the subject matter that are the focus features of the query trends.

[0074] Additionally or alternatively, one or more web resources 418 may be obtained based on data describing query trends and / or one or more query suggestions. One or more web resources 418 may be associated with content items selected and / or viewed by multiple users when entering a query for a query trend. Alternatively and / or additionally, one or more web resources 418 may be search results actively obtained for a query suggestion. In some implementations, one or more web resources 418 may be obtained based on the determined query trends and then processed using the generation model 416 to generate one or more query suggestions.

[0075] The one or more web resources 418 may be processed using the generation model 416 to generate a content summary describing the information described in the one or more web resources 418. The content summary may include a natural language summary describing one or more shared details of the content item in response to the query suggestion. The content summary may be formatted as a dialog in response to the prompt of the query suggestion.

[0076] One or more query suggestions and content summaries may be provided for display to a particular user as suggestions for what to search for 420. Alternatively and / or in addition, one or more query suggestions and one or more web resources may be provided for display to a particular user as suggestions for what to search for 420.

[0077] In some implementations, a generation model 416 (e.g., a large language model) can process multiple queries 412 to generate one or more query suggestions. For example, the generation model 416 can determine query trends (e.g., an influx of trends associated with a particular topic), and can generate query suggestions based on the determined query trends. Query suggestions can be generated that are more detailed, subtle, and / or denser than the queries in the multiple queries. In some implementations, query suggestions can be generated to simulate user vocabulary, user style, user tone, and / or user query length. Query suggestions can be generated to satisfy one or more query preferences (e.g., length preference, tone preference, style preference, etc.).

[0078] In some implementations, the plurality of queries 412 may include a plurality of previously used queries for a particular user. The generative model 416 may process the plurality of queries 412 to generate interesting, subtle, fun, serious, objective, and / or other types of query suggestions. Specifically, the predictive capabilities of the generative model may be used to predict queries that may be of interest to a particular user, and interesting and subtle query suggestions may be generated based on the predictions.

[0079] In some implementations, the generation model 416 may identify one or more representative queries from the plurality of queries that may describe the subject matter of the query trend. The generation model 416 may then rewrite the one or more representative queries to make them more detailed and more indicative of the subject matter of the trend. Alternatively and / or in addition, the one or more representative queries may be rewritten to personalize the query suggestions for a particular user.

[0080] Figure 5 A block diagram of an example query news push system 500 is depicted according to an example embodiment of the present disclosure. Specifically, the query news push system 500 can identify query suggestions 504, web resources 506, and / or change event summaries 508 for each change event 502, which can then be provided in a news push format along with other query suggestions.

[0081] For example, change events 502 associated with one or more topics may be determined. Change events 502 may be associated with one or more changes to one or more topics. Data describing change events 502 may be processed to generate query suggestions 504. Query suggestions 504 may be processed using a search engine to determine one or more web resources 506 associated with query suggestions 504. One or more web resources 506 may include content items that include information about various aspects of one or more topics that have changed.

[0082] One or more web resources 506 may be processed to generate a change event summary 508. The change event summary 508 may be based on one content item and / or multiple content items. In some implementations, multiple content items associated with a change event may be obtained and processed. Overlaps between multiple content items may be determined, and a change event summary may be generated based on the determined overlaps and / or the determined shared features.

[0083] Change event summary 508 and / or one or more web resources 506 may be associated with query suggestion 504 to generate query content pair 510. Query content pair 510 may be processed to generate suggestion interface element 512, which may include query suggestion 504 and change event summary 508 in a conversational format (e.g., two individuals talking (e.g., question and answer)). Suggestion interface element 512 may be selected to perform one or more operations.

[0084] For example, the suggestion interface element 512 may be selected to search 514 the query suggestion 504, generate model-generated articles 516, and / or navigate to resources 518. The search 514 may be performed by processing the query suggestion 504 using a search engine to obtain and display a plurality of search results for the query. The generation model-generated articles 516 may include processing a plurality of web resources associated with the change event 502 using a generation model to generate an article describing the change event 502 with details about what changed, an overview of the subject, and / or the potential impact of the change. In some implementations, the suggestion interface element 512 may be selected to navigate to one or more web resources 506 and / or provide one or more web resources 506 in a viewing window.

[0085] Figure 6 A diagram of an example news push interface 600 according to an example embodiment of the present disclosure is depicted. Specifically, the news push interface 600 may include a query input box 602, multiple query suggestions (e.g., 604, 608, and 612), and multiple corresponding change event summaries (e.g., 606, 610, and 614). The query input box 602 may be configured to receive a search input (e.g., a text query, an image query, an audio query, and / or a multimodal query). Multiple query suggestions (e.g., 604, 608, and 612) and multiple corresponding change event summaries (e.g., 606, 610, and 614) may be provided in a news push format that displays the query and summary in pairs in a question-and-answer format. Multiple query suggestions (e.g., 604, 608, and 612) and multiple corresponding change event summaries (e.g., 606, 610, and 614) may be generated according to the tone, dialect, style, grammar, and / or terminology of the user to whom the suggestions are provided. A plurality of query suggestions (eg, 604, 608, and 612) and a plurality of corresponding change event summaries (eg, 606, 610, and 614) may be selected to perform a search, obtain additional data, and / or generate a content item.

[0086] Figure 7 Depicted is a flowchart of an example method performed in accordance with an example embodiment of the present disclosure. Figure 7 The steps performed in a specific order are depicted for the purpose of illustration and discussion, but the method of the present disclosure is not limited to the specific illustrated order or arrangement. The various steps of the method 700 may be omitted, rearranged, combined and / or adjusted in various ways without departing from the scope of the present disclosure.

[0087] At 702, a computing system may obtain a plurality of queries. The plurality of queries may be obtained from a plurality of users. The plurality of users may be global users and / or may be from a particular region. In some implementations, the computing system may determine that the plurality of users are associated with a particular user for whom the computing system is determining recommendations in the current instance. For example, a plurality of users may be determined that have one or more shared attributes (e.g., shared interests, shared contacts, shared schedules, shared activities, etc.) with the particular user. Search query data associated with each of the plurality of users may be periodically obtained and processed.

[0088] At 704, the computing system may process the plurality of queries to determine a query trend. The query trend may describe an increase in the number of queries associated with a particular topic. In some implementations, the query trend may be a global query trend. Alternatively and / or in addition, the query trend may be a regional trend, a localized trend, a trend based on an age group, a trend based on an occupation, and / or another subgroup trend.

[0089] At 706, the computing system may process the data describing the query trend using a language model to generate one or more query suggestions. The one or more query suggestions may be generated in a question format. For example, the computing system may determine a query trend associated with a potential trade acquisition between two football teams, and the language model may process the data describing the query trend to generate a query suggestion including "Who are the potential trade assets for the rumored trade between football team A and football team B?"

[0090] In some implementations, processing data describing query trends using a language model to generate one or more query suggestions may be performed in response to determining that a particular topic is associated with one or more content items previously viewed by a particular user.

[0091] At 708, the computing system may obtain one or more web resources in response to the one or more query suggestions. The one or more web resources may be obtained by processing the one or more query suggestions using a search engine. In some implementations, the one or more web resources may be one or more top search results for a given query associated with the query suggestion. Alternatively and / or in addition, the one or more web resources may include one or more content items obtained from one or more databases. The one or more databases may include one or more selected databases determined to include trusted and authentic information. The one or more databases may be general knowledge databases and / or one or more specialized databases.

[0092] At 710, the computing system may process one or more web resources using a language model to generate a content summary. The content summary may describe the content of the one or more web resources. In some implementations, the content summary may be generated in an answer format. The content summary may be generated as a question in response to one or more query suggestions. Additionally or alternatively, the content summary may include a natural language summary that includes one or more details obtained from the one or more web resources.

[0093] At 712, the computing system may provide one or more query suggestions and content summaries for display. The one or more query suggestions and content summaries may be provided in a question-and-answer format. In some implementations, additional query suggestions and additional content summaries may be provided adjacent to the one or more query suggestions and the one or more content summaries. The one or more query suggestions may be selected to navigate to a search results interface. Additionally and / or alternatively, a content summary may be selected to obtain a long-form model-generated article that includes additional details about a particular topic associated with the query trend.

[0094] Figure 8 Depicted is a flowchart of an example method performed in accordance with an example embodiment of the present disclosure. Figure 8 The steps performed in a specific order are depicted for the purpose of illustration and discussion, but the method of the present disclosure is not limited to the specific illustrated order or arrangement. The various steps of method 800 may be omitted, rearranged, combined and / or adjusted in various ways without departing from the scope of the present disclosure.

[0095] At 802, a computing system may obtain web data. The web data may describe content provided by a collection of web resources. The web data may include data from one or more specific databases and / or may be obtained from multiple databases. The web data may be generated by crawling multiple web pages.

[0096] At 804, the computing system may process the web data to determine a plurality of change events that occurred. The plurality of change events may include a plurality of updates associated with a plurality of topics. Each of the plurality of change events may be determined based on determining an influx of articles associated with the respective topic. Alternatively and / or in addition, each of the plurality of change events may be determined based on determining an influx of social media posts associated with the respective topic.

[0097] At 806, the computing system may process data describing the plurality of change events using a language model to generate a plurality of query suggestions. The language model may include a generative model. Each query suggestion may be associated with a respective change event in the plurality of change events. The language model may include a machine learning generative model trained to generate natural language data that is customized to simulate the vocabulary, intonation, and / or style of a particular user.

[0098] At 808, the computing system may obtain a plurality of corresponding web resources associated with the plurality of change events. The plurality of corresponding web resources may include one or more corresponding web resources for each of the plurality of change events. The plurality of corresponding web resources may be obtained by processing a plurality of query suggestions using a search engine. Alternatively and / or in addition, the plurality of corresponding web resources may be obtained by identifying content items associated with the corresponding change events based on data and / or web data describing the corresponding change events.

[0099] At 810, the computing system may associate each of the plurality of query suggestions with a respective web resource in the plurality of respective web resources to generate a query suggestion web resource pair for each of the plurality of change events. Associating the plurality of query suggestions with the respective web resource in the plurality of respective web resources to generate a query suggestion web resource pair for each of the plurality of change events may include generating a graphical user interface element for each of the query suggestion web resource pairs.

[0100] At 812, the computing system may provide a news push interface for display. The news push interface may include a set of query suggestion web resource pairs associated with at least a subset of multiple change events. The news push interface may include one or more graphical user interface elements for each of the query suggestion web resource pairs. The graphical user interface elements may include optional query elements, optional summary elements, and / or optional web resource elements. The optional query element may be selected to navigate to a search result interface that displays search results for the corresponding query suggestions. The optional summary element may be selected to generate and / or open a summary interface that provides a summary of change events associated with the corresponding change events. The optional resource element may be selected to open a web page viewing interface that displays the corresponding web resource.

[0101] Fig. 9 900 according to an example embodiment of the present disclosure. Specifically, Fig. 9 An example query recommendation interface 910 , an example generated query anchor interface 920 , and an example card inspired journey interface 930 are depicted.

[0102] The query recommendation interface 910 may include one or more generated query suggestions 912, which are displayed together with a discovery push card 914 that may be associated with the corresponding query suggestion 912. For example, the discovery push card 914 may include a graphic card describing a web resource associated with the query suggestion 912. Alternatively and / or in addition, the discovery push card 914 may include a model generation summary that is generated based on processing one or more web resources in response to a query. In some implementations, the query suggestions 912 and the discovery push card 914 may be provided together with one or more additional graphic cards 916 associated with one or more additional web resources that are associated with the query suggestion 912. The query recommendation interface 910 may include multiple query suggestions 912 and discovery push card 914 pairs.

[0103] The generated query anchor interface 920 may provide model-generated query suggestions 922 for display together with a discovery push card 924 associated with the model-generated query suggestions. In some implementations, one or more web resources may be determined to be relevant to a user and / or a set of users. The generated model may then process the one or more web resources to generate model-generated query suggestions 922. In addition, the discovery push card 924 may be generated to describe one or more web resources and may include a title, URL, and / or one or more content items from one or more web resources. The query recommendation interface 910 may include multiple model-generated query suggestions 922 and discovery push card 924 pairs.

[0104] The card-inspired journey interface 930 may include a discovery push card 932 provided for display with one or more action elements 934. For example, the discovery push card 932 may be associated with a query suggestion and / or may be generated based on a web resource identified without an initial query suggestion. The web resource associated with the discovery push card 932 may be processed to determine the topic associated with the web resource, the content item type, and / or one or more other attributes of the web resource. Based on the topic, content type, and / or other attributes, one or more candidate actions may be determined to be associated with the web resource. The candidate actions may include subsequent queries, question-and-answer knowledge retrieval, application redirection, shopping, ordering, mapping, and the like. For example, the web resource associated with the discovery push card 932 may be processed to generate a query that involves obtaining other web resources about a topic associated with the web resource and / or obtaining additional information associated with a subtopic mentioned in the web resource. In some implementations, the web resource associated with the discovery push card 932 may be processed using a generative model to generate a model-generated question-and-answer set that describes information that can be learned from the web resource.

[0105] One or more action elements 934 may then be generated using the candidate actions. For example, a query suggestion for obtaining additional information may be provided with the discovery push card 932 and may be selected to perform a search. Model-generated questions and answers may be provided in a drop-down format so that questions are displayed first and answers may be viewed by selecting the associated question. Each action element 934 may be selected to perform a corresponding action (e.g., search, application redirection, model reasoning, information display, etc.).

[0106] Fig. 10A A block diagram of an example computing system 100 that performs query and content suggestions according to an example embodiment of the present disclosure is depicted. System 100 includes a user computing system 102, a server computing system 130, and / or a third computing system 150 that are communicatively coupled via a network 180.

[0107] The user computing system 102 may include any type of computing device, such as, for example, a personal computing device (e.g., a laptop or desktop computer), a mobile computing device (e.g., a smartphone or tablet computer), a game console or controller, a wearable computing device, an embedded computing device, or any other type of computing device.

[0108] The user computing system 102 includes one or more processors 112 and a memory 114. The one or more processors 112 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be one processor or a plurality of processors operatively connected. The memory 114 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 114 may store data 116 and instructions 118 that are executed by the processor 112 to cause the user computing system 102 to perform operations.

[0109] In some implementations, the user computing system 102 may store or include one or more machine learning models 120. For example, the machine learning model 120 may be or may otherwise include various machine learning models, such as a neural network (e.g., a deep neural network) or other types of machine learning models, including nonlinear models and / or linear models. The neural network may include a feedforward neural network, a recurrent neural network (e.g., a long short-term memory recurrent neural network), a convolutional neural network, or other forms of neural networks. The machine learning model 120 may include one or more generative models (e.g., a large language model, a text-to-image model, an image plus commentary model, etc.). The one or more generative models may include one or more transformer models, and may include an autoregressive language model and / or an image diffusion model.

[0110] In some implementations, one or more machine learning models 120 may be received from the server computing system 130 via the network 180, stored in the user computing device memory 114, and then used or otherwise implemented by the one or more processors 112. In some implementations, the user computing system 102 may implement multiple parallel instances of a single machine learning model 120 (e.g., to perform parallel machine learning model processing across multiple instances of input data and / or detected features).

[0111] More specifically, the one or more machine learning models 120 may include one or more detection models, one or more classification models, one or more segmentation models, one or more enhancement models, one or more generation models, one or more natural language processing models, one or more optical character recognition models, and / or one or more other machine learning models. The one or more machine learning models 120 may include one or more transformer models. The one or more machine learning models 120 may include one or more neural radiation field models, one or more diffusion models, and / or one or more autoregressive language models.

[0112] One or more machine learning models 120 can be used to detect one or more object features. The detected object features can be classified and / or embedded. The classification and / or embedding can then be used to perform a search to determine one or more search results. Alternatively and / or in addition, one or more detected features can be used to determine that an indicator (e.g., a user interface element indicating the detected feature) is to be provided to indicate that the feature has been detected. The user can then select the indicator to perform feature classification, embedding, and / or searching. In some implementations, the classification, embedding, and / or searching can be performed before selecting the indicator.

[0113] In some implementations, the one or more machine learning models 120 may process image data, text data, audio data, and / or potentially coded data to generate output data, which may include image data, text data, audio data, and / or potentially coded data. The one or more machine learning models 120 may perform optical character recognition, natural language processing, image classification, object classification, text classification, audio classification, context determination, action prediction, image correction, image enhancement, text enhancement, sentiment analysis, object detection, error detection, restoration, video stabilization, audio correction, audio enhancement, and / or data segmentation (e.g., mask-based segmentation).

[0114] Additionally or alternatively, one or more machine learning models 140 may be included in or otherwise stored and implemented by a server computing system 130 that communicates with the user computing system 102 according to a client-server relationship. For example, the machine learning model 140 may be implemented by the server computing system 140 as part of a network service (e.g., a viewfinder service, a visual search service, an image processing service, an environmental computing service, and / or an overlay application service). Thus, one or more models 120 may be stored and implemented at the user computing system 102, and / or one or more models 140 may be stored and implemented at the server computing system 130.

[0115] In some implementations, the computing system 100 may utilize one or more soft hints to adjust one or more machine learning models (120 and / or 140) for downstream tasks. One or more soft hints may include a set of adjustable parameters that can be trained (or tuned) when the parameters of one or more machine learning models (120 and / or 140) are fixed. One or more soft hints may be trained for a specific task and / or a specific set of tasks. Alternatively and / or in addition, one or more soft hints may be trained to adjust one or more machine learning models (120 and / or 140) to perform reasoning for a specific individual and / or one or more entities so that the output is customized for the specific individual and / or specific entity. One or more soft hints may be obtained and processed by one or more machine learning models (120 and / or 140) using one or more inputs.

[0116] The user computing system 102 may also include one or more user input components 122 that receive user input. For example, the user input component 122 may be a touch-sensitive component (e.g., a touch-sensitive display screen or a touchpad) that is sensitive to the touch of a user input object (e.g., a finger or a stylus). The touch-sensitive component may be used to implement a virtual keyboard. Other example user input components include a microphone, a traditional keyboard, or other devices by which a user can provide user input.

[0117] In some implementations, the user computing system may store and / or provide one or more user interfaces 124, which may be associated with one or more applications. One or more user interfaces 124 may be configured to receive input and / or provide data for display (e.g., image data, text data, audio data, one or more user interface elements, augmented reality experiences, virtual reality experiences, and / or other data for display). The user interface 124 may be associated with one or more other computing systems (e.g., server computing system 130 and / or third-party computing system 150). The user interface 124 may include a viewfinder interface, a search interface, a generative model interface, a social media interface, and / or a media content gallery interface.

[0118] The user computing system 102 may include one or more sensors 126 and / or receive data from the one or more sensors. The one or more sensors 126 may be housed in a housing assembly that houses one or more processors 112, memory 114, and / or one or more hardware components that may store one or more software packages and / or cause the execution of one or more software packages. The one or more sensors 126 may include one or more image sensors (e.g., cameras), one or more lidar sensors, one or more audio sensors (e.g., microphones), one or more inertial sensors (e.g., inertial measurement units), one or more biosensors (e.g., heart rate sensors, pulse sensors, retinal sensors, and / or fingerprint sensors), one or more infrared sensors, one or more position sensors (e.g., GPS), one or more touch sensors (e.g., conductive touch sensors and / or mechanical touch sensors), and / or one or more other sensors. One or more sensors may be used to obtain data associated with a user environment (e.g., an image of the user environment, a record of the environment, and / or the user's location).

[0119] The user computing system 102 may include and / or be part of a user computing device 104. The user computing device 104 may include a mobile computing device (e.g., a smartphone or tablet computer), a desktop computer, a laptop computer, a smart wearable device, and / or a smart appliance. Additionally and / or alternatively, the user computing system may obtain data from one or more of the user computing devices 104 and / or generate data using the one or more user computing devices. For example, a smartphone camera may be used to capture image data describing an environment, and / or an overlay application of the user computing device 104 may be used to track and / or process data provided to the user. Similarly, one or more sensors associated with a smart wearable device may be used to obtain data about the user and / or about the user's environment (e.g., a camera housed in a user's smart glasses may be used to obtain image data). Additionally and / or alternatively, data may be obtained and uploaded from other user devices that may be specifically used for data acquisition or generation.

[0120] The server computing system 130 includes one or more processors 132 and memory 134. The one or more processors 132 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be one processor or a plurality of processors operatively connected. The memory 134 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 134 may store data 136 and instructions 138 executed by the processor 132 to cause the server computing system 130 to perform operations.

[0121] In some implementations, the server computing system 130 includes or is otherwise implemented by one or more server computing devices. In instances where the server computing system 130 includes multiple server computing devices, such server computing devices may operate according to a sequential computing architecture, a parallel computing architecture, or some combination thereof.

[0122] As described above, the server computing system 130 may store or otherwise include one or more machine learning models 140. For example, the model 140 may be or may otherwise include a variety of machine learning models. Example machine learning models include neural networks or other multi-layer nonlinear models. Example neural networks include feedforward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks. Fig. 10B Discuss example model 140 .

[0123] Additionally and / or alternatively, server computing system 130 may include and / or be communicatively connected to a search engine 142 that may be used to crawl one or more databases (and / or resources). Search engine 142 may process data from user computing system 102, server computing system 130, and / or third-party computing system 150 to determine one or more search results associated with the input data. Search engine 142 may perform term-based searches, tag-based searches, Boolean-based searches, image searches, embedding-based searches (e.g., nearest neighbor searches), multimodal searches, and / or one or more other search techniques.

[0124] The server computing system 130 may store and / or provide one or more user interfaces 144 for obtaining input data and / or providing output data to one or more users. The one or more user interfaces 144 may include one or more user interface elements, which may include input fields, navigation tools, content snippets, selectable tiles, widgets, data display carousels, dynamic animations, information pop-ups, image enhancements, text-to-speech, speech-to-text, augmented reality, virtual reality, feedback loops, and / or other interface elements.

[0125] User computing system 102 and / or server computing system 130 may train models 120 and / or 140 via interaction with a third-party computing system 150 communicatively coupled via network 180. Third-party computing system 150 may be separate from server computing system 130 or may be a part of server computing system 130. Alternatively and / or in addition, third-party computing system 150 may be associated with one or more web resources, one or more web platforms, one or more other users, and / or one or more contexts.

[0126] The third-party computing system 150 may include one or more processors 152 and a memory 154. The one or more processors 152 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be one processor or a plurality of processors operatively connected. The memory 154 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc., and combinations thereof. The memory 154 may store data 156 and instructions 158 that are executed by the processor 152 to cause the third-party computing system 150 to perform operations. In some implementations, the third-party computing system 150 includes or is otherwise implemented by one or more server computing devices.

[0127] The network 180 may be any type of communication network, such as a local area network (e.g., an intranet), a wide area network (e.g., the Internet), or some combination thereof, and may include any number of wired or wireless links. In general, communications over the network 180 may be performed via any type of wired and / or wireless connection using a variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or protection schemes (e.g., VPN, secure HTTP, SSL).

[0128] The server computing system 130 (and / or the user computing system 102, via one or more entered preferences) can prompt the generation of a model for a particular query and content suggestion task (e.g., via few shot prompting, zero shot prompting, and / or via soft prompting). The prompt may include specifying a set number of queries, article data, post data, and / or historical data to be obtained and processed to perform model inference. In some implementations, the prompt may include specifying "can you recommend some new questions that the user will find interesting?" The prompt may include requesting the generation of a model to generate a "why". The prompt may include requesting the acquisition and processing of user data to perform user-specific customization of event determination and / or query generation.

[0129] The machine learning models described in this specification can be used for a variety of tasks, applications, and / or use cases.

[0130] In some implementations, the input to the machine learning model of the present disclosure may be image data. The machine learning model may process the image data to generate an output. As an example, the machine learning model may process the image data to generate an image recognition output (e.g., recognition of the image data, potential embedding of the image data, encoded representation of the image data, hashing of the image data, etc.). As another example, the machine learning model may process the image data to generate an image segmentation output. As another example, the machine learning model may process the image data to generate an image classification output. As another example, the machine learning model may process the image data to generate an image data modification output (e.g., a change to the image data, etc.). As another example, the machine learning model may process the image data to generate an encoded image data output (e.g., an encoded and / or compressed representation of the image data, etc.). As another example, the machine learning model may process the image data to generate an amplified image data output. As another example, the machine learning model may process the image data to generate a prediction output.

[0131] In some implementations, the input to the machine learning model of the present disclosure may be text or natural language data. The machine learning model may process the text or natural language data to generate an output. As an example, the machine learning model may process the natural language data to generate a language encoding output. As another example, the machine learning model may process the text or natural language data to generate a potential text embedding output. As another example, the machine learning model may process the text or natural language data to generate a translation output. As another example, the machine learning model may process the text or natural language data to generate a classification output. As another example, the machine learning model may process the text or natural language data to generate a text segmentation output. As another example, the machine learning model may process the text or natural language data to generate a semantic intent output. As another example, the machine learning model may process the text or natural language data to generate an amplified text or natural language output (e.g., text or natural language data of higher quality than the input text or natural language, etc.). As another example, the machine learning model may process the text or natural language data to generate a prediction output.

[0132] In some implementations, the input to the machine learning model of the present disclosure may be speech data. The machine learning model may process the speech data to generate an output. As an example, the machine learning model may process the speech data to generate a speech recognition output. As another example, the machine learning model may process the speech data to generate a speech translation output. As another example, the machine learning model may process the speech data to generate a potential embedding output. As another example, the machine learning model may process the speech data to generate an encoded speech output (e.g., an encoded and / or compressed representation of the speech data, etc.). As another example, the machine learning model may process the speech data to generate an amplified speech output (e.g., speech data of higher quality than the input speech data, etc.). As another example, the machine learning model may process the speech data to generate a text representation output (e.g., a text representation of the input speech data, etc.). As another example, the machine learning model may process the speech data to generate a predicted output.

[0133] In some implementations, the input to the machine learning model of the present disclosure may be sensor data. The machine learning model may process the sensor data to generate an output. As an example, the machine learning model may process the sensor data to generate a recognition output. As another example, the machine learning model may process the sensor data to generate a prediction output. As another example, the machine learning model may process the sensor data to generate a classification output. As another example, the machine learning model may process the sensor data to generate a segmentation output. As another example, the machine learning model may process the sensor data to generate a segmentation output. As another example, the machine learning model may process the sensor data to generate a visualization output. As another example, the machine learning model may process the sensor data to generate a diagnostic output. As another example, the machine learning model may process the sensor data to generate a detection output.

[0134] In some cases, the input includes visual data, and the task is a computer vision task. In some cases, the input includes pixel data for one or more images, and the task is an image processing task. For example, the image processing task may be image classification, where the output is a set of scores, each score corresponding to a different object class and representing the possibility that one or more images depict an object belonging to an object class. The image processing task may be object detection, where the image processing output identifies one or more regions in one or more images, and for each region, identifies the possibility that the region depicts an object of interest. As another example, the image processing task may be image segmentation, where the image processing output defines the corresponding possibility for each category in a predetermined set of categories for each pixel in one or more images. For example, the set of categories may be foreground and background. As another example, the set of categories may be an object class. As another example, the image processing task may be depth estimation, where the image processing output defines a corresponding depth value for each pixel in one or more images. As another example, the image processing task may be motion estimation, where the network input includes multiple images, and the image processing output defines the motion of the scene depicted at the pixel between images in the network input for each pixel of one of the input images.

[0135] The user computing system may include multiple applications (e.g., applications 1 to N). Each application may include its own corresponding machine learning library and machine learning model. For example, each application may include a machine learning model. Example applications include text messaging applications, email applications, dictation applications, virtual keyboard applications, browser applications, etc.

[0136] Each application can communicate with multiple other components of the computing device (such as, for example, one or more sensors, a context manager, a device state component, and / or additional components). In some implementations, each application can communicate with each device component using an API (e.g., a public API). In some implementations, the API used by each application is specific to the application.

[0137] The user computing system 102 may include multiple applications (e.g., applications 1 to N). Each application communicates with a central intelligence layer. Example applications include text messaging applications, email applications, dictation applications, virtual keyboard applications, browser applications, etc. In some implementations, each application may communicate with the central intelligence layer (and the models stored therein) using an API (e.g., a public API across all applications).

[0138] The central intelligence layer may include multiple machine learning models. For example, a corresponding machine learning model (e.g., model) may be provided for each application and managed by the central intelligence layer. In other implementations, two or more applications may share a single machine learning model. For example, in some implementations, the central intelligence layer may provide a single model (e.g., a single model) for all applications. In some implementations, the central intelligence layer is included within or otherwise implemented by an operating system of computing system 100.

[0139] The central intelligence layer can communicate with a central device data layer. The central device data layer can be a centralized data repository for the computing system 100. The central device data layer can communicate with many other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, the central device data layer can communicate with each device component using an API (e.g., a private API).

[0140] Fig. 10BA block diagram of an example computing system 50 for performing queries and content recommendations according to an example embodiment of the present disclosure is depicted. Specifically, the example computing system 50 may include one or more computing devices 52, which may be used to obtain and / or generate one or more data sets, which may be processed by a sensor processing system 60 and / or an output determination system 80 for feedback to a user, thereby providing information about features in one or more obtained data sets. The one or more data sets may include image data, text data, audio data, multimodal data, potentially encoded data, etc. The one or more data sets may be obtained via one or more sensors associated with the one or more computing devices 52 (e.g., one or more sensors in the computing devices 52). In addition and / or alternatively, the one or more data sets may be stored data and / or retrieved data (e.g., data retrieved from a web resource). For example, a user may interact with images, text, and / or other content items. Interaction with the content items may then be used to generate one or more determinations.

[0141] One or more computing devices 52 may obtain and / or generate one or more data sets based on image capture, sensor tracking, data storage retrieval, content download (e.g., downloading images or other content items from web resources via the Internet), and / or via one or more other technologies. One or more data sets may be processed using a sensor processing system 60. The sensor processing system 60 may perform one or more processing techniques using one or more machine learning models, one or more search engines, and / or one or more other processing techniques. One or more processing techniques may be performed in any combination and / or individually. One or more processing techniques may be performed serially and / or in parallel. Specifically, one or more data sets may be processed using a context determination block 62, which may determine a context associated with one or more content items. The context determination block 62 may identify and / or process metadata, user profile data (e.g., preferences, user search history, user browsing history, user purchase history, and / or user input data), previous interaction data, global trend data, location data, time data, and / or other data to determine a specific context associated with a user. A context may be associated with an event, a determined trend, a particular action, a particular type of data, a particular environment, and / or another context associated with a user and / or retrieved or obtained data.

[0142] The sensor processing system 60 may include an image pre-processing block 64. The image pre-processing block 64 may be used to adjust one or more values ​​of an acquired and / or received image to prepare the image to be processed by one or more machine learning models and / or one or more search engines 74. The image pre-processing block 64 may resize the image, adjust saturation values, adjust resolution, strip and / or add metadata, and / or perform one or more other operations.

[0143] In some implementations, the sensor processing system 60 may include one or more machine learning models, which may include a detection model 66, a segmentation model 68, a classification model 70, an embedding model 72, and / or one or more other machine learning models. For example, the sensor processing system 60 may include one or more detection models 66 that may be used to detect specific features in a processed dataset. Specifically, one or more detection models 66 may be used to process one or more images to generate one or more bounding boxes associated with detected features in the one or more images.

[0144] Additionally and / or alternatively, one or more segmentation models 68 may be utilized to segment one or more portions of a data set from one or more data sets. For example, one or more segmentation models 68 may utilize one or more segmentation masks (e.g., one or more segmentation masks generated manually and / or based on one or more bounding boxes) to segment a portion of an image, a portion of an audio file, and / or a portion of text. Segmentation may include isolating one or more detected objects and / or removing one or more detected objects from an image.

[0145] One or more classification models 70 may be used to process image data, text data, audio data, potentially encoded data, multimodal data, and / or other data to generate one or more classifications. One or more classification models 70 may include one or more image classification models, one or more object classification models, one or more text classification models, one or more audio classification models, and / or one or more other classification models. One or more classification models 70 may process data to determine one or more classifications.

[0146] In some implementations, one or more embedding models 72 may be used to process data to generate one or more embeddings. For example, one or more embedding models 72 may be used to process one or more images to generate one or more image embeddings in an embedding space. The one or more image embeddings may be associated with one or more image features of the one or more images. In some implementations, the one or more embedding models 72 may be configured to process multimodal data to generate multimodal embeddings. The one or more embeddings may be used to classify, search, and / or learn an embedding space distribution.

[0147] The sensor processing system 60 may include one or more search engines 74 that can be used to perform one or more searches. The one or more search engines 74 can crawl one or more databases (e.g., one or more local databases, one or more global databases, one or more private databases, one or more public databases, one or more dedicated databases, and / or one or more general databases) to determine one or more search results. The one or more search engines 74 can perform feature matching, text-based searches, embedding-based searches (e.g., k-nearest neighbor searches), metadata-based searches, multimodal searches, web resource searches, image searches, text searches, and / or application searches.

[0148] Additionally and / or alternatively, the sensor processing system 60 may include one or more multimodal processing blocks 76 that may be used to assist in processing multimodal data. The one or more multimodal processing blocks 76 may include generating multimodal queries and / or multimodal embeddings for processing by one or more machine learning models and / or one or more search engines 74.

[0149] The output of the sensor processing system 60 may then be processed using an output determination system 80 to determine one or more outputs to be provided to the user. The output determination system 80 may include a heuristic-based determination, a machine learning model-based determination, a user selection-based determination, and / or a context-based determination.

[0150] The output determination system 80 can determine how and / or where to provide one or more search results (and / or query suggestions) in the search results interface 82. In addition and / or alternatively, the output determination system 80 can determine how and / or where to provide one or more machine learning model outputs in the machine learning model output interface 84. In some implementations, one or more search results and / or one or more machine learning model outputs can be provided for display via one or more user interface elements. One or more user interface elements can be superimposed on the displayed data. For example, one or more detection indicators can be superimposed on the detected object in the viewfinder. One or more user interface elements can be selected to perform one or more additional searches and / or one or more additional machine learning model processes. In some implementations, user interface elements can be provided as dedicated user interface elements for specific applications and / or can be uniformly provided across different applications. One or more user interface elements may include pop-up displays, interface overlays, interface tiles and / or small pieces, carousel interfaces, audio feedback, animations, interactive widgets, and / or other user interface elements.

[0151] Additionally and / or alternatively, data associated with the output of the sensor processing system 60 may be utilized to generate and / or provide an augmented reality experience and / or a virtual reality experience 86. For example, one or more acquired data sets may be processed to generate one or more augmented reality rendering assets and / or one or more virtual reality rendering assets, which may then be used to provide an augmented reality experience and / or a virtual reality experience 86 to a user. The augmented reality experience may render information associated with an environment into the corresponding environment. Alternatively and / or additionally, objects associated with the processed data sets may be rendered into the user environment and / or the virtual environment. Rendering data set generation may include training one or more neural radiance field models to learn a three-dimensional representation of one or more objects.

[0152] In some implementations, one or more action prompts 88 may be determined based on the output of the sensor processing system 60. For example, a search prompt, a purchase prompt, a generate prompt, a reservation prompt, a call prompt, a redirect prompt, and / or one or more other prompts may be determined to be associated with the output of the sensor processing system 60. The one or more action prompts 88 may then be provided to the user via one or more selectable user interface elements. In response to selection of the one or more selectable user interface elements, a corresponding action of the corresponding action prompt may be performed (e.g., a search may be performed, a purchase application programming interface may be utilized, and / or another application may be opened).

[0153] In some implementations, one or more generative models 90 may be used to process one or more data sets and / or output of the sensor processing system 60 to generate model-generated content items that may then be provided to a user. The generation may be prompted based on a user selection and / or may be performed automatically (e.g., automatically based on one or more conditions, which may be associated with a threshold amount of unrecognized search results).

[0154] The output determination system 80 may use the data enhancement block 92 to process one or more data sets and / or the output of the sensor processing system 60 to generate enhanced data. For example, the data enhancement block 92 may be used to process one or more images to generate one or more enhanced images. Data enhancement may include data correction, data cropping, removal of one or more features, addition of one or more features, resolution adjustment, lighting adjustment, saturation adjustment, and / or other enhancements.

[0155] In some implementations, one or more data sets and / or outputs of the sensor processing system 60 may be stored based on the determination of the data storage block 94 .

[0156] The output of the output determination system 80 may then be provided to the user via one or more output components of the user computing device 52. For example, one or more user interface elements associated with the one or more outputs may be provided for display via a visual display of the user computing device 52.

[0157] The process may be performed iteratively and / or continuously. One or more user inputs to the provided user interface elements may modulate and / or influence the continuous processing loop.

[0158] The technology discussed herein relates to servers, databases, software applications, and other computer-based systems and the actions taken and the information sent to and from these systems. The inherent flexibility of computer-based systems allows for multiple possible configurations, combinations, and partitioning of tasks and functions between and among components. For example, the processes discussed herein can be implemented using a single device or component or multiple devices or components working in combination. Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0159] Although the subject matter has been described in detail with respect to various specific example embodiments of the subject matter, each example is provided by way of explanation, rather than limitation of the present disclosure. Those skilled in the art, after understanding the foregoing, can easily produce changes, modifications, and equivalents of these embodiments. Therefore, the present disclosure does not exclude such modifications, changes, and / or additions to the subject matter that will be readily understood by those of ordinary skill in the art. For example, a feature shown or described as part of one embodiment can be used together with another embodiment to produce yet another embodiment. Therefore, the present disclosure is intended to cover these changes, changes, and equivalents.

Claims

1. A computer-implemented method for proactive querying and content suggestion, the method comprising: obtaining, by a computing system including one or more processors, web data, wherein the web data describes one or more topics; processing, by the computing system, the web data to determine an occurred change event, wherein the change event comprises an update associated with the one or more topics; processing, by the computing system, data describing the change event using a generative model to generate one or more query suggestions; Obtaining, by the computing system, one or more web resources associated with the change event; processing, by the computing system, the one or more web resources using the generative model to generate a change event summary; as well as The one or more query suggestions and the change event summary are provided by the computing system for display.

2. The method of claim 1, further comprising: obtaining, by the computing system, user data associated with a user; determining, by the computing system based on the user data, that the user is interested in one or more specific topics; as well as Wherein processing, by the computing system, the data describing the change event using the generative model to generate the one or more query suggestions is performed in response to: A determination is made by the computing system that the change event is associated with the one or more specific topics.

3. The system of claim 1, further comprising: determining, by the computing system, that a second change event occurred, wherein the second change event includes an update associated with one or more second topics; processing, by the computing system, data describing the second change event using the generative model to generate one or more second query suggestions; obtaining, by the computing system, one or more second web resources associated with the second change event; processing, by the computing system, the one or more second web resources using the generative model to generate a second change event summary; as well as The one or more second query suggestions and the second change event summary are provided by the computing system for display.

4. A system as claimed in claim 3, wherein the one or more query suggestions and the change event summary are provided as a first pair, wherein the one or more second query suggestions and the second change event summary are provided as a second pair, and wherein the first pair and the second pair are provided in a news push interface.

5. The system of claim 1, further comprising: receiving, by the computing system, a selection of at least one of the one or more query suggestions or the change event summary; as well as A search results interface is provided by the computing system for display, wherein the search results interface includes a plurality of search results responsive to a search query associated with the one or more query suggestions. The system of claim 5 , wherein the plurality of search results include the one or more web resources.

7. The system of claim 1, further comprising: receiving, by the computing system, a selection of at least one of the one or more query suggestions or the change event summary; as well as A model-generated content item is provided by the computing system for display, wherein the model-generated content item is generated using the generative model.

8. The system of claim 7, wherein the model-generated content item includes information associated with the change event, and wherein the information includes details obtained from one or more databases formatted by the generative model into natural language text.

9. The system of claim 1, wherein the web data is obtained prior to receiving a search query.

10. The system of claim 9, wherein the one or more query suggestions and the summary of change events are provided for display prior to receiving a search query.

11. A computing system for active query and content suggestion, the system comprising: one or more processors; as well as one or more non-transitory computer-readable media collectively storing instructions that, when executed by the one or more processors, cause the computing system to perform operations comprising: obtaining a plurality of queries, wherein the plurality of queries are obtained from a plurality of users; processing the plurality of queries to determine a query trend, wherein the query trend describes an increase in a number of queries associated with a particular topic; processing the data describing the query trends using a language model to generate one or more query suggestions; obtaining one or more web resources responsive to the one or more query suggestions; Processing the one or more web resources using the language model to generate a content summary, wherein the content summary describes the content of the one or more web resources; and The one or more query suggestions and the content summary are provided for display.

12. The system of claim 11, wherein the one or more query suggestions are generated in a question format.

13. The system of claim 12, wherein the content summary is generated in an answer format, and wherein the content summary is generated in response to a question of the one or more query suggestions.

14. The system of claim 11, wherein the content summary comprises a natural language summary including one or more details obtained from the one or more web resources.

15. The system of claim 11, wherein processing the data describing the query trends using the language model to generate the one or more query suggestions is performed in response to: It is determined that the particular topic is associated with one or more content items previously viewed by a particular user.

16. The system of claim 11, wherein the query trend is a global query trend.

17. One or more non-transitory computer-readable media collectively storing instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations comprising: obtaining web data, wherein the web data describes content provided by a collection of web resources; processing the web data to determine a plurality of change events that occurred, wherein the plurality of change events include a plurality of updates associated with a plurality of topics; processing the data describing the plurality of change events using a language model to generate a plurality of query suggestions, wherein the language model comprises a generative model, and wherein each query suggestion is associated with a respective change event of the plurality of change events; obtaining a plurality of corresponding web resources associated with the plurality of change events, wherein the plurality of corresponding web resources include one or more corresponding web resources for each of the plurality of change events; associating each of the plurality of query suggestions with a respective web resource of the plurality of respective web resources to generate a query suggestion-web-resource pair for each of the plurality of change events; and A news feed interface is provided for display, wherein the news feed interface includes a set of query suggestion web resource pairs associated with at least a subset of the plurality of change events.

18. The one or more non-transitory computer-readable media of claim 17, wherein each change event in the plurality of change events is determined based on: The influx of articles associated with the corresponding topic is determined.

19. The one or more non-transitory computer-readable media of claim 17, wherein each change event in the plurality of change events is determined based on: Determine the influx of social media posts associated with the respective topic.

20. The one or more non-transitory computer-readable media of claim 17, wherein the language model comprises a machine learning generative model trained to generate natural language data customized to simulate the vocabulary, intonation, and style of a particular user.

Images