Identifying and issuing repeatable queries

Abstract

To improve the ease of use of a search engine.SOLUTION: A set of search queries that have been issued by multiple user devices can be obtained. For each query instance, contextual data can be obtained. A first query and its contextual data can be input to a model that outputs the query's likelihood to be issued in the future. The model can be trained using contextual data for training queries and corresponding labels for the training queries. A learning model outputs the first query's likelihood to be issued in future, and this query is stored as a repeatable query if the likelihood satisfies a repeatability threshold. Subsequently, a stored repeatable query is issued upon a selection of a user selectable interface component and the search engine provides search results for the query.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] This specification relates to the identification and issuance of repeatable search queries (also referred to herein as "repeatable queries"), i.e., search queries that have been previously issued and are expected to be issued in the future by one or more user devices. [Background technology]

[0002] A user of a user device submits search queries to a search engine (or another search application) seeking information on different topics, and in response, the search engine (or another search application) provides search results that may be responsive to those topics. In some instances, a user may periodically submit the same search queries previously submitted by that user or by other users. For example, a user may periodically issue a search query such as "famous quotes." In this example, each time this search query is received by the search engine, the search engine searches and identifies a set of search results that are responsive to this query. In some instances, for such repeated queries, the search engine's processing results in identifying and providing the same set of search results. Summary of the Invention [Means for solving the problem]

[0003] In general, one inventive aspect of the subject matter described herein may be embodied in a method, the method including operations of obtaining a set of search queries issued by a plurality of user devices; determining, for each instance of each query in the set of search queries, context data representing a context in which the query was issued and a user interaction, the user interaction being a user interaction with a search result page provided in response to the query; and, for a first query in the set of search queries, inputting the determined context data for each instance of the query in which the first query was issued into a learning model, the learning model (1) outputs a likelihood that the search query will be issued in the future; and (2) is trained using the determined context data for the set of training queries and a set of labels, each of the labels in the set of labels corresponding to one of the labels in the set of training queries, each label being determined based on whether the training query has been issued a threshold number of times. obtaining from the learning model a likelihood that the first query will be issued in the future based on the input context data; identifying the first query as a repeatable query based on the likelihood that the first query will be issued in the future satisfying a repeatability threshold; storing the first query along with other search queries identified as repeatable queries; providing on the user device a user selectable interface component that, when selected by the user device, results in issuance of a query from among the repeatable queries without receiving user input of a component of the query, the issuance of the query being based on a user selection of a selectable shortcut link included in the user selectable interface component; receiving from the user device a first selection of the user selectable interface component requesting issuance of a particular query from among the repeatable queries; and in response to receiving the first selection from the user device,and providing a first search result page including the search results for the particular query. Other embodiments of this aspect include computer programs configured to perform the actions of the corresponding systems, devices, apparatus, and methods. The computer programs (e.g., instructions) may be encoded on a computer storage device. These and other embodiments may each optionally include one or more of the following features:

[0004] In some implementations, storing the first query along with other search queries identified as repeatable queries may include storing, by the search engine, the first query along with search results responsive to the first query.

[0005] In some implementations, the search engine can include one or more of the stored search results in a first search result page.

[0006] In some implementations, the contextual data for each instance of each query in the set of search queries may include one or more of the language in which the query is written, the geographic location from which the query is issued, geographic locations of interest to the user device that issued the query, the number of times the query was issued by a particular user device, the number of unique user devices that issued the query a threshold number of times, query embeddings that express semantic relationships between the query and other queries, a determination as to whether the query is directed to a particular web location or website, a selection by the user device of one or more search results provided on a search results page for the query, a time spent by the user device viewing one or more search results provided on the search results page for the query, and a selection of navigation interface elements on the search results page provided for the query.

[0007] In some implementations, the user selectable interface component may be a selectable shortcut link included at a particular location on the user device, where selection of the selectable shortcut link by the user device causes a particular query associated with the shortcut link to be issued to a search engine.

[0008] In some implementations, the user-selectable interface component may be a drop-down menu that lists a subset of repeatable queries from which the user can select a particular query, the selection of a particular query causing the particular query to be submitted to the search engine.

[0009] In some implementations, the subset of repeatable queries may include queries that are expected to be repeated by the user device based on a search history that includes queries previously issued by the user device.

[0010] In some implementations, the method may include an operation of determining, in response to identifying the first query as a repeatable query, that the user device is expected to issue the first query in the future based on a search history for the user device, and an operation of providing, in a notification to the user device, a set of search results for the first query without receiving any request from the user device to perform the first query, in response to determining that the user device is expected to issue the first query in the future.

[0011] In some implementations, the method may include an act of receiving, from the user device, a second selection of a user-selectable interface component requesting that a particular query be issued, and an act of providing, by the search engine, in response to receiving the second selection from the user device, a second search results page including search results for the particular query, where the search results page is different from the first search results page.

[0012] In some implementations, the second search results page may differ from the first search results page when the search results on the second result page are ordered differently than the search results on the first search results page, when the search results on the second result page differ from the search results on the first search results page, or when the second result page includes dynamic content that is not included on the first result page.

[0013] Certain embodiments of the subject matter described herein may be implemented to achieve one or more of the following advantages. For example, the innovations described herein may conserve computing resources used in providing search results in response to an issued search query. Traditionally, when a user writes and submits a search query to a search engine, the search engine searches and identifies search results responsive to the search query. In contrast, the innovations described herein may reduce computing resources required to search and identify search results for some identified search queries. For example, the innovations described herein may identify a set of search queries as repeatable queries and store responsive search results for these queries. Later, when one of the repeatable queries is issued by the user device, the search engine may bypass the search engine's processing and provide responsive search results for the previously stored, repeated query.

[0014] The innovations described herein can also save time and resources typically required in manually issuing search queries. Traditionally, a user manually types each search query into an interface provided by a search engine (e.g., a text box provided on the search engine's website) and then selects an interface component (e.g., a "Go" or "Search" button) that causes the search query to be issued to the search engine. In contrast, the innovations described herein provide user-selectable interface components (e.g., a drop-down menu with repeatable queries, a device home screen quick launch icon that issues a particular repeatable query directly to the search engine) that allow selection of an already-formed search query that causes the selected user query to be issued to the search engine. In some implementations, search results for search queries that the user is expected to issue in the future are provided in a notification to the user device, without user selection being required. As a result, user resources and processing by interface functions on the search engine's website (or other application) required in entering and processing search queries typed by the user may be saved.

[0015] The innovations described herein can also conserve network resources by fetching search results for frequently repeated queries in the background when a user's device is on a local (e.g., home or work) wireless network. Relatedly, when a user device is determined to be in an unreliable network (e.g., a remote location with poor cellular LTE service when the user device is not connected to the network for a threshold amount of time over a time interval), search results for repeated queries may be stored on the user device for longer and therefore more likely to be available to the user when the user's device is offline or otherwise only intermittently connected to the network.

[0016] The innovations described herein can also improve the usability of search engines by allowing users to issue several repeatable queries without having to enter any portion of such queries (i.e., any text that is part of the query). Relatedly, the innovations described herein can improve and / or increase user engagement by providing newer or different search results (or other dynamic content) in response to search queries issued (or anticipated to be issued) periodically by users.

[0017] The details of one or more embodiments of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims. [Brief description of the drawings]

[0018] [Figure 1] 1 is a block diagram of an example environment in which a search engine provides search results in response to a search query issued by a user device. [Diagram 2] 2 is a block diagram illustrating the structure and operation of the search engine of FIG. 1 in identifying and enabling issuance of repeatable queries. [Diagram 3] FIG. 1 is a flow diagram of an example process for identifying and issuing repeatable queries. [Figure 4] FIG. 1 is a block diagram of an exemplary computer system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Like reference numbers and designations in the various drawings indicate like elements.

[0020] This specification relates to identifying and issuing repeatable search queries. As summarized below and described in more detail throughout this specification, a search engine may include a repeatable query manager that identifies a set of repeatable queries and enables a user device to quickly select and issue one or more of these queries.

[0021] To accomplish this, the repeatable query manager of the search engine first obtains a set of search queries issued by different user devices, e.g., from the search engine's query log. For each instance of each query in the set of search queries, the repeatable query manager determines context data, the context data representing the context in which the query was issued and user interactions, the user interactions being user interactions with a search result page provided in response to the query. Examples of such context data may include the language in which the query is written, the geographic location from which the query is issued, the geographic locations of interest to the user device that issued the query, the number of times the query has been issued generally or by a particular user device, the number of unique user devices that have issued the query a threshold number of times, embeddings in the query that represent semantic relationships between the query and other queries, a determination as to whether the query is directed to a particular web location or website, a selection by the user device of one or more results provided on a search result page for the query, a time spent by the user device viewing one or more results provided on a search result page for the query, and a selection of navigation interface elements (e.g., a back button, a next page button, etc.) on a search result page provided for the query.

[0022] To determine whether a first query from the set of queries is a repeatable query, the repeatable query manager first inputs the query and corresponding context data for each instance of the query into a learning model. The learning model outputs the likelihood that the query will be issued in the future. The learning model is trained using the context data determined for the set of training queries and a set of labels, each of which corresponds to one of the labels in the set of training queries, and each label indicates whether the training query has been issued a threshold number of times. Each label indicates whether the query has been observed to be repeated or otherwise repeatable. Based on the data input to the learning model, the learning model outputs the likelihood that the first query will be issued in the future.

[0023] The repeatable query manager then compares the likelihood output by the learning model to a repeatability threshold. If the likelihood meets (e.g., satisfies or exceeds) the repeatability threshold, the repeatable query manager identifies the query as repeatable, otherwise, identifies the query as not repeatable. If the first query is identified as repeatable, the repeatable query manager stores the first query along with other search queries identified as repeatable queries.

[0024] When a user accesses an interface provided by the search engine (e.g., a landing web page for the search engine's website), the repeatable query manager provides user-selectable interface components that, when selected by the user device, result in the issuance of a query from among the repeatable queries without receiving user input of a query component. As used herein, a query component is text entered by a user of the user device, e.g., using a keyboard, in making a search query. For example, in the query "John Doe Sighting," a query component can be the letter "J," the word "John Doe," or the entire query.

[0025] In some implementations, the user-selectable interface component may be a drop-down menu that lists a subset of previously identified repeatable queries. Selection by a user device of any repeatable query listed in such a drop-down menu causes the user device to submit that query to a search engine. In some implementations, the user-selectable interface component may also be a shortcut link included in a particular user device location (e.g., a home screen of the user device). Selection by the user device of the shortcut link results in launching an application (e.g., a web browser) and submitting the repeatable search query associated with the shortcut link to the search engine.

[0026] The search engine receives from the user device a selection of a user-selectable interface component requesting submission of a particular query from among the repeatable queries, and in response to receiving the user device's selection, the search engine provides a search result page including search results for the submitted query.

[0027] These and additional features are described in more detail below with reference to FIGS.

[0028] In addition to the descriptions throughout this specification, a user may be provided with controls that allow the user to make choices regarding both whether and when the systems, programs, or features described herein may enable collection of user information (e.g., information about the user's social network, social actions or activities, occupation, user preferences, or the user's current location) and whether content or communications from the server are sent to the user. In addition, some data may be treated in one or more ways before being stored or used such that personally identifiable information is removed. For example, the user's identification information may be treated such that personally identifiable information cannot be determined about the user, or the user's geographic location may be generalized, in which case location information is obtained (such as to the city, zip code, or state level) such that the user's specific location cannot be determined. Thus, the user has control over what information is collected about the user, how that information is used, and what information is provided to the user.

[0029] FIG. 1 is a block diagram of an example environment 100 in which a search engine provides search results in response to search queries issued by user devices.

[0030] A computer network 102, such as a local area network (LAN), a wide area network (WAN), the Internet, or a combination thereof, connects publisher websites 104, user devices 106, and search engines 108. The environment 100 may include multiple publisher websites 104 and user devices 106.

[0031] A website 104 includes one or more resources associated with a domain name and hosted by one or more servers. An exemplary website is a collection of web pages that may be formatted in HyperText Markup Language (HTML), which may include text, images, multimedia content, and programming elements such as scripts. Each website 104 is typically maintained by a content publisher, which is an entity that controls, manages, and / or owns the website 104.

[0032] A resource is any data that may be provided over the network 102 by a publisher 104 and that is associated with a resource address. Resources include HTML pages, word processing documents, and portable document format (PDF) documents, images, videos, and feed sources, to name just a few. Resources may include content such as words, phrases, pictures, and the like, and may include embedded information (such as meta-information and hyperlinks) and / or embedded instructions (such as scripts).

[0033] User devices 106 are electronic devices capable of requesting and receiving resources over the network 102. Exemplary user devices 106 include personal computers, mobile communication devices, and other devices capable of sending and receiving data over the network 102. User devices 106 typically include user applications, such as a web browser, to facilitate sending and receiving data over the network 102. A web browser can enable a user to view and interact with text, images, video, music, and other information that are typically located on web pages at websites on the World Wide Web or a local area network.

[0034] To facilitate searching for these resources, search engine 108 typically identifies resources by crawling publisher websites 104 and indexing the resources provided by publisher websites 104. Indexed, and possibly cached, copies of the resources are stored in index 110.

[0035] A user device 106 submits a search query to the search engine 108. The search query is submitted in the form of a search request that includes the search request and, possibly, a unique identifier that identifies the user device 106 submitting the request. The unique identifier may be data from a cookie stored on the user device, or a user account identifier if the user maintains an account with the search engine 108, or some other identifier that identifies the user device 106 or the user using the user device 106.

[0036] In response to a search request, the search engine 108 can use the index 110 to identify resources relevant to the query. The search engine 108 identifies the resources in the form of search results and returns the search results to the user device 106 in a search result page resource. Search results are data generated by the search engine 108 that identify resources that satisfy a particular search query and include resource locators for the resources. Exemplary search results may include web page titles, snippets of text extracted from the web page, and URLs of the web pages.

[0037] The search results may be ranked based on scores relating to the resources identified by the search results, such as information retrieval ("IR") scores, and possibly a separate ranking of each resource relative to other resources (e.g., authority scores). The search results may be ordered according to these scores and provided to the user device 106 in that order.

[0038] The user device 106 receives the search result page and renders the page for presentation to the user. In response to a user's selection of a search result on the user device 106, the user device 106 requests a resource identified by a resource locator included in the selected search result. A publisher of the website 104 that hosts the resource receives the request for the resource from the user device 106 and provides the resource to the requesting user device 106.

[0039] In some implementations, queries submitted from the user device 106 are stored in the query log 112. Click data for the query, the web pages referenced by the search results, and / or other interactions of the user with the query on the search result page (e.g., time spent viewing the search results, selecting a particular search result, selecting different navigation elements such as a back button or a next button) are stored in the click log 114. The query log 112 and the click log 114 define search history data 116, including data from and related to previous search requests associated with a unique identifier of the user device 106. The click log defines actions taken in response to search results provided by the search engine 108. The query log 112 and the click log 114 can be used to map queries submitted by the user device to web pages identified in the search results and actions taken by the user (i.e., the data is associated with identifiers from the search requests so that the search history for each identifier can be accessed). Thus, the click log 114 and the query log 112 can be used by search engines to determine the sequence of queries submitted by user devices, the actions taken in response to the queries, and how frequently the queries are submitted.

[0040] The search engine 108 also includes a repeatable query manager 118, which identifies repeatable queries, i.e., search queries that have been previously issued and are expected to be repeated by one or more users in the future. The structure and operation of the repeatable query manager 118 in identifying repeatable queries and making these queries available for submission by users of the search engine will be further described with reference to Figures 2 and 3.

[0041] FIG. 2 is a block diagram 200 illustrating the structure and operation of the search engine of FIG. 1 in identifying and enabling issuance of repeatable queries.

[0042] The repeatable query manager 118 of the search engine 108 identifies repeatable queries using search queries previously issued to the search engine 108. The repeatable query manager 118 of the search engine 108 includes a context analyzer 202, a learning model 204, and a repeatable query cache 206. The operation of each of these components is summarized below, but will be described in more detail with reference to FIG. 3. Although the repeatable query manager 118 is shown in FIG. 2 as part of the search engine 108, in some implementations the repeatable query manager 118 may be, for example, a web component that is separate (executes instructions on a separate process) from the search engine 108. In such implementations, the repeatable query manager 118 communicates with the search engine 108 over a network. Similarly, in some implementations the components of the repeatable query manager 118 may be, for example, separate web components that communicate with each other and / or with the search engine 108 over a network.

[0043] The context analyzer 202 determines context data for previously issued search queries, for example from the query log 112 and the click log 114. The learning model 204 determines the likelihood that a particular query will be issued in the future by the user device 106. The learning model 204 receives as input a particular query and context data for that query, and outputs the likelihood that the query will be issued in the future by the user device 106. The learning model 204 may be a supervised machine learning model, a rules engine, or another suitable statistical model. In an implementation in which the learning model 204 is implemented as a supervised machine learning model, the learning model 204 is trained using a set of training search queries and their context data, as well as a corresponding set of labels that identify whether the training queries will be repeated in the future. In particular, each label indicates whether the query has been observed to be repeated or otherwise repeatable.

[0044] In response to the output of the learning model 204, which specifies the likelihood that a query will be repeated in the future, the repeatable query manager 118 may identify the input query as a repeatable query by comparing this likelihood to a repeatability threshold (as further described with reference to FIG. 3). If the repeatable query manager 118 identifies the query as a repeatable query, the repeatable query manager 118 stores the query in the repeatable query cache 206 along with other queries identified as repeatable queries.

[0045] In some implementations, the repeatable query manager 118 makes a subset of the repeatable queries stored in the repeatable query cache 206 available on a user interface provided by the search engine 108. This is illustrated in the user device 106-B shown in FIG. 2, where the user device 106-B includes a repeatable query selector 216. The repeatable query selector 216 can be a drop-down menu (or another suitable user interface element capable of displaying a selectable list of items) that lists a subset of the repeatable queries. Selection of any query from the drop-down by the user, followed by selection of the search button 212, results in the query being issued to the search engine 108. In some implementations, selection of any query from the repeatable query selector 216 results in automatic issuance of a search query to the search engine. In other words, in such implementations, the user of the user device 106-B does not need to separately select the search button 212 to issue a search query. Such issuance of a repeatable query does not require the user device to enter any query text in the search box 214.

[0046] In some implementations, the repeatable query manager 118 can provide an option, such as a “Save as Shortcut” button 218, on a user interface provided by the search engine 108 to save one or more repeatable queries as shortcut links on a home screen of the user device. In such implementations, the user device 106-B can select a particular query from the repeatable query selector 216 and then select the “Save as Shortcut” 218 button. This causes the query to be saved, for example, as an icon (repeat query icon 210) on the home screen of the user device (or another device location configured by the user of the user device) as shown on the user device 106-A. Alternatively or additionally, the repeatable query manager 118 can provide a “Save as Shortcut” 218 button on a search result page, the selection of which also causes the generation of the repeat query icon 210 on the home screen of the user device (or other device location selected by the user). Subsequent selection by the user of the icon 210 to repeat the query results in the launch of an application (eg, a web browser) that launches the search engine 108 and issues the repeatable query.

[0047] In some implementations, the repeatable query manager 118 stores the repeatable queries, as well as the search results corresponding to those queries, in a repeatable query cache 206. In other implementations, the search results corresponding to the repeatable queries are stored in a cache of the index 110. In either implementation, the search engine 108 retrieves and serves the stored search results in response to issuance of a repeatable query by a user of the user device 106.

[0048] The above-described operations of the repeatable query manager 118, as well as additional operations thereof, are further described with reference to FIG.

[0049] FIG. 3 is a flow diagram of an exemplary process 300 for identifying and issuing repeatable queries. The operations of the process 300 are described below as being performed by components of the system described and illustrated in FIG. 1 and FIG. 2. The operations of the process 300 are described below for illustrative purposes only. The operations of the process 300 may be performed by any suitable device or system, for example, any suitable data processing device. The operations of the process 300 may also be implemented as instructions stored on a computer-readable medium (transient or non-transient). Execution of the instructions causes one or more data processing devices to perform the operations of the process 300. As described above with reference to FIG. 2, the repeatable query manager 118 (or any components thereof) may be, but need not be, part of the search engine 108.

[0050] The repeatable query manager 118 obtains (at 302) a set of search queries. In some implementations, the repeatable query manager 118 obtains, from the query log 112, a set of search queries previously issued by one or more user devices.

[0051] The repeatable query manager 118 determines (at 304) context data for each instance of each query in the set of search queries. In some implementations, for each instance of each query in the retrieved set of search queries, the context analyzer 202 determines context data, where the context data represents the context in which the query was issued and the user interaction, or the context data represents the context in which the query was issued or the user interaction, where the user interaction is with a search result page provided in response to the query. The context data may be stored in and retrieved directly from the query log 112 and / or the click log 114. The context data may also be retrieved by processing data stored in the query log 112 and / or the click log 114.

[0052] The following paragraphs provide examples of the types of context data that the context analyzer 202 can obtain for each query instance. The examples below are illustrative and not an exclusive list of the types of context data that can be determined.

[0053] An example of context data includes the language in which a query is written / entered. If the query log 112 includes a "query language" field (or other similar field), the context analyzer 202 can obtain the language value or identifier stored in this field. Alternatively, the context analyzer 202 can enter the text of the query into a language identifier that identifies the language of the entered query text.

[0054] Another example of context data includes the geographic location from which a query is issued. The geographic location may include, for example, a country, city, state, or other suitable location identifier for the origin of the query. The query log 112 typically includes such geographic location of the query, which may then be obtained by the context analyzer 202.

[0055] Another example of contextual data includes a geographic location (e.g., a country, city, state, or other suitable location identifier) ​​of interest to the user device 106 that issued the query. In some implementations, this geographic location may be determined using a search history or content viewing history of the user device, which may be obtained, for example, from a query log 112, a click log 114, or from other sources, such as a profile of the user device for the search engine 108.

[0056] Another example of context data includes the number of times a query has been issued in general and / or by one or more user devices. The context analyzer 202 can obtain this information by parsing the query log 112 to count the number of times a particular query has been issued, as well as the number of times a particular query has been issued by different (i.e., unique) user devices.

[0057] Another example of context data includes the number of unique user devices that issued a query a threshold number of times. In some implementations, the context analyzer 202 parses the query log 112 to count the number of times a particular query was issued by different (i.e., unique) user devices. The context analyzer 202 then compares this count to a threshold (e.g., 2, 3, or another value) to determine whether the number of user devices that issued a query did so the threshold number of times.

[0058] Another example of context data includes determining embeddings of a query that represent semantic relationships between the query and other queries. Such embeddings may be implemented using neural network-based techniques for natural language processing (NLP), which may include, for example, Bidirectional Encoder Representations from Transformers.

[0059] Another example of context data includes determining whether a query is targeted to a particular web location or website. In some implementations, the context analyzer 202 can compare the text of the query to, for example, a list of known websites / web locations, and if a match is found, the context analyzer 202 determines that the particular query is targeted to the identified web location or website. In some implementations, determining whether the query is targeted to a particular web location includes (1) determining whether a site link appears on the search results for the query, and (2) determining, for example, a percentage of user interactions for the highest returned result in the set of search results that indicates the likelihood that a majority of users are interested in that particular result.

[0060] Another example of contextual data includes interactions by a user device on a search result page. Such interactions are stored in a click log 114, which may include, for example, among other things, a selection by the user device 106 of one or more search results provided on the search result page for the query, a time of viewing by the user device of one or more results provided on the search result page for the query, a selection of or interaction with one or more navigational interface elements (e.g., a next button to navigate to the next set of results, a back button to return to the original search page, entering text into a search box to modify a query, and / or pressing a search button (or similar interface component) to issue another search).

[0061] The repeatable query manager 118 inputs (at 306) the context data for the first query into the learning model 204. In some implementations, the context analyzer 202 inserts the first query and one or more of the context data items for each instance of the query into the learning model. As described with reference to FIG. 2, the learning model may be a supervised machine learning model that (1) outputs a likelihood that the query will be issued in the future and (2) is trained using the determined context data for a set of training queries and a set of labels, each of which corresponds to a respective label included in the set of training queries, where each label indicates whether the training query has been issued a threshold number of times (2, 3, or any other selected value). In particular, each label indicates whether the query has been observed to be repeated or otherwise repeatable. In some implementations, the training queries include only queries that have been repeated within a period of time (e.g., 28 days, 1 month, etc.). Such techniques avoid training the model with queries that are repeated only after a long time interval and thus are not queries that users are likely to repeat in shorter intervals. Furthermore, in some implementations, an additional model (e.g., a supervised machine learning model, a rules engine, or another statistical model) may be used with the learning model to determine the specific time at which a query is expected to be repeated. For example, this additional model may be trained using known repeated queries and the time at which such queries are issued. Thus, queries identified by the learning model as repeated queries may be input into this additional model, which in turn identifies the time at which the query is expected to be repeated. For example, if the additional model is trained using search queries (e.g., search queries for "weather") that are issued 70% of the time in the morning, the additional model may predict that this query is expected to be repeated in the morning.

[0062] The repeatable query manager 118 obtains (at 308) a likelihood that the first query will be issued in the future by the user device from the learning model 204. In some implementations, based on the input first query and the corresponding context data, the learning model 204 outputs a likelihood that the first query will be issued in the future by the user device.

[0063] The repeatable query manager obtains (at 310) this output and uses it to determine whether the first query is a repeatable query. In some implementations, the repeatable query manager 118 determines whether the first query is repeatable by comparing the likelihood that the first query will be issued in the future to a repeatability threshold. The repeatability threshold may be a value specified by the search engine 108 operator that is intended to indicate a high likelihood of query repeatability. If the likelihood meets (e.g., satisfies or exceeds) the repeatability threshold, the repeatable query manager 118 identifies the first query as a repeatable query. On the other hand, if the likelihood does not meet the repeatability threshold, the repeatable query manager 118 does not identify the first query as a repeatable query.

[0064] If the first query is identified as a repeatable query, the repeatable query manager 118 stores (at 312) the first query along with other search queries that were identified as repeatable queries. In some implementations, the repeatable query manager 118 stores the repeatable query in the repeatable query cache 206. In such implementations, the repeatable query manager 118 stores the first query along with the other repeatable queries in the repeatable query cache 206.

[0065] The repeatable query manager 118 provides, or causes to be provided, on the user device a user-selectable interface component that, when selected by the user device (at 314), results in the issuance of a query from among the repeatable queries without receiving user input of the query components. In some implementations, when a user navigates to an interface provided by a search engine (e.g., the search engine's website), the repeatable query manager 118 provides a user-selectable interface component on that interface. This user-selectable interface component may be a drop-down menu or another selectable interface component that can list a set of repeatable search queries (stored in the repeatable query cache 206) and allow the user to select one (or more) of the listed search queries.

[0066] The repeatable query manager 118 can provide, or cause to be provided, for viewing on a selectable user interface, a subset of the repeatable queries that may be listed and made available in the user-selectable interface components. The listing of the repeatable search queries included in the user-selectable interface components is provided by the repeatable query manager 118 without receiving any user input of components of the query (also referred to herein simply as "search query components" or "query components"). In other words, the listing of the repeatable search queries provided by the repeatable query manager 118 is independent of (or any part of) the query entered by the user.

[0067] The repeatable query manager 118 can select (and provide for display in a user-selectable interface component) a subset of the repeatable queries in different ways. For example, the repeatable query manager 118 can select a random set of N queries listed in the repeatable query cache 206. As another example, the repeatable query manager 118 can select a set of N queries from the repeatable query cache 206 that are most frequently issued by different user devices. As another example, the repeatable query manager 118 can select a set of N queries from the repeatable query cache 206 that are most frequently issued by a particular user device that is currently accessing an interface provided by a search engine. As another example, the repeatable query manager 118 can select a set of N repeatable queries from the repeatable query cache 206 based on the search history of the user device and / or the type of content accessed by the user device. This may include correlations (e.g., using rule-based engines, statistical modeling techniques such as machine learning algorithms, decision trees, etc.) between the content of search queries issued by the user device, search results selected by the user device, and / or types of content accessed by the user device and the repeatable queries stored in the repeatable query cache 206. The top N search queries identified by such correlations may then be provided by the repeatable query manager 118 for presentation in a user-selectable interface component.

[0068] Additionally or alternatively, the user-selectable interface component may also be a selectable shortcut link included at a particular location on the user device, for example, at the home screen of the user device. As described in FIG. 1, this selectable shortcut link may be generated by the repeatable query manager 118 from an initial interface provided by the search engine (e.g., a landing web page reached when initially navigating to the search engine's website) or from a search result page. Selection of the selectable shortcut link by the user causes the particular repeatable query associated with the shortcut link to be issued to the search engine 108.

[0069] The repeatable query manager 118 receives (at 316) from a user device a selection of a user-selectable interface component requesting issuance of a particular query from among the repeatable queries. In some implementations, a user of the user device 106 selects one of the repeatable queries listed in a user-selectable interface component (e.g., a drop-down menu). The selection of a particular repeatable query by the user device is transmitted to the search engine 108. In some implementations, a user of the user device 106 selects a shortcut link associated with a particular repeatable query.

[0070] In response to receiving a selection from the user device 106, the search engine 108 provides (at 318) a search results page including search results for the particular query. Upon selection of the user device, the search engine 108 executes the query and provides a search results page including search results responsive to the selected query.

[0071] In some implementations, the repeatable query manager 118 can provide the same set of search results each time a user device issues a particular repeatable query. In such implementations, the repeatable query manager 118 can store this set of search results along with the repeatable query in the repeatable query cache 206 (or alternatively, in the index 110).

[0072] In some implementations, the repeatable query manager 118 can provide a different search result page each time the user device issues a particular repeatable query. In such implementations, the repeatable query manager 118 maintains a record of each time the user device issues a particular repeatable query, and the corresponding search result page and the set of search results provided on that result page (e.g., including the order in which the search results are provided). When the user device issues a particular repeatable query for the first time, the search engine 108 provides a first search result page including a first set of search results. In this example, when the user device issues a particular repeatable query for a second time, the repeatable query manager 118 determines that the user device is issuing this query for a second time, and thus causes the search engine to provide a second search result page including a second set of search results that differs from the first search result page. For example, the second search result page may differ from the first search result page in that the search results on the second search result page are ordered differently than the search results on the first search result page. In other words, the first and second sets of search results may include the same search results, but may be ordered differently. As another example, one or more search results in the second set of search results may differ from the search results in the first set of search results. As another example, the second search results page may differ from the first search results page in that the second search results page may include dynamic content that is either not included on the first search results page or different from the dynamic content included on the first search results page. In such an implementation, the search engine 108 may prioritize indexing of sites connected to repeatable queries each time a query is issued in anticipation that users will desire dynamic (e.g., different or newer) content for such queries. By prioritizing content for indexing, the techniques described herein have the advantage of conserving search engine 108 resources required for indexing.

[0073] As described above, the repeatable query manager 118 can identify the first query as a repeatable query based on the search history or content access history of the user device. In some implementations, the search engine 108 can send a notification containing search results for a particular query to the user device without receiving a selection of any of the user-selectable interface components or other request by the user device to issue that particular query. For example, the search engine 108 can identify a particular query (e.g., in a message indicating that a user of the device may be interested in results for a particular repeatable query) and send a notification (e.g., as a device notification or email) to the user device 106 containing a set of search results for that query. In some implementations, the notification containing the search results is sent to the user device 106 at a time when the user is expected to issue the query. This can be determined in part based on the content of the query itself, which can include some time element that can be used to determine the time for delivery of the results. For example, if a user is expected to repeat the query "concerts this weekend," the presence of "weekend" can be used to infer that search results for this query should be delivered in the notification every week, during the week, and ahead of the weekend. Additional factors that may be used in determining when a user is expected to issue a query may also include data about user interaction with the search results in the notification (e.g., time accessed, date accessed, which result was selected, etc.).

[0074] 4 is a block diagram of an exemplary computer system 400 that may be used to perform the operations described above. The system 400 includes a processor 410, a memory 420, a storage device 430, and an input / output device 440. Each of the components 410, 420, 430, and 440 may be interconnected using, for example, a system bus 450. The processor 410 is capable of processing instructions for execution within the system 400. In some implementations, the processor 410 is a single-threaded processor. In another implementation, the processor 410 is a multi-threaded processor. The processor 410 is capable of processing instructions stored in the memory 420 or on the storage device 430.

[0075] The memory 420 stores information within the system 400. In one implementation, the memory 420 is a computer-readable medium. In some implementations, the memory 420 is a volatile memory unit. In another implementation, the memory 420 is a non-volatile memory unit.

[0076] The storage device 430 is capable of providing mass storage for the system 400. In some implementations, the storage device 430 is a computer-readable medium. In various different implementations, the storage device 430 may include, for example, a hard disk device, an optical disk device, a storage device shared over a network by multiple computing devices (e.g., a cloud storage device), or some other mass storage device.

[0077] The input / output device 440 provides input / output operations for the system 400. In some implementations, the input / output device 440 may include one or more of a network interface device, e.g., an Ethernet card, a serial communication device, e.g., and an RS-232 port, and / or a wireless interface device, e.g., and an 802.11 card. In another implementation, the input / output device may include a driver device configured to receive input data and send output data to other input / output devices, e.g., a keyboard, a printer, and a display device 460. However, other implementations, such as mobile computing devices, mobile communication devices, set-top boxes, television client devices, etc., may also be used.

[0078] Although an exemplary processing system has been described in FIG. 4, implementations of the subject matter and functional operations described herein can be implemented in other types of digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed herein and their structural equivalents, or in combinations of one or more of them.

[0079] The subject matter and embodiments of the operations described herein may be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed herein and their structural equivalents, or in any combination of one or more of them. The subject matter described herein may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded on (one or more) computer storage media for execution by or to control the operation of a data processing device. Alternatively or additionally, the program instructions may be encoded on an artificially generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, generated to encode information for transmission to a suitable receiver device for execution by a data processing device. The computer storage medium may be or may be included in a computer readable storage device, a computer readable storage substrate, a random or serial access memory array or device, or any combination of one or more of them. Furthermore, the computer storage medium is not a propagating signal, and the computer storage medium may be a source or destination of computer program instructions encoded in an artificially generated propagating signal. A computer storage medium may also be, or be included in, one or more separate physical components or media (eg, multiple CDs, disks, or other storage devices).

[0080] The operations described herein may be implemented as operations performed by a data processing apparatus on data stored on one or more computer-readable storage devices or received from other sources.

[0081] The term "data processing apparatus" encompasses all kinds of apparatus, devices, and machines for processing data, including, by way of example, a programmable processor, a computer, a system on a chip, or a plurality or combination of the above. The apparatus may include special purpose logic circuitry, e.g., a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). In addition to hardware, the apparatus may also include code that creates an execution environment for the computer program in question, e.g., code constituting a processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination of one or more of these. The apparatus and execution environment may implement a variety of different computing model infrastructures, such as web services, distributed computing, and grid computing infrastructures.

[0082] A computer program (also known as a program, software, software application, script, or code) may be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program may correspond to a file in a file system, but need not. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple cooperating files (e.g., files that store one or more modules, subprograms, or portions of code). A computer program may be deployed to be executed on one computer or on multiple computers located at one site or distributed across multiple sites and interconnected by a communication network.

[0083] The processes and logic flows described herein may be performed by one or more programmable processors executing one or more computer programs to perform actions by operating on input data and generating output. The processes and logic flows may also be performed by, and an apparatus may be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0084] Processors suitable for executing computer programs include, by way of example, both general-purpose and special-purpose microprocessors. Generally, a processor will receive instructions and data from a read-only memory or a random access memory, or both. The essential elements of a computer are a processor for performing actions according to instructions, and one or more memory devices for storing instructions and data. Generally, a computer will also include one or more mass storage devices, such as magnetic disks, magneto-optical disks, or optical disks, for storing data, or be operatively coupled to receive data therefrom or transfer data thereto, or both. However, a computer need not have such devices. Furthermore, a computer may be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device (e.g., a universal serial bus (USB) flash drive), to name just a few. Suitable devices for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including, by way of example, semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices, magnetic disks, e.g., internal hard disks or removable disks, magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0085] To provide for interaction with a user, embodiments of the subject matter described herein may be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user, and a keyboard and pointing device, e.g., a mouse or trackball, by which the user may provide input to the computer. Other types of devices may also be used to provide for interaction with the user, e.g., feedback provided to the user may be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback, and input from the user may be received in any form, including acoustic input, speech input, or tactile input. In addition, the computer may interact with the user by sending documents to and receiving documents from a device used by the user, e.g., by sending a web page to a web browser on the user's client device in response to a request received from the web browser.

[0086] An embodiment of the subject matter described herein may be implemented in a computing system that includes back-end components, e.g., as a data server, or includes middleware components, e.g., an application server, or includes front-end components, e.g., a client computer having a graphical user interface or web browser through which a user may interact with an implementation of the subject matter described herein, or includes any combination of one or more such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include local area networks ("LANs") and wide area networks ("WANs"), internetworks (e.g., the Internet), and peer-to-peer networks (e.g., ad-hoc peer-to-peer networks).

[0087] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some embodiments, a server sends data (e.g., HTML pages) to a client device (e.g., for the purpose of displaying the data to and receiving user input from a user interacting with the client device). Data generated at the client device (e.g., the result of a user interaction) may be received at the server from the client device.

[0088] Although this specification contains many specific implementation details, these should not be interpreted as limitations on the scope of any invention or what may be claimed, but rather as descriptions of features specific to certain embodiments of a particular invention. Some features described in the context of separate embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments separately or in any suitable subcombination. Furthermore, although features may be described above as working in some combinations and may even be initially claimed as such, one or more features from a claimed combination may, in some cases, be deleted from the combination, and the claimed combination may be directed to a subcombination or a variation of the subcombination.

[0089] Similarly, although operations are shown in the figures in a particular order, this should not be understood as requiring such operations to be performed in the particular order or sequence shown, or that all illustrated operations be performed, to achieve desirable results. In some situations, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the program components and systems described may generally be incorporated together in a single software product or packaged in multiple software products.

[0090] Above, specific embodiments of the present subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order or sequence shown to achieve desirable results. In some implementations, multitasking and parallel processing may be advantageous. [Explanation of symbols]

[0091] 100 Exemplary Environment, Environment 102 Computer Networks, Networks 104 Publisher Website, Website, Publisher 106, 106-A, 106-B User Devices 108 Search Engines 110 Index 112 Query Log 114 Click Log 116 Search History Data 118 Repeatable Query Manager 202 Context Analyzer 204 Learning Model 206 Repeatable Query Cache 210 Repeat Query Icon 212 Search button 214 Search Box 216 Repeatable Query Selectors 218 "Save as shortcut" button, "Save as shortcut" 400 Exemplary Computer System, System 410 Processor, component 420 Memory, components 430 Storage devices, components 440 Input / Output Devices, Components 450 System Bus 460 Other input / output devices, such as keyboards, printers, and display devices

Claims

1. 1. A computer-implemented method comprising: obtaining, by a computing system having one or more processors, a training dataset, the training dataset including a plurality of context datasets, a plurality of labels, and a plurality of training queries, each of the plurality of context datasets including context data determined for a respective training query of the plurality of training queries, each of the plurality of labels indicating whether each of the training queries has been issued a threshold number of times; processing, by the computing system, a context data set of the plurality of context data sets with a learning model to determine a likelihood of a future issuance of a first training query associated with the context data set; determining, by the computing system, that the first training query is a repeatable query if the likelihood of the first training query being issued in the future exceeds a repeatability threshold; training the learning model based on: (i) determining, by the computing system, that the first training query is the repeatable query; and (ii) a label in the plurality of labels for the first training query indicating that the first training query has been issued the repeatable threshold number of times. method.

2. obtaining a set of search queries issued by a plurality of user devices; For the set of search queries, determining a set of respective context data sets based on a query log for the set of search queries; determining a plurality of repeatable queries based on processing the respective sets of context data sets with the learning model to determine a plurality of respective likelihoods; storing the plurality of repeatable queries along with other repeatable search queries previously identified as repeatable queries. The method of claim 1.

3. The step of storing the plurality of repeatable queries along with other repeatable search queries previously identified as repeatable queries further includes storing, by a search engine, the plurality of repeatable queries along with search results in response to the plurality of repeatable queries. The method of claim 2.

4. Each of the respective sets of context data sets for the set of search queries comprises: The language in which a particular query is written, and the geographic location from which the particular query is issued; and and a geographic location of interest to the user device that issued the particular query. The method of claim 2.

5. Each of the respective sets of context data sets for the set of search queries comprises: The number of times a particular query is issued by a particular user device; and and a number of unique user devices that have issued the particular query a threshold number of times. The method of claim 2.

6. Each of the respective sets of context data sets for the set of search queries comprises: an embedding of a particular query from among a plurality of repeatable queries, the embedding expressing a semantic relationship between the particular query and other queries; and determining whether the particular query is directed to a particular web location or website by comparing text of the particular query to a list of known web sites or web locations; Contains at least one of The method of claim 2.

7. providing on a user device a user selectable interface component that, when selected by a user device, results in issuance of a query from among the repeatable queries without receiving user input of a query element; receiving a first selection of the user-selectable interface component from the user device requesting issuance of a particular query from among the repeatable queries; and in response to receiving the first selection from the user device, providing, by a search engine, a first search result page including search results for the particular query. The method of claim 2.

8. receiving a second selection of the user selectable interface component from the user device requesting issuance of the particular query; and in response to receiving the first selection from the user device, providing, by a search engine, a second search results page including search results for the particular query. The method of claim 7.

9. when the search results on the second search result page are ordered differently than the search results on the first search result page; or the search results on the second search results page differ from the search results on the first search results page; or when the second search result page includes dynamic content that is not included in the first search result page, The second search results page is different from the first search results page. The method of claim 8.

10. The dynamic content includes content that is more recently generated than content from the first search results page. The method of claim 9.

11. one or more processors; When executed by the one or more processors, obtaining a training dataset, the training dataset including a plurality of context datasets, a plurality of labels, and a plurality of training queries, each of the plurality of context datasets including context data determined for a respective training query of the plurality of training queries, each of the plurality of labels indicating whether the respective training query has been issued a threshold number of times; processing a context data set of the plurality of context data sets with a learning model to determine a likelihood of a first training query associated with the context data set being issued in the future; determining that the first training query is a repeatable query if the likelihood of the first training query being issued in the future exceeds a repeatability threshold; training the learning model based on (i) determining that the first training query is the repeatable query; and (ii) a label for the first training query among the plurality of labels indicating that the first training query has been issued the repeatable threshold number of times. and one or more non-transitory computer-readable storage media that collectively store a plurality of instructions that cause a computing system to perform operations that include: Computing system.

12. The operation includes: obtaining a search query; determining respective context data for the search queries based on a query log of the search queries; and processing the respective context data through the learning model to determine a likelihood of each of the search queries being issued in the future.

12. The computing system of claim 11.

13. The contextual data represents a context in which the search query was issued and a user interaction with a search result page provided in response to the search query.

13. The computing system of claim 12.

14. The instruction: identifying the search queries as repeatable queries based on the respective likelihoods of future issuance of the search queries satisfying a repeatability threshold; storing the search query along with other repeatable search queries previously identified as repeatable queries; and providing on the user device a user-selectable interface component that, when selected by the user device, results in issuance of a query from among the repeatable queries without receiving user input of a query element.

13. The computing system of claim 12.

15. The operation includes: receiving a first selection of the user-selectable interface component from the user device requesting issuance of a particular query from among the repeatable queries; and in response to receiving the first selection from the user device, providing, by a search engine, a first search result page including search results for the particular query.

15. The computing system of claim 14.

16. When executed by one or more processors: obtaining a training dataset, the training dataset including a plurality of context datasets, a plurality of labels, and a plurality of training queries, each of the plurality of context datasets including context data determined for a respective training query of the plurality of training queries, each of the plurality of labels indicating whether the respective training query has been issued a threshold number of times; processing a context data set of the plurality of context data sets with a learning model to determine a likelihood of a first training query associated with the context data set being issued in the future; determining that the first training query is a repeatable query if the likelihood of the first training query being issued in the future exceeds a repeatability threshold; training the learning model based on (i) determining that the first training query is the repeatable query; and (ii) a label for the first training query among the plurality of labels indicating that the first training query has been issued the repeatable threshold number of times. One or more non-transitory computer-readable storage media that collectively store a plurality of instructions that cause a computing system to perform operations that include:

17. The context data set includes a selection by a user device of one or more search results provided on a search result page for each query.

17. One or more non-transitory computer-readable storage media as recited in claim 16.

18. The contextual data set includes a time of viewing by the user device of one or more search results provided on a search result page for each query.

17. One or more non-transitory computer-readable storage media as recited in claim 16.

19. The context data set includes a selection of navigational interface elements on a search result page provided for each query.

17. One or more non-transitory computer-readable storage media as recited in claim 16.

20. The learning model includes a machine learning model and a rule engine.

17. One or more non-transitory computer-readable storage media as recited in claim 16.

Images