Personalized related queries for search segments

Abstract

A mechanism for personalizing a user's quick answers is described. A query history consists of queries previously submitted by a user. Each entry in the query history includes a query, an associated quick answer, and a user ID associated with the user who submitted the query. A query database is created by submitting the query for each query entry in the query history to a trained machine learning classifier to classify the query using the associated quick answer segment. The quick answer segment is combined with other information in the query history to create an entry into the query database. When a current query from a user is received, the query database is searched and previous queries from the same user with the same quick answer segment are extracted. A subset of the resulting queries is combined with a subset of queries from other users to personalize search results.

Description

Technical Field

[0001] The present invention relates generally to personalized search and more particularly to improvements in search services, and to improvements in presenting a personalized search experience in search services for quick answers. Background Art

[0002] When internet search services were first developed, results pages consisted of a list of "blue links" that, when activated by the user, would take the user to websites potentially related to the entered search query. As search services have become increasingly sophisticated, user expectations have also increased. Today, users expect to enter a query such as "How tall is Mount Everest?" and receive an immediate answer, rather than being presented with a list of blue links where the user can find the answer to their query. These so-called quick answers have improved the user search experience, allowing users to find answers to their queries without having to navigate to multiple websites.

[0003] It is in this context that the present embodiment arises. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Figure 1 An example user interface showing a personalized search query for quick answers is illustrated, in accordance with aspects of the present disclosure.

[0005] Figure 2 An example architecture according to some aspects of the present disclosure is illustrated.

[0006] Figure 3 An example flow diagram according to some aspects of the present disclosure is illustrated.

[0007] Figure 4 A diagram illustrating an example according to some aspects of the present disclosure illustrates identification of segments associated with a query.

[0008] Figure 5 A diagram illustrating an example according to some aspects of the present disclosure illustrates identification of queries associated with segments.

[0009] Figure 6 A diagram illustrating an example according to some aspects of the present disclosure illustrates identification of queries associated with segments.

[0010] Figure 7 Representative architectures for implementing the systems and other aspects disclosed herein, or for performing the methods disclosed herein, are illustrated. DETAILED DESCRIPTION

[0011] The following description includes illustrative systems, methods, user interfaces, techniques, instruction sequences, and computing machine program products that illustrate illustrative embodiments. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide an understanding of the various embodiments of the subject matter of the invention. However, it will be apparent to one skilled in the art that the embodiments of the subject matter of the invention may be practiced without these specific details. In summary, well-known instruction examples, protocols, structures, and techniques are not shown in detail.

[0012] Overview

[0013] The following summary is provided to introduce a selection of design concepts in a simplified form that are further described below in the description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Its sole purpose is to present some concepts in a simplified form as a prelude to the detailed description that will be presented later.

[0014] When a user submits a query that lends itself to quick answers, today's internet search engines provide one or more quick answers. Thus, if a user submits the query "How tall is Mount Everest," the search engine might present a quick answer informing the user that Mount Everest is 29,029 feet, or 8,848 meters. Quick answers provided by internet search engines typically fall into quick answer segments, sometimes referred to herein as segments. Segments are categories for quick answers. Examples of segments include, but are not limited to, health, politics, news, finance, sports, and esports.

[0015] Some search engines categorize search results into "verticals," which are similar to but separate from segments in at least two ways. First, verticals encompass search results, while segments are reserved for quick answers. Additionally, verticals are typically fixed categories for search results, while segments tend to be more fluid and dynamic, so the segments present in a search engine can change from time to time.

[0016] When a user submits a current query that generates one or more quick answers, embodiments of the present disclosure present a set of personalized query suggestions in conjunction with the quick answers. The set of personalized query suggestions may include queries previously submitted by the user and queries submitted by other users that meet specific criteria, such as trending queries that are related and / or relevant to the current query.

[0017] In one example embodiment, queries for a time period are analyzed and used to create a query database. Each entry in the query database includes a dataset that allows for the extraction of previous queries submitted by one or more users and related to a given segment. Thus, when a current query is submitted, the segment associated with the current query is identified. The segment is used to extract previous queries submitted by the user. The queries can be ranked, and a subset of the ranked queries can be selected based on selection criteria. The selected subset can be presented with a quick answer that is close to the current query.

[0018] Additionally or alternatively, queries from other users may be presented. These queries may generate quick answers, or may be of a type that does not generate quick answers. In one aspect, queries from other users that generate one or more results identical to the current query may be identified, ranked, and a subset selected for presentation to the user. In another aspect, queries from other users that generate quick answers in the same segment as the current query may be identified, ranked, and a subset selected for presentation to the user. In yet another aspect, a combination thereof may be identified, ranked, and a subset selected for presentation to the user.

[0019] The user's interaction and / or engagement with the presented queries can be used as part of a feedback loop to adjust the ranking and selection of queries by the user or other users. For example, when the user interacts with one of the presented query suggestions, the system can make the query more likely to be suggested again in the future. Additionally or alternatively, when the user does not interact with one of the presented query suggestions, the system can make the query more likely not to be suggested in the future.

[0020] describe

[0021] Figure 1 An example 100 user interface 102 is illustrated that illustrates a personalized search query for quick answers, according to some aspects of the present disclosure. The user interface 102 includes a search bar 104 in which a user can enter a query. In other aspects, the user interface includes one or more additional areas (106, 108, 118, 110, 112, 114, 116), which can appear alone or in any combination. In addition, the arrangement of the areas can vary in position from embodiment to embodiment. Thus, area 110 can appear on the left side of the screen instead of the right side, or can appear in a different vertical position relative to the other areas displayed. Finally, these areas can be different sizes or shapes than the areas depicted in the user interface 102. The displayed areas are the areas that together make up the search results page that is displayed in response to the submitted query.

[0022] Area 106 displays additional controls that can be activated by the user to adjust the content seen in the search results and / or adjust settings or invoke tools available to the user. For example, a series of verticals can be displayed that, when activated, limit the query results to those in a particular vertical. Activation of a particular vertical typically causes the query and the intended vertical to be resubmitted to the search engine. Example verticals may include, but are not limited to, images, videos, maps, news, shopping, finance, etc.

[0023] Region 108 displays one or more quick responses in response to the submitted query. Region 118 displays personalized query suggestions and / or a collection of related queries, which are identified according to the mechanisms described herein. In this application, query suggestions and related queries will be treated the same and will not be distinguished herein.

[0024] Area 112 includes the body of the search results that respond to the query. These are the "blue links" and other information about the search results.

[0025] Additionally or alternatively, the results page may include one or more additional areas 110, 114, 116. These may be different "panes" that can display additional information related to the query. For example, focused information related to the query may be displayed on such an area in a manner that is easy for the user to consume. For example, if a user submits the query "MLB records," the quick answer pane 102 may display the records of different Major League Baseball teams according to their league and their division, along with key statistics. Area 110 may then be used to display summary information about the current Major League Baseball season, highlight different players, and the like. As another example, searching for a stock symbol may result in key stock price information being displayed in the quick answer area 108, while one or more additional areas in the additional areas 110, 114, and / or 116 may be used to display additional information about the company corresponding to the stock price.

[0026] However, to date, no search engine has displayed a collection of personalized query suggestions in a results page (e.g., in area 118) that includes suggestions such as previous queries submitted by the user and / or additional queries submitted by other users that are related to the query that generated the quick answer. Embodiments of the present disclosure identify personalized query suggestions and display them near the quick answer generated by the user's query.

[0027] Figure 2 An example architecture 200 is illustrated according to some aspects of the present disclosure. Figure 1. The architecture includes both offline processes and processes executed in response to submitted user queries. In this context, “offline” means that personalized suggestions are identified and displayed for a user query before the user query is received.

[0028] The offline process begins by assembling a query history 204 from queries 202 submitted by multiple users. As explained in more detail below, the query history may include a user identifier (user ID) that identifies the user who submitted the query 202. The query history 204 may include queries from various users over a specified time period. In one representative embodiment, the specified time period is approximately 6 months. In this context, approximately 6 months means that the period is 6 months within a week or two of the 6-month period. In other words, it does not need to be exactly 6 months, but can be within a period of plus or minus two weeks of the 6-month period. In other embodiments, other time periods may be used. The only requirement is that there are enough queries that allow the selection and ranking process to proceed as discussed herein. Ultimately, the number of personalized query suggestions presented to the user is small (typically between 1 and 10), so the total number of source queries 202 does not need to be large.

[0029] Each entry in the query history 204 is presented to a trained machine learning classifier 206. The purpose of the classifier 206 is to identify which quick answer segment the query belongs to. The problem that the classifier solves is, given a query and / or the answer / result to the query, which quick answer segment the query falls into. Identifying which query belongs to which quick answer segment is not an easy problem to solve. Queries have a one-to-many relationship with query results (one query produces many results), and query results may include more than one quick answer and / or more than one quick answer segment. Additionally, since queries can be expressed in a variety of alternative ways, all of which have the same meaning, simply matching language to quick answer segments is not always effective. In some embodiments, one classifier is used, and in some embodiments, more than one classifier is used.

[0030] In a representative example, the machine learning model of the classifier 206 can be trained to classify the query into one or more quick answer segments. In some embodiments, multiple classifiers 206 are used, each classifier being customized for one or more quick answer segments. In embodiments of the present disclosure, a variety of machine learning models can be effectively utilized. For example, logistic regression (LR), naive Bayes, random forest (RF), neural networks (NN) including deep neural networks (DNN), matrix factorization, and support vector machines (SVM) tools can be used to predict items and / or parameters associated therewith.

[0031] A machine learning model for predictive parameters can be trained by collecting data points consisting of query-quick answer segment pairs. These data points are collected when the search engine generates quick answers within a specific segment when a query is submitted. The machine learning model will learn the characteristics of the submitted query that result in a quick answer within a specific quick answer segment.

[0032] If the quick answer segments change over time, the machine learning model parameters are updated by retraining the model using training data that reflects the changes in the quick answer segments or by adjusting the machine learning model parameters using updated training data.

[0033] Query database 208 is thus a database from which queries associated with segments can be retrieved. Although a single query database 208 is illustrated, multiple databases can be used if desired. Additionally, query database 208 can use any type of data storage structure or format that allows for retrieval of the desired information as discussed herein.

[0034] When a current user query 210 is received by a search system 212, search results are received that include one or more quick answers 214. The quick answer segment(s) 216 associated with each quick answer 214 can be identified from the search results themselves and / or by submitting the query 210 to an appropriate classifier (e.g., one or more of the classifiers 206).

[0035] Extraction process 218 searches for queries submitted by the user who submitted the current query 210. This is illustrated by extraction process 218, which uses the user ID 220 associated with the user to retrieve queries with matching user ID 220 from the query database. In this embodiment, retrieved queries 222 represent queries previously submitted by the user. In some embodiments, both user ID 220 and quick answer segments 216 are present. In this embodiment, retrieved queries 222 represent queries previously submitted by the user that share a common quick answer segment 216 with the currently submitted query 210. The retrieved user queries are referred to as user queries 222.

[0036] Additionally or alternatively, embodiments of the present disclosure may retrieve queries from the query database 208 from other users. These retrieved queries 224 are queries that can be used to expand query suggestions beyond previous queries submitted by the user. Thus, queries from other users having quick answer segments that match the quick answer segments 216 may be retrieved from the query database 208 via the extraction process 218. In an alternative embodiment, as explained below, queries from other people that have one or more results that match the results of the current query 210 may be retrieved from the query database 208 via the extraction process 218. The resulting queries 224 are queries that have at least one common result with the current query 210. The common results may be quick answer results, non-quick answer results, or a combination thereof. Queries from other users are referred to herein as expanded queries 224.

[0037] The user queries 222 are subjected to a ranking and selection process 226 to produce a subset of user queries 230. The ranking and selection process may rank the user queries 222 according to one or more criteria. For example, the user queries 222 may be ranked using one or more criteria including, but not limited to:

[0038] • Time: Queries submitted more recently by a user may be ranked higher or lower than queries submitted less recently by the user.

[0039] Frequency: Ranking may be proportional to the number of times a user submits the same or similar query, with more frequent submissions of queries resulting in higher rankings.

[0040] Interaction with results: Queries with results that the user has shown to interact and / or engage with are ranked higher than queries with results that the user has not interacted with;

[0041] Similarity: queries that are more or less similar to the current query 210 are ranked higher or lower; and

[0042] Any other criteria or criteria.

[0043] After query ranking, subset 230 can be selected based on any selection criteria. For example, the top N queries can be selected. Additionally or alternatively, queries ranked above / below a threshold can be selected. Additionally or alternatively, a combination of the two, such as the top N queries ranked above / below a threshold, can be selected. Other selection criteria can also be used.

[0044] The expanded queries 224 can be subjected to a ranking and selection process 228, which can be the same as or similar to the ranking and selection process 226 for the user query 222. Additionally or alternatively, the ranking and selection process 228 can select one or more expanded queries for the subset 232 on a random basis. In other words, queries are randomly selected from some or all of the expanded queries 224. Thus, the expanded queries 224 can be ranked and selected based on any of the above criteria and / or on a random basis to produce the resulting subset 232.

[0045] The user query 230 and / or the expanded query 232 can be combined with one or more quick answers and placed near one or more quick answers. This is performed by a combination process 234, which creates a layout of search results (not shown), quick answers (not shown), query suggestions (e.g., user query 230 and / or expanded query 232), and / or other information to produce one or more results pages 236.

[0046] The system can monitor whether and how the user interacts with the presented query suggestions, and whether and how the user interactions are used as feedback 238 to the adjustment process. For example, if the user does not interact with the user query from subset 230 and / or the expanded query from subset 232, the extraction process 218 and / or the ranking and selection processes 226, 228 can be adjusted so that the user query and / or the expanded query will be less likely to be selected. This can be performed by marking entries in the query database 208 so that they will not be extracted by the extraction process 218 (e.g., will not be part of the expanded query 224 and / or the user query 222); by adjusting the ranking and / or selection process so that they are less likely to be selected, or a combination thereof.

[0047] Figure 3 An example flowchart 300 is illustrated according to some aspects of the present disclosure. Flowchart 300 partially illustrates Figure 2 An example of how the architecture can work.

[0048] The process begins at operation 302 and proceeds to operation 304, which identifies quick answer segments associated with a query in the query history 306 as previously described. As discussed herein, this can be performed using one or more trained machine learning models. Operation 304 is an offline process as described, meaning it is performed before receiving the current user query. As previously described, the results of operation 304 can be stored in one or more query databases.

[0049] Operation 308 identifies quick answer segments associated with the current query 310 that has been submitted to the search system by the user. As discussed, this can be performed by evaluating the results to identify quick answer segments associated with the quick answers in the search results. Additionally or alternatively, a classifier employing a trained machine learning model can be utilized, as discussed herein.

[0050] Both operations 312 and 314 may be performed, or only one or the other, depending on whether query suggestions need to be presented including previous queries from the user submitting the current query 310 , expanded queries from other users, or both.

[0051] Operation 312 generates previous queries submitted by the user. Thus, the operation retrieves previous user queries associated with the quick answer segment associated with the current query 310. The queries can be queries that generate quick answers, queries that do not generate quick answers, or any combination thereof. These can be retrieved using appropriate parameters from one or more query databases as described herein.

[0052] Operation 314 generates queries from other users that are related to the current query. As described herein, this can be accomplished by retrieving queries from one or more query databases that have associated quick answer segments that match the quick answer segments associated with the current query 310. Additionally or alternatively, queries from other users that produce results that match (e.g., the same or similar results) the results produced by the current query can be retrieved.

[0053] The query set from operation 312 and / or the query set generated by operation 314 may be ranked and / or selected as described herein to generate selected query suggestions in operation 316. The query suggestions from operation 316 are presented proximate to the quick answers associated with the current query 310.

[0054] The process ends at operation 318 .

[0055] Figure 4 A diagram 400 illustrating an example according to some aspects of the present disclosure illustrating identification of quick answer segments associated with a query presents an alternative to an offline process of creating one or more query databases, such as query database 208, in greater detail.

[0056] As discussed, identifying one or more quick answer segments associated with a query is not an easy problem to solve. A query 402 with an associated user ID 404 may generate multiple search results 406 and / or multiple quick answers 408. Each quick answer may have one or more associated quick answer segments 410. Furthermore, two different queries may generate the same or similar search results, quick answers, and their associated quick answer segments, increasing the complexity of the problem.

[0057] Thus, in some embodiments, query history process 412 collects query history 424, which includes entries for quick answers and / or search results. For example, query history 424 may include a query-quick answer-user ID tuple, meaning that each entry in the query history includes a query, a quick answer generated by the query, and the user ID of the user who submitted the query. A query-query answer-user ID tuple 414 may be created for each of these combinations. Thus, a query submitted by a single user that generates three quick answers will generate three separate query-quick answer-user ID tuples 414.

[0058] Although not shown, if the query answer segment 410 is known, it can be added to the appropriate tuple. However, if the query answer segment changes so that an answer that was previously associated with one query answer segment is now associated with a different query answer segment, the collected query answer segment may not be usable.

[0059] For search results 406 that are not quick answers 408, the query history may include a query-search result-user ID tuple 416, which means that each entry in the search result includes the query, the search result generated by the query, and the user ID. In some embodiments, the user ID is optional. As before, an appropriate tuple is generated for each combination. Thus, a query submitted by a single user that generates five search results will generate five separate query-search result-user ID (if used) tuples 416.

[0060] As discussed herein, in some embodiments, the search history 424 may include approximately 6 months of data, although longer or shorter time periods may be used.

[0061] One or more classifiers 418, including one or more trained machine learning models, can be used to identify a quick answer segment associated with each entry in the query history 424. The machine learning model can be trained as described herein. The result can be a query database 426, where each entry in the quick answer includes a query, a quick answer generated by the query, a user ID, and a quick answer segment, as shown at 420.

[0062] In some embodiments, queries associated with search results (not quick answers) can also be classified using one or more appropriately trained classifiers 418. Therefore, in some embodiments, query-search result pairs and / or query-search result-user ID tuples can be coupled with query answer segments 422, as shown in 422. These tuples can be stored in a query database 426. In other embodiments, answer segments associated with query-search result pairs are identified using different processes. This alternative is discussed in more detail below. In these embodiments, query database entries for queries associated with search results can include query-search result pairs and / or query-search result-user ID tuples.

[0063] Figure 5 A diagram 500 illustrating an example of some aspects of the present disclosure illustrates the identification of queries associated with a segment. The diagram specifically shows how previous user queries associated with the same quick answer segment can be identified (e.g., Figure 3 Operation 312 and / or Figure 2 Examples of the extraction process 218 and / or selection and ranking process 226).

[0064] A current user query 502 is received from a user associated with a user ID 504. As discussed herein, such a query may generate one or more search results 506 and / or one or more quick answers 508, the one or more quick answers 508 having one or more associated quick answer segments 510. To retrieve previous queries that generated quick answers from a query database 526 (e.g., 208, 424), each combination of a user ID 504, a quick answer 512, and its associated segments 514 is used to search the query database 526. Specifically, a quick answer 512 is a quick answer from the set of generated quick answers 508, and an associated quick answer segment 514 of a quick answer 512 is a quick answer segment from the set of quick answer segments 510 associated with the selected quick answer 512.

[0065] Therefore, a database search 516 is performed on query database 526 to extract queries having a user ID that matches user ID 504 and having a segment that matches segment 514. In an alternative embodiment, if the query database does not categorize the query with an associated segment, the query has a user ID that matches (is the same as or similar to) user ID 504 and has a quick answer that matches quick answer 512.

[0066] Resulting queries 518 are queries that are presented for further ranking and selection as described herein.

[0067] If the query database 526 includes query-search result-user ID tuples (with or without associated quick answer segments), the same process can be used to extract queries that are not associated with quick answers. For example, the user ID 504 and the search result (e.g., one of the search results in the set of search results 506) can be used to search the query database 526, and extract those queries 518 that have search results that match (e.g., are the same or similar to) the search results from the set 506. If the segments are present in the database, the segments can also be matching criteria in addition to or in addition to the search results.

[0068] Figure 6 A diagram 600 illustrating an example of some aspects of the present disclosure illustrates the identification of queries associated with a segment. The diagram shows how queries can be extracted from the database without regard to user ID or users other than the user who submitted the current query. The diagram specifically shows how previous user queries associated with the same quick answer segment can be identified (e.g., Figure 3 Operation 314 and / or Figure 2 Examples of the extraction process 218 and / or selection and ranking process 228).

[0069] A current user query 602 is received from a user associated with a user ID 604. As discussed herein, such a query may generate one or more search results 606 and / or one or more quick answers 608, one or more quick answers 608 having one or more associated quick answer segments 610. To retrieve previous queries related to the current query 602 from a query database 626 (e.g., 208, 424), each combination of user ID 604 and / or search results 612 and / or their associated segments 614 is used to search the query database 626. Specifically, the search results 612 are search results from the set of search results 606.

[0070] In a first aspect, query database 626 does not include segments and contains query-search result pairs, with or without the user ID of the user who submitted the query. If it is desired to retrieve queries related to segment 614 submitted by other users, database search 616 is used to retrieve queries 618 from query database 626, which have search results that match search result 612 and have a user ID that does not match user ID 614. If it is desired to retrieve queries without regard to user ID, user ID 604 is not part of the search criteria.

[0071] If the entries in query database 626 have associated quick answer segments, then queries related to current query 602 can be retrieved by selecting those queries 681 that have segments that match segment 614 and have a user ID that is equal to or not equal to user ID 604, depending on whether queries submitted by the user or others are desired. If queries are to be retrieved regardless of user ID, user ID 604 is not part of the search criteria.

[0072] If segment 610 contains only a single segment, segment 614 is known. If segment 610 contains multiple segments, a properly trained machine learning model can classify search result 612 as a quick answer segment, and that segment can be used for segment 614.

[0073] Example machine architecture and machine-readable medium

[0074] Figure 7 A representative machine architecture suitable for implementing the systems disclosed herein, etc., or for performing the methods disclosed herein is illustrated. Figure 7 The machines are shown as standalone devices that are suitable for implementing the above concepts. For the server aspects described above, multiple such machines operating in a data center, part of a cloud architecture, etc. may be used. In terms of the server, not all of the functions and devices shown are utilized. For example, while the systems, devices, etc. that a user uses to interact with the server and / or cloud architecture may have screens, touch screen inputs, etc., servers typically do not have screens, touch screens, cameras, etc., and typically interact with users through connected systems with appropriate input and output aspects. Therefore, the following architecture should be viewed as encompassing multiple types of devices and machines, and various aspects may or may not be present in any particular device or machine depending on its form factor and use (e.g., servers rarely have cameras, and wearable devices rarely include disks). However, Figure 7 The exemplary explanations are suitable to allow those skilled in the art to determine how to implement the previously described embodiments using appropriate combinations of hardware and software, with appropriate modifications to the embodiments shown for the specific devices, machines, etc. used.

[0075] While a single machine is illustrated, the term "machine" shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

[0076] The example machine 700 includes at least one processor 702 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), an advanced processing unit (APU), or a combination thereof), one or more memories, such as main memory 704, static memory 706, or other types of memory, which communicate with each other via a link 708. The link 708 can be a bus or other type of connection. The machine 700 can include other optional aspects, such as a graphics display unit 710 including any type of display. The machine 700 may also include other optional aspects, such as an alphanumeric input device 712 (e.g., a keyboard, a touch screen, etc.), a user interface (UI) navigation device 714 (e.g., a mouse, a trackball, a touch device, etc.), a storage unit 716 (e.g., a disk drive or other storage device(s)), a signal generating device 718 (e.g., a speaker), sensor(s) 721 (e.g., a global positioning sensor, accelerometer(s), microphone(s), camera(s), etc.), an output controller 728 (e.g., a wired or wireless connection for connecting and / or communicating with one or more other devices (e.g., a universal serial bus (USB), near field communication (NFC), an infrared (IR) device, a serial / parallel bus, etc.)), and a network interface device 720 (e.g., wired and / or wireless) for connecting to and / or communicating via one or more networks 726.

[0077] Executable instructions and machine storage media

[0078] Various memories (i.e., 704, 706 and / or the memory of processor 702) and / or storage unit 716 may store one or more sets of instructions and data structures (e.g., software) 724 embodied or utilized by one or more methods or functions described herein. These instructions, when executed by processor(s) 702, cause various operations to implement the disclosed embodiments.

[0079] As used herein, the terms "machine storage medium," "device storage medium," and "computer storage medium" have the same meaning and are used interchangeably in this disclosure. The terms refer to a single or multiple storage devices and / or media (e.g., a centralized or distributed database, and / or associated caches and servers) that store executable instructions and / or data. Thus, these terms should be considered to include storage devices such as solid-state memory, as well as optical and magnetic media, including memory internal or external to the processor. Specific examples of machine storage medium, computer storage medium, and / or device storage medium include non-volatile memory, which includes, by way of example, semiconductor memory devices (e.g., erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)), FPGAs, and flash memory devices, magnetic disks (such as internal hard disks and removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. The terms machine storage medium, computer storage medium, and device storage medium specifically and expressly exclude carrier waves, modulated data signals, and other such transient media, at least some of which are encompassed by the term "signal media" discussed below.

[0080] Signal medium

[0081] The term "signal medium" should be taken to include any form of modulated data signal, carrier wave, etc. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.

[0082] Computer readable media

[0083] The terms "machine-readable medium," "computer-readable medium," and "device-readable medium" are synonymous and are used interchangeably in this disclosure. These terms are defined to include both machine storage media and signal media. Thus, these terms include both storage devices / medium and carrier / modulated data signals.

[0084] Example Embodiments

[0085] Example 1. A method of illustrating personalization, comprising:

[0086] identifying multiple queries across at least one user group;

[0087] for each query in the plurality of queries, identifying at least one associated quick answer segment;

[0088] Receive the current query from the user;

[0089] identifying a current quick answer segment associated with the current query;

[0090] retrieving a previous query received from the user that has an associated quick answer segment that is equal to the current quick answer segment;

[0091] selecting a subset of the previous queries received from the user; and

[0092] The subset of quick answers proximately associated with the current query is presented on a results page that includes results based on the current query.

[0093] Example 2. The method of claim 1, wherein the plurality of queries comprises queries received by all users over a specified time period.

[0094] Example 3. The method of claim 2, wherein the specified time period comprises approximately six months.

[0095] Example 4. The method of claim 1, 2, or 3, wherein identifying the at least one associated quick answer segment comprises:

[0096] accessing a machine learning model trained to classify received queries into quick answer segments; and

[0097] A subset of the plurality of queries is presented to the machine learning model to identify at least one quick answer segment associated with each query in the subset.

[0098] Example 5. The method of claim 1, 2, 3, or 4, wherein identifying at least one associated quick answer comprises:

[0099] For each query in the plurality of queries, results received from the search system in response to submitting each query in the plurality of queries to the search system are evaluated.

[0100] Example 6. The method of claim 1, 2, 3, 4, or 5, wherein identifying the current quick answer segment comprises evaluating results received from a search system in response to submitting the current query to the search system.

[0101] Example 7. The method of claim 1, 2, 3, 4, 5, or 6, wherein identifying the plurality of queries across at least one user group comprises:

[0102] For each query in the plurality of queries:

[0103] identifying a query quick answer in search results received by submitting a query of the plurality of queries to a search system;

[0104] Creating a query record, the query record including the query, the user ID of the user who submitted the query, and the query quick answer; and

[0105] The query records are assembled into a query history.

[0106] Example 8. The method of claim 7, further comprising:

[0107] For each query record in the query history:

[0108] submitting the query in the query record to a trained machine learning model and receiving a quick answer segment in response;

[0109] The quick answer segment is combined with the query record to create an entry into a query database.

[0110] Example 9. The method of claim 8, wherein retrieving a previous query received from the user that has an associated quick answer segment that is equivalent to the current quick answer segment comprises:

[0111] The query database is searched and queries having a user ID associated with the user and having a quick answer segment equal to the current quick answer segment are extracted therefrom.

[0112] Example 10. The method of claim 1, 2, 3, 4, 5, 6, 7, 8, or 9, further comprising:

[0113] retrieving previous queries received from other users, the previous queries having at least one search result in common with the current query;

[0114] selecting a subset of the previous queries from other users; and

[0115] The subset of the previous queries from other users is presented, the subset being close to the subset of the previous queries received from the user.

[0116] Example 11. The method of claim 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10, wherein identifying the plurality of queries across at least one user group comprises:

[0117] For each query in the plurality of queries:

[0118] identifying a query quick answer in search results received by submitting a query of the plurality of queries to a search system;

[0119] Creating a query record, the query record including the query, the user ID of the user who submitted the query, and the query quick answer; and

[0120] Assembling the query records into a query history;

[0121] For each query record in the query history:

[0122] submitting the query in the query log to a trained machine learning model and receiving a quick answer segment in response;

[0123] combining the quick answer segment with the query record to create an entry into a query database; and

[0124] wherein retrieving a previous query received from the user that has an associated quick answer segment that is equal to the current quick answer segment comprises:

[0125] The query database is searched and queries having a user ID associated with the user and having a quick answer segment equal to the current quick answer segment are extracted therefrom.

[0126] Example 12. The method of claim 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, or 11, wherein the plurality of queries comprises queries received from a subset of users over a period of time.

[0127] Example 13. The method of claim 12, wherein the time period comprises approximately six months.

[0128] Example 14. An apparatus comprising components for performing the method of any of the preceding examples.

[0129] Example 15. A machine-readable storage device comprising machine-readable instructions, which, when executed, are used to implement the method or apparatus as described in any of the above examples.

[0130] Example 16. A method of illustrating personalization, comprising:

[0131] identifying multiple queries across at least one user group;

[0132] for each query in the plurality of queries, identifying at least one associated quick answer segment;

[0133] Receive the current query from the user;

[0134] identifying a current quick answer segment associated with the current query;

[0135] retrieving a previous query received from the user that has an associated quick answer segment that is identical to the current quick answer segment;

[0136] selecting a subset of the previous queries received from the user; and

[0137] The subset of quick answers proximately associated with the current query is presented on a results page that includes results based on the current query.

[0138] Example 17. The method of claim 16, wherein the plurality of queries comprises queries received by all users over a specified time period.

[0139] Example 18. The method of claim 17, wherein the specified time period comprises approximately six months.

[0140] Example 19. The method of claim 16, wherein identifying the at least one associated quick answer segment comprises:

[0141] accessing a machine learning model trained to classify received queries into quick answer segments; and

[0142] A subset of the plurality of queries is presented to the machine learning model to identify at least one quick answer segment associated with each query in the subset.

[0143] Example 20. The method of claim 16, wherein identifying at least one associated quick answer comprises:

[0144] For each query in the plurality of queries, results received from the search system in response to submitting each query in the plurality of queries to the search system are evaluated.

[0145] Example 21. The method of claim 16, wherein identifying the current quick answer segment comprises evaluating results received by the search system in response to submitting the current query to the search system.

[0146] Example 22. The method of claim 16, wherein identifying the plurality of queries across at least one user group comprises:

[0147] For each query in the plurality of queries:

[0148] identifying a query quick answer in search results received by submitting a query of the plurality of queries to a search system;

[0149] Creating a query record, the query record including the query, the user ID of the user who submitted the query, and the query quick answer; and

[0150] The query records are assembled into a query history.

[0151] Example 23. The method of claim 22, further comprising:

[0152] For each query record in the query history:

[0153] submitting the query in the query log to a trained machine learning model and receiving a quick answer segment in response;

[0154] The quick answer segment is combined with the query record to create an entry into a query database.

[0155] Example 24. The method of claim 23, wherein retrieving a previous query received from the user that has an associated quick answer segment that is equal to the current quick answer segment comprises:

[0156] The query database is searched and queries having a user ID associated with the user and having a quick answer segment equal to the current quick answer segment are extracted therefrom.

[0157] Example 25. The method of claim 16, further comprising:

[0158] retrieving previous queries received from other users, the previous queries having at least one search result in common with the current query;

[0159] selecting a subset of the previous queries from other users; and

[0160] The subset of the previous queries from other users is presented, the subset being close to the subset of the previous queries received from the user.

[0161] Example 26. A system comprising a processor and computer-executable instructions that, when executed by the processor, cause the system to perform operations comprising:

[0162] identifying multiple queries across at least one user group;

[0163] for each query in the plurality of queries, identifying at least one associated quick answer segment;

[0164] Receive the current query from the user;

[0165] identifying a current quick answer segment associated with the current query;

[0166] retrieving a previous query received from the user that has an associated quick answer segment that is equal to the current quick answer segment;

[0167] selecting a subset of the previous queries received from the user; and

[0168] The subset of quick answers proximately associated with the current query is presented on a results page that includes results based on the current query.

[0169] Example 27. The system of claim 26, wherein the plurality of queries comprises queries received by all users over a specified time period.

[0170] Example 28. The system of claim 27, wherein the specified time period comprises approximately six months.

[0171] Example 29. The system of claim 26, wherein identifying the at least one associated quick answer segment comprises:

[0172] accessing a machine learning model trained to classify received queries into quick answer segments; and

[0173] A subset of the plurality of queries is presented to the machine learning model to identify at least one quick answer segment associated with each query in the subset.

[0174] Example 30. The system of claim 26, wherein identifying at least one associated quick answer comprises:

[0175] For each query in the plurality of queries, results received from the search system in response to submitting each query in the plurality of queries to the search system are evaluated.

[0176] in conclusion

[0177] In view of the many possible embodiments to which the principles of the present invention and the foregoing examples can be applied, it should be understood that the examples described herein are illustrative only and should not be considered to limit the scope of the present invention. Therefore, the present invention as described herein encompasses all such embodiments that may fall within the scope of the appended claims and any equivalents thereof.

Claims

1. A method for providing query suggestions, the method comprising: receiving a query from a user at a computer-implemented search engine; Upon receiving the query: searching an index using the query to identify search results for the query; as well as identifying a quick answer to the query, wherein the quick answer is to be simultaneously presented on a search engine results page along with a link to the search result, and further wherein a quick answer segment is assigned to the quick answer, the quick answer segment indicating a category of the quick answer; identifying, after identifying the quick answer to the query, a previous query previously received from the user, wherein the previous query has the quick answer segment assigned thereto, and further wherein the previous query is identified based on the quick answer segment assigned to the identified quick answer to the query matching the quick answer segment assigned to the previous query; The search engine results page is presented to the user, the search engine results page comprising: the links to the search results identified based on the query; said quick answer to said query; and The previous query is presented as a query suggestion for the user, wherein the query suggestion is presented to visually indicate to the user that the quick answer segment is assigned to both the query suggestion and the quick answer.

2. The method of claim 1, wherein the quick answer segment is one of: health, politics, news, finance, sports, or esports.

3. The method according to claim 1, further comprising: Identifying a second previous query previously received from a second user, wherein the second previous query is identified based on the quick answer segment assigned to the identified quick answer to the query that matches the quick answer segment assigned to the second previous query, and further wherein the search engine results page includes the second previous query as a second query suggestion for the user.

4. The method according to claim 1, further comprising: Identifying the quick answer segment based on the query, wherein identifying the quick answer segment based on the query comprises: Accessing a machine learning model that is trained to classify received queries into quick answer segments, wherein the quick answer segments are assigned to the queries by the machine learning model.

5. The method of claim 1 , wherein identifying the previous query comprises: identifying a number of previous queries having the quick answer segment assigned thereto, the previous queries previously submitted by the user; as well as The previous queries are ranked based on a selection criterion, wherein a highest-ranked previous query among the previous queries is selected as the identified previous query. The method of claim 1 , wherein the search engine results page further comprises a second quick answer assigned to the query suggestion.

7. The method according to claim 1, further comprising: receiving a selection of the query suggestion; as well as Based on the selected query suggestion, a weight assigned to the query suggestion is updated to increase the likelihood that the query suggestion will be presented to the user after the user issues another query.

8. The method according to claim 1, further comprising: Get the queries submitted by each user within a specified time period; as well as The retrieved queries are provided to a machine learning classifier, wherein the machine learning classifier is configured to output an indication of whether each of the retrieved queries will have the quick answer segment assigned thereto.

9. The method according to claim 1, further comprising: The quick answer segment for the query is identified based on the search results.

10. A computing system comprising: processor; as well as a memory storing instructions that, when executed by the processor, cause the processor to perform actions, the actions comprising: receiving a query from a user at a computer-implemented search engine executing on the computing system; Upon receiving the query: searching an index using the query to identify search results for the query; and identifying a quick answer to the query, wherein the quick answer to the query is to be presented on a search engine results page concurrently with a link to the search result, and further wherein a quick answer segment is assigned to the quick answer, the quick answer segment indicating a category of the quick answer; subsequent to identifying the quick answer to the query, identifying a previous query previously submitted to the search engine, wherein the previous query had the quick answer segment assigned thereto, and further wherein the previous query is identified based on the quick answer segment assigned to the identified quick answer to the query matching the quick answer segment assigned to the previous query; The search engine results page is presented to the user, the search engine results page comprising: the links to the search results identified based on the query; said quick answer to said query; and The previous query as a query suggestion, wherein the query suggestion is presented to visually indicate to the user that the quick answer segment is assigned to both the query suggestion and the quick answer.

11. The computing system of claim 10, wherein the quick answer segment is one of: health, politics, news, finance, sports, or esports.

12. The computing system of claim 10, wherein the actions further comprise: Identifying a second previous query previously received from a second user, wherein the second previous query is identified based on the quick answer segment assigned to the identified quick answer to the query that matches the quick answer segment assigned to the second previous query, and further wherein the search engine results page includes the second previous query as a second query suggestion for the user.

13. The computing system of claim 10, wherein the actions further comprise: Identifying the quick answer segment based on the query, wherein identifying the quick answer segment based on the query comprises: Accessing a machine learning model that is trained to classify received queries into quick answer segments, wherein the quick answer segments are assigned to the queries by the machine learning model.

14. The computing system of claim 10, wherein identifying the previous query comprises: identifying a number of previous queries having the quick answer segment assigned thereto, the previous queries previously submitted by the user; as well as The previous queries are ranked based on a selection criterion, wherein a highest-ranked previous query among the previous queries is selected as the identified previous query.

15. The computing system of claim 10, wherein the search engine results page further comprises a second quick answer assigned to the query suggestion.

16. A computer storage medium comprising executable instructions that, when executed by a processor of a machine, cause the machine to perform operations comprising: receiving a query from a user at a computer-implemented search engine executing on a computing system; Upon receiving the query: searching an index using the query to identify search results for the query; as well as identifying a quick answer to the query, wherein the quick answer to the query is to be presented on a search engine results page concurrently with a link to the search result, and further wherein a quick answer segment is assigned to the quick answer, the quick answer segment indicating a category of the quick answer; identifying, after identifying the quick answer to the query, a previous query previously received from the user, wherein the previous query has the quick answer segment assigned thereto, and further wherein the previous query is identified based on the quick answer segment assigned to the identified quick answer to the query matching the quick answer segment assigned to the previous query; The search engine results page is presented to the user, the search engine results page comprising: the links to the search results identified based on the query; said quick answer to said query; and The previous query is presented as a selectable query suggestion to the user, wherein the selectable query suggestion is presented to visually indicate to the user that the quick answer segment is assigned to both the query suggestion and the quick answer.

17. The computer storage medium of claim 16, wherein the quick answer segment is one of: health, politics, news, finance, sports, or esports.

18. The computer storage medium of claim 16, the operations further comprising: Identifying a second previous query previously received from a second user, wherein the second previous query is identified based on the quick answer segment assigned to the identified quick answer to the query that matches the quick answer segment assigned to the second previous query, and further wherein the search engine results page includes the second previous query as a second query suggestion for the user.

19. The computer storage medium of claim 16, the operations further comprising: Identifying the quick answer segment based on the query, wherein identifying the quick answer segment based on the query comprises: Accessing a machine learning model that is trained to classify received queries into quick answer segments, wherein the quick answer segments are assigned to the queries by the machine learning model.

20. The computer storage medium of claim 16, wherein identifying the previous query comprises: identifying a number of previous queries having the quick answer segment assigned thereto, the previous queries previously submitted by the user; as well as The previous queries are ranked based on a selection criterion, wherein a highest-ranked previous query among the previous queries is selected as the identified previous query.

Images