Generating query variants using a trained generative model

Abstract

This application discloses generating query variants using a trained generative model. Systems, methods, and computer-readable media related to generating query variants for submitted queries. In many implementations, the query variants are generated using a generative model. The generative model is effective because it can automatically generate variants of a query based on tokens of the query being applied to the generative model and optionally based on applying additional input features to the generative model.

Description

[0001] Case Analysis

[0002] This application is a divisional application of Chinese invention patent application 201880028212.7, filed on April 27, 2018. Technical Field

[0003] This application relates to generating query variants using a trained generative model. Background Technology

[0004] Rule-based search query rewriting has already been utilized in the query processing component of search systems. For example, some rule-based rewriting can generate a rewritten query by removing certain stop words (such as "this," "one," etc.) from the query. The rewritten query can then be submitted to the search system, and the search results for the rewritten query are returned in response.

[0005] Furthermore, search systems already utilize sets of similar queries to recommend, for example, additional queries related to the submitted query (e.g., "people are also searching for X"). Queries similar to a given query are typically determined by navigation clusters. For example, for the query "funny cat pictures," similar queries for "funny cat pictures with captions" can be determined based on similar queries that users frequently submit after submitting "funny cat pictures." Therefore, similar queries for a given query are often predefined. Summary of the Invention

[0006] This specification relates to systems, methods, and computer-readable media relating to the generation of query variants for submitted queries. In many implementations, query variants are generated at runtime using a trained generative model. The generative model is efficient because it can proactively generate query variants based on tokens applied to the query and optionally based on additional input features applied to the generative model. In this way, the generative model can be used to generate variants of any query even without training it based on the query. Therefore, the generative model can be used to generate variants of novel queries and so-called “tail” queries (i.e., queries with a submission frequency and / or number of submissions below a threshold). As a result, queries can be processed more efficiently because richer query inputs can lead to more effective identification of relevant results. For example, queries cannot be excluded simply because of a low submission frequency and / or a small number of submissions. The efficiency improvement depends on the speed at which relevant results can be obtained, as the user does not need to resubmit the modified query if the initial query does not generate any relevant results. The disclosed implementation enables the automatic testing of multiple query variants. Furthermore, convergence of results can be ensured by training a model for generating variants, thereby improving efficiency not only by processing multiple queries simultaneously but also by generating targeted query variants. Thus, the implementation of this invention optimizes the use of technical resources required for query processing, including the processing power and power consumption of the processor implementing the disclosed method.

[0007] In some implementations, the generative model is a neural network model, such as a neural network model with one or more "memory layers". The memory layer includes one or more recurrent neural network (RNN) units, such as long short-term memory ("LSTM") units and / or gated recurrent units ("GRU").

[0008] In some implementations of generative models that are neural network models with storage layers, the generative model is a sequence-to-sequence model. For example, a sequence-to-sequence model can be a model in which a query token is applied as input to the model (e.g., on a token-by-token basis or on a combination basis) and an encoding of the token is generated at layers of the network. Furthermore, the generated encoding can be decoded at other layers of the network, where the resulting decoding (directly or indirectly) indicates a variant of the query. For example, the resulting decoding can be applied to one or more softmax layers of the network to generate variants of the query. In some versions of those implementations, the generative model has the same or similar architecture as a sequence-to-sequence neural machine translation model and is trained using query variant-specific training data. Query variant-specific training data can be based, for example, on: query pairs, each query pair having a "click" on the same document (e.g., to train equivalent query variant generation); sequentially submitted query pairs (e.g., for training follow-up query variant generation); and / or original canonicalized query pairs (e.g., for training for canonicalized query variant generation). Such a model can optionally be pre-trained based on translation training data.

[0009] In some implementations, the generative model is trained as a "multi-task" model because it is trained to be able to generate any of a variety of query variants. In some of those implementations, the type of query variant to be generated for a given pass of the generative model can be indicated based on the type value input applied to the model in that pass, where the type value input indicates the type of query variant to be generated. The types of query variants can include, for example, equivalent queries, follow-up queries, generalized queries, canonicalized queries, language translation queries, implication queries, canonical queries, and / or clarifying queries (i.e., queries provided as output to the user for clarification). Other and / or alternative types can be defined, including those with larger or smaller granularities. During the training of the generative model, training data from various types can be utilized, where each instance of the training data includes a type value input indicating the type of query variant for that instance, and can be used as training instance input during training. In some of these implementations, training a multi-task model in this way can leverage information sharing between various types of training data, which may result in a more robust performance of the trained multi-task model.

[0010] Once trained, the multi-task model can be used to generate a first type of variant for a query in a first pass (e.g., based on applying a first type of value as input) and a second type of variant for a query in a second pass (e.g., based on applying a second type of value as input). Additional variants of additional types can be generated in additional passes. As described herein, the number of additional passes generated using the multi-task model can vary depending on the query. For example, the number of additional passes can be controlled in a self-organizing manner based on, for example, one or more variants generated in a pass and / or responses to one or more such variants. Furthermore, subsequent passes can generate variants of the same type as the previous variants generated in previous passes. In some of those cases, as described herein, subsequent passes may utilize information based on previous variants (e.g., the previous variant itself and / or responses to previous variants) and / or other previous variants, which may result in variants in subsequent passes that differ from previous variants.

[0011] In some implementations, generative models are used to generate multiple variations of the original query, each of which is submitted to a search system, and a corresponding response is received for each variation. Output can be generated based on one or more responses and provided in response to the original query. For example, the output may include a “best” response (e.g., as indicated by a response score provided by the search system), multiple “best” responses and / or variations and one or more corresponding responses (e.g., when the variation is a follow-up type). In this and other ways, output can be provided in response to the original query using responses to variations of the original query, where the output directly answers the original query. Furthermore, responses to variations of the original query can be used to validate / prove one or more responses to the original query and / or one or more responses to other variations. For example, the accuracy of a “response” to the original query can be determined based on whether a positive response is provided for a variation of the original query. For example, based on whether other positive responses are provided for follow-up type variations, and / or based on whether one or more variations of equivalent, generalized, and / or language translation types yield positive similar responses (similar to the response to the original query). In this and other ways, unverified / unproven responses can be identified and not used in the provided output, and / or if used in the provided output, they can be marked as unverified (e.g., marked as "potentially forged").

[0012] In some implementations and / or cases, the search system returns multiple responses as responses to variations. In some other implementations and / or cases, the search system provides a single response as a response to a variation. In some of those implementations, the single response includes an "answer" (e.g., a response that the search system already considers to be an answer to the variation) or an indication that no answer is known. In other implementations, an indication that no answer is known may be that the search system lacks any response. The search system may be a search system operating across multiple domains or a search system specific to one or more specific domains (e.g., an online shopping domain). The response returned by the search system may be, for example, search results (e.g., fragments of content from a document and links to that document), answers (e.g., content considered an authoritative answer by the search system), images, videos, or knowledge graph entities, or an "empty" response (e.g., a "no answer" response). In some cases, the generated variations may be additionally or alternatively provided as output to the user (the user who submitted the original query) as a prompt for clarification, and the user interface input provided by the user in response to this prompt may be used as a "response" to the variation. Such user-provided responses can be used to influence the generation of further variations. For example, such user-provided responses can be used to generate context vectors, which are then passed to the generative model in further iterations of the generated variants.

[0013] In some implementations, multiple generative models can be generated, each trained on training data based on past query submissions from a unique set of users. For example, a first generative model can be generated based on training data from past query submissions by users with attributes A and B. A second generative model can be generated based on training data from past query submissions by users with attributes B and C. For queries submitted by users with attributes B and C (but not attribute A), the second generative model (not the first) can be selected to generate variants for that user—because user attributes B and C match the attributes used to train the second generative model. In this way, a generative model can be selected from multiple available generative models, such that the selected model is tailored to the user's attributes. This can lead to the generation of query variants that are more suitable for the user using the selected generative model. For example, very different variants can be generated for scientific researchers compared to, for example, freelance writers.

[0014] In some implementations, multiple generative models can be generated, each trained on training data based on past query submissions associated with specific attributes (e.g., specific user attributes, specific time attributes, and / or other attributes). For example, a first generative model can be generated based on training data that is based on past query submissions associated with online shopping tasks. Past query submissions can be identified based on whether they were submitted to an online shopping search system, based on the user's selection of shopping content associated with the submission (e.g., certain advertisements), based on shopping-centric search results, based on the user completing a transaction after submission, etc. A second generative model can be generated based on training data that is based on past query submissions associated with different specific attributes. For example, a second generative model can be generated based on training data that is based on past query submissions associated with a location-based task (e.g., to any location, any restaurant location, meeting location, etc.). Past query submissions can be identified based on whether they were submitted before and / or during travel to a location, based on submissions that are close to the planned calendar entry in time, etc. For a submitted user query, the user's task can be predicted, and a generative model corresponding to the predicted task can be selected to generate a variant of the submitted query. For example, if a user's calendar entries and / or electronic communications indicate that the user is traveling to a location (or will soon be traveling to that location), the second generative model in the previous example can be selected based on the model associated with the task of traveling to the location. In this way, a generative model can be selected from multiple available generative models, thereby allowing the selected generative model to be tailored to the user's task, such as the prediction task being performed or to be performed. This may result in generating query variants that are more suitable for the user's current task using the selected generative model. As described above and elsewhere in this document, in various implementations, the generative model can be a multi-task model, enabling the generation of various different types of query variants. Some of those various implementations enable the use of generative models to generate variants that expand the user query and enable the exploration of multiple paths for expanding the query. Such variants can be provided to the user simultaneously or sequentially (e.g., optionally, without first issuing a query based on such a variant) to allow the user to explore various paths for expanding the query. Alternatively, responses to such variations can be obtained from a search system and presented to the user so that the user can explore various responses to the extensions of the query.

[0015] Some of the implementations described herein can be used to generate variations of queries submitted by users who may have difficulty formulating them (e.g., due to physical impairment). For example, a query can be formulated by the user using gaze-guided (or other effortless) user interface input and query variations generated according to the techniques described herein. In this way, query variations can be generated and presented to the user without requiring the user to manually generate such variations.

[0016] As described herein, query-based tokens are applied to the generative model, and optionally, based on the application of additional input features, the generative model can be used to proactively generate variants of the query. In some of these implementations, additional input features may include attributes associated with the user submitting the query, time attributes, and / or other features. Attributes associated with the user may include, for example, the user's location (e.g., Louisville, Kentucky; at a "restaurant"; Southeastern United States), tasks associated with the user (e.g., cooking, repairing a car, planning a trip), and / or the weather at the user's location. Tasks associated with the user may be tasks that the user is currently engaged in or will be engaged in by the user. In some implementations, tasks are predicted based on various signals, such as stored user calendar entries, user electronic communications (e.g., chat messages or other communications sent to or by the user), past queries submitted by the user, etc. Time attributes may include, for example, the current time, the day of the current week, and / or the current date. In this way, based on the application of additional input features to the generative model, query variants utilizing the generative model can be generated that are personalized to the user and / or the current context.

[0017] In some implementations, based on content being assigned to one or more variants, a generative model can be used to generate variants of the query, and advertisements or other content, for the client device that generates the query. In some of those implementations, techniques such as those described herein can be used to customize the variants generated by the generative model according to the client device and / or the user of the client device. For example, the generative model can be selected based on one or more user attributes, and / or user-associated attributes can be provided as input to the generative model and used to generate variants.

[0018] In some implementations, generative models are used to generate variants (and / or variant tokens) of the original query at each of multiple time steps. In some of those implementations, whether and / or which variant is generated at a given time step can be based on current state features. Current state features may include, for example, features based on: the search system response to the original query; the search system response to a variant of the original query generated at a previous time step; a variant of the original query generated at a previous time step; the user's response to a variant of the original query (e.g., a clarifying variant offered to the user as a hint); and / or the original query. In this way, the generation of variants of the query during a session can be dynamically influenced based on previously generated variants of the query during the session, responses to previously generated variants, and / or the original query. For example, in some implementations, one or more of such current state features can be used to determine whether another variant should be generated, or alternatively, can be used to determine whether a response to a previous variant (and / or to the original query) should be provided instead of generating another variant in response to the original query. Similarly, in some other or alternative implementations, one or more such current state features can be applied (directly or indirectly) as input to the generative model to influence the generation of variants at that time step. For example, a vector summary of the current state features can be generated and applied as input to the generative model to influence the generated variants.

[0019] In some implementations, a trained control model is used at each of multiple time steps to determine whether to generate variants and / or to provide features as input to the generative model to influence variant generation at that time step. For example, the trained control model could be a feedforward neural network model or a recurrent neural network (RNN) model. Current state features can be applied as input to the trained control model to generate values ​​on the model indicating whether to generate additional variants and / or features to be provided to the generative model to influence variant generation (if another variant is generated) (e.g., a vector summary of the current state features and / or a reward signal). In this way, in a participant-commentator environment, the control model can act as the "commentator," while the generative model can act as the "participant." Therefore, the trained control model can be used to determine whether to generate additional variants and / or features to influence such generation based on observed current state features. In this way, the trained control model can control the number of additional variants generated for a given query. This control can result in a varying number of variants generated depending on the query, as the control model dynamically determines the number of iterations for generating variants for a given query based on, for example, variants generated in previous iterations of a given query and / or responses to such variants. It should be understood that this dynamic control can typically lead to the generation of a relatively large number (e.g., greater than 5, greater than 10, or greater than 15) of variants and / or the consideration of a relatively large number of responses to such variants.

[0020] In some implementations, the control model and / or generative model can be trained, at least in part, based on reinforcement learning. In some of those implementations, the control and generative models are trained separately but combined. When training the control and / or generative models based on reinforcement learning, generated variants can be submitted to a search system, and responses from the search system (and optionally, the absence of responses) can indicate rewards. For example, for a response to a query variant, i.e., a “response” response, a reward proportional to (or otherwise related to) the quality of the response can be assigned (e.g., as indicated by a response score provided by the search system for a “response” response). In some of those examples, no reward will be assigned if no response is provided in response to a query variant and / or when the response is considered (e.g., based on the output from the search system) not a “response” response. In other words, only the final “response” response will be rewarded, and intermediate actions will be updated based on such rewards (e.g., using Monte Carlo Q-learning methods). In this way, Q-feature learning or other reinforcement features can be performed based on rewards conditioned on responses provided by the search system with which the reinforcement learning is conducted. In the reinforcement learning implementation described in this paper, the state at a given time step is indicated by one or more state features (e.g., those described above), and the action can be a query variant (i.e., generating an additional query variant) or providing a "response". Each action in the action space can be paired with a string that defines the corresponding question or "response".

[0021] In some implementations, a method implemented by one or more processors is provided, comprising receiving an original query generated based on user input via a user interface of a client device. The method further comprises applying a token of the original query and one or more attributes associated with the user as input to a trained generative model. The trained generative model is a sequence-to-sequence deep neural network model having one or more storage layers. The method further comprises generating at least one variant of the original query based on the application of the token and one or more attributes to the trained generative model. The method further comprises generating an output based on at least one variant and at least one search system response to the at least one variant. The method further comprises providing output for presentation via a client device in response to the original query.

[0022] In some implementations, a method implemented by one or more processors is provided, comprising: receiving an original query; applying a token of the original query as input to a trained generative model; and generating multiple variants of the original query based on the application of the token of the original query to the trained generative model. The original query may be generated based on user input via a user interface of a client device. Each generated variant differs from the original query, and generating the variant includes producing the variant based on learned parameters of the trained generative model. The trained generative model is trained to generate multiple types of query variants, and the generated variants include a first variant and a second variant, the first variant being a first type of multiple types of query variants, and the second variant being a second type of multiple types of query variants. The method further comprises: generating output based on at least one of the multiple variants and / or based on at least one search system response to at least one of the multiple variants; and providing the output for presentation via a client device in response to the original query.

[0023] In some implementations, a method implemented by one or more processors is provided, the method comprising receiving an original query generated based on user input via a user interface of a client device. The method further comprises: training the trained generative model based on past query submissions from a set of users having one or more attributes common to the user; and selecting a trained generative model from a plurality of trained generative models. The method further comprises: applying a token of the original query as input to the selected trained generative model; generating at least one variant of the original query based on the token of the original query applied to the trained generative model; and generating an output based on the at least one variant and / or at least one search system response to the at least one variant. The method further comprises: providing the output for presentation via the client device in response to the original query.

[0024] In some implementations, a method implemented by one or more processors is provided, comprising: receiving an original query; applying a token of the original query as input to a trained generative model; and generating a variant of the original query based on the input on the trained generative model. The original query may be generated based on user input via a user interface of a client device. The variant generated on the trained generative model differs from the original query, and generating the variant includes producing the variant based on learned parameters of the trained generative model. The method further comprises: submitting the variant of the query to a search system to determine a variant response to the variant of the query; applying additional input to the trained generative model; and generating additional variants of the original query based on the additional input. The additional input applied to the trained generative model includes at least one of the following: a token of the original query and a variant token of the variant of the original query. The generated additional variant differs from the variant and the original query, and generating the additional variant of the original query includes producing the additional variant based on learned parameters of the trained generative model. The method further comprises submitting the additional variant of the original query to a search system to determine an additional variant response to the additional variant of the original query. The method further includes: generating output based on variant responses and / or additional variant responses; and providing output for presentation via a client device in response to the original query.

[0025] In some implementations, a method implemented by one or more processors is provided, the method including receiving an original query generated based on user input via a user interface of a client device. The method further includes determining a prediction task for the user and applying a token of the original query and one or more task attributes for the user's prediction task as input to a trained generative model. The method also includes generating at least one variant of the original query based on the application of the token and one or more task attributes to the trained generative model. The method further includes: generating output based on at least one variant and / or at least one search system response to at least one variant; and providing the output for presentation via the client device in response to the original query.

[0026] In some implementations, a method implemented by one or more processors is provided, the method including receiving an original query generated based on user input via a user interface of a client device. The method also includes determining a prediction task for the user. The method further includes selecting a trained generative model from a plurality of trained generative models based on a trained generative model trained on past query submissions associated with the prediction task. The method also includes: applying a token of the original query as input to the selected trained generative model; generating at least one variant of the original query based on the token applied to the trained generative model; and generating an output based on the at least one variant and / or at least one search system response to the at least one variant. The method also includes providing the output in response to the original query.

[0027] The various implementations disclosed herein may include one or more non-transitory computer-readable storage media storing instructions executable by a processor (e.g., a central processing unit (CPU), graphics processing unit (GPU), and / or tensor processing unit (TPU)) to perform one or more methods as described herein. Other various implementations may include a system of one or more computers including one or more processors operable to execute the stored instructions to perform one or more methods as described herein.

[0028] It should be understood that all combinations of the foregoing and additional concepts described in more detail herein are considered part of the subject matter disclosed herein. For example, all combinations of the claimed subject matter appearing at the end of this disclosure are considered part of the subject matter disclosed herein. Attached Figure Description

[0029] Figure 1 This is a block diagram of an example environment that can implement the implementation disclosed in this article.

[0030] Figure 2 An example of training a generative model based on the implementation disclosed in this paper is shown.

[0031] Figure 3 An example is shown of using a generative model to generate one or more variations of a query.

[0032] Figure 4 Another example is shown of using a generative model to generate one or more variations of a query, where a control model is used to control the generation of variations.

[0033] Figure 5 This is a flowchart illustrating a method for training a generative model according to the implementation disclosed herein.

[0034] Figure 6This is a flowchart illustrating a method for generating one or more variations of a query using a generative model.

[0035] Figure 7 This is a flowchart illustrating a method for generating one or more variations of a query using a generative model, wherein a control model is used to control the generation of variations.

[0036] Figure 8A and Figure 8B Each example graphical user interface is shown for providing output based on a variant generated according to the implementation disclosed herein.

[0037] Figure 9 An example architecture for a computing device is shown.

[0038] Specific implementation method

[0039] Figure 1 An example environment in which the implementation disclosed herein can be implemented is shown. Figure 1 Example environments include client device 106, query system 110, search system 140, generative model training engine 120, and training instance engine 122. Such systems and engines can each be implemented in one or more computing devices that communicate, for example, via a communication network. The communication network can include a wide area network (WAN) such as the Internet, one or more intranets, and / or one or more bus subsystems. The communication network can optionally utilize one or more standard communication technologies, protocols, and / or inter-process communication technologies.

[0040] Query system 110, search system 140, generative model training engine 120, and training instance engine 122 are example components in which the techniques and / or systems, components, and techniques described herein can be implemented and interfaced with the example components. Figure 1 The operations performed by one or more of systems 110, 140 and engines 120, 122 can each be distributed across multiple computer systems. In some implementations, one or more aspects of systems 110, 140 and engines 120, 122 can be combined in a single system, and / or one or more aspects can be implemented on client device 106. For example, in some of those implementations, aspects of query system 110 can be combined with aspects of search system 140.

[0041] A user of client device 106 can formulate a query by providing user interface input via one or more user interface input devices of client device 106. Client device 106 submits the query to query system 110. In some cases, the query is in text form. In other cases, the query may be submitted in audio and / or other forms and converted into text form by query system 110 (or other components).

[0042] For a received query, query system 110 generates one or more variations of the received query and provides the output to client device 106, wherein the output is based on one or more variations. In some implementations, the output includes one or more variations to be provided as suggested alternative variations for the user to consider. In some implementations, the output additionally or alternatively includes content based on one or more responses from search system 140, wherein the responses are based on submitting one or more variations to search system 140. Search system 140 may determine the response based on access to one or more resources 166 and may utilize various techniques, such as conventional information retrieval techniques. Response-based content may be, for example, a graphical and / or audible “response” or other search results based on the response (e.g., similar to it). In the case of providing response-based content, query system 110 may provide the content directly to client device 106, or search system 140 may provide the content to client device 106. In some implementations, query system 110 and search system 140 may optionally be controlled by the same party and / or work collaboratively with each other. Additional and / or alternative outputs can be provided based on the generated variants (such as ads assigned to the generated variants in one or more databases).

[0043] exist Figure 1 In the query system 110, there are variant engine 112 and controller engine 114. In some implementations, one or more aspects of variant engine 112 and controller engine 114 may be combined and / or implemented in a component separate from query system 110, such as client device 106. In some implementations, controller engine 114 may be omitted.

[0044] Variant engine 112 utilizes one or more trained generative models 152 to generate one or more query variants for a submitted query. In some implementations, variant engine 112 includes one or more CPUs, GPUs, and / or TPUs operating on the trained generative models 152. Variant engine 112 generates variants of the submitted query by applying a token of the query as input to one of the generative models 152 and generating variants on the generative model based on that input. In many implementations, when generating variants, variant engine 112 further applies additional input features as input to the generative model and generates variants based on the additional input features.

[0045] In some implementations, additional input features may include attributes associated with the user who submitted the query, time attributes, and / or other features. For example, when generating a variant of the original query, variant engine 112 may apply the token of the original query, attributes of the user who submitted the query (e.g., the user's location, the task performed by the user), and time attributes (e.g., day of the current week, current time of day) as input to one of the generative models 152, and generate variants on the generative model based on the applied inputs.

[0046] In some implementations, the additional input features applied at a given iteration for generating variants of the original query may additionally or alternatively include features based on variants of the original query generated at previous iterations and / or features based on the search system response to such variants. For example, when generating variants for the original query, variant engine 112 may generate variants at each of multiple time steps. At a given time step, variant engine 112 may apply features as input to one of the generative models 152 based on: the search system response to the original query; the search system response to variants of the original query generated at previous time steps; variants of the original query generated at previous time steps; and / or the original query. In this way, variant generation at a given time step may be influenced by previously generated variants, responses to previously generated variants, and / or the original query.

[0047] In some implementations, additional input features applied in a given iteration of generating variants of the original query may additionally or alternatively include type values. For example, in some implementations, one of the generative models 152 may be a "multi-task" model, as it is trained to be able to generate any of a variety of query variants. In some of those implementations, the variant engine 112 may be applied as input to one of the generative models 152 to indicate the type of query variant to be generated. The type of query variant may include, for example, an equivalent query, a follow-up query, a generalized query, a canonicalized query, a language translation query, and / or an implication query. In some implementations, the variant engine 112 selects different type values ​​at each iteration in multiple iterations of generating variants, thereby utilizing the same generative model to generate multiple variants of completely different types.

[0048] In some implementations, the variant engine 112 can access multiple generative models 152, and the variant engine 112 selects a subset of one or more of the multiple generative models 152 to generate variants for a submitted query based on one or more parameters. For example, multiple generative models 152 can be provided, wherein each generative model is trained based on training data based on past query submissions from a unique set of users. For example, a first generative model can be generated based on training data based on past query submissions from users with attributes A and B. A second generative model can be generated based on training data based on past query submissions from users with attributes B and C. For a submitted query from a user with attributes B and C (but not A), the variant engine 112 can select the second generative model when generating a variant of that query (without needing to select the first generative model), because user attributes B and C match the attributes used to train the second generative model.

[0049] exist Figure 1 The diagram also illustrates a generative model training engine 120 and a training instance engine 122. The training instance engine 122 generates training instances and stores them in a training instance database 164. For example, the training instance engine 122 can generate multiple training instances based on a query submission database 162, which stores a large number of past user query submissions. The generative model training engine 120 trains a generative model 152 based on the training instances stored in the database 164. As described herein, in some implementations, reinforcement learning techniques that do not depend on the training instances in the training instance database 164 can optionally be used to further train one or more of the generative models 152. The following section discusses... Figure 2 Additional descriptions of the implementations of engines 120 and 122, as well as databases 162 and 164, are provided in the relevant descriptions.

[0050] When provided, controller engine 114 works in conjunction with variant engine 112 and: controls whether variant engine 112 generates variants; and / or generates parameters that affect variant generation and provides the parameters to variant engine 112. Controller engine 114 may optionally utilize one or more trained control models 154 to control whether variant engine 112 generates variants and / or generate parameters that affect variant generation. In some implementations, variant engine 112 includes one or more CPUs, GPUs, and / or TPUs operating on the trained control model 154.

[0051] In some implementations, for a submitted query, controller engine 114 determines whether variant engine 112 should generate any variants for the submitted query. For example, controller engine 114 may make this determination based on the submitted query itself and / or on the response (if any) from search system 140 to the submitted query. For example, controller engine 114 may determine to generate a variant only if search system 140 does not return a response or any returned response is of insufficient quality (e.g., has a score provided by the search system that does not meet a threshold). In some of those implementations, controller engine 114 applies a token of the submitted query and / or features of the response to the submitted query to one of control models 154 and generates an output on control model 154 indicating whether a variant should be generated. In some other or alternative implementations, controller engine 114 applies a token of the submitted query and / or features of the response to one of control models 154 and generates an output on control model 154 provided to variant engine 112, which is then applied as input to the generative model when generating variants (thus affecting the generation of variants).

[0052] As described herein, in some implementations, variant engine 112 generates variants of the submitted query at each of multiple time steps. In some of those implementations, controller engine 114 determines when variant generation should stop. In other words, whether variant engine 112 generates a variant at a given time step may depend on authorization from controller engine 114. Furthermore, for each time step, controller engine 114 can provide features that affect variant generation at that time step. In determining whether variant generation should stop and / or in generating features that affect variant generation, controller engine 114 may utilize at least one of one or more control models 154.

[0053] As an example, controller engine 114 can be applied as input to one of the control models 154 based on: a search system response to the original query; a search system response to a variant of the original query generated by variant engine 112 at a previous time step; a variant of the original query generated by variant engine 112 at a previous time step; and / or the original query. Controller engine 114 can generate an output on the control model based on the applied input and use that output to determine whether to instruct variant engine 112 to generate additional variants or to stop variant generation. When variant generation stops, controller engine 114 can alternatively provide the previously generated variants and / or responses to the previously generated variants as output in response to the submitted query. In this way, in a participant-commentator environment, controller engine 114 can act as a "commentator," while variant engine 112 can act as a "participant." See below for reference. Figure 4 Additional description of the implementation of the interaction between controller engine 114, control model 154, and variant engine 112.

[0054] Turning Figure 2 This shows an example of generative model 152A trained on generative model 152. Training instance 164A is retrieved from training instance database 164. Training instance engine 122A ( Figure 1 ) can be based on, for example, a query previously submitted by the user and stored in the submitted query database 162 ( Figure 1 A pair of queries is used to generate training instance 164A. As an example, this pair of queries could include an earlier query from the user, “Did Roger Moore drive an Aston Martin in The Persuader?”, and a later query from the user, “What car did Roger Moore drive in The Persuader?” (which is of the same type relative to the earlier query). As another example, this pair of queries could include an earlier query from the user, “Did Leonardo da Vinci paint the Mona Lisa?”, and a later query from the user, “Who commissioned Leonardo da Vinci to paint the Mona Lisa?” (which is of the follow-up type relative to the earlier query).

[0055] Training instance 164A includes training instance input, which includes queries (e.g., the earlier submitted query of the pair), attributes, and type. These attributes may include, for example, attributes of the user who submitted the query, time attributes of the query (e.g., the day of the week in which it was submitted), characteristics of the search system's response to the query, etc. The type may be a type value indicating which type of variant is included in the training instance output. In some implementations, the type may be specified manually, or it may be inferred by training instance engine 122 based on characteristics of the query pair used to generate training instance 164A (e.g., the size of the time interval between the query submissions of the query pair, a comparison of the query system's responses to the query pair). Training instance 164A also includes training instance output, which includes variants (e.g., the later submitted one of the pair).

[0056] The generative model training engine 120 applies the training instance inputs of the training instances as inputs to the generative model 152A. The generative model training engine 120 also generates outputs on the generative model 152A based on the applied inputs and the current learned parameters of the generative model 152A. The generative model training engine 120 further generates gradients based on a comparison of the generated outputs with the training instance outputs of training instance 164A, and updates the generative model 152A based on these gradients (e.g., backpropagating gradients across the entire generative model 152A).

[0057] When generating output based on the applied input, the generative model training engine 120 may apply all or part of the input to the encoder layer 153A of the generative model 152A and generate encoding on the encoder layer 153A. For example, the token of the original query of the input may be applied to the encoder layer 153A. The engine 120 may further apply the encoding to the decoder layer 154A of the generative model 152A and generate decoding of the encoding on the decoder layer 154A. The engine 120 may then apply the generated encoding to the softmax layer 155A and generate output on the softmax layer 155A based on the application of the generated encoding. In some implementations, the engine 120 applies the attributes and / or type of the input to other layers and / or applies it as a "side input" to one of the encoder layer 153A, decoder layer 154A, and / or softmax layer 155A. In some of those implementations, the engine 120 applies the attributes and / or type to other layers downstream of the encoder layer 153A but upstream of the decoder layer 154A.

[0058] Although Figure 2Only a single training instance 164A is shown, but it should be understood that many other training instances will be utilized in training the generative model 152A. Note that in some implementations, a single training instance 164A and other training instances are selected such that the generative model 152A is trained specifically to suit certain attributes. For example, the generative model 152A can be trained by selecting only training instances (or biased training instances) generated based on past submissions by users with specific attributes. For example, in such a selection, user attributes explicitly included in the training instance input can be utilized. Moreover, for example, the generative model 152A can be trained by selecting only training instances (or biased training instances) associated with certain task attributes. For example, selection can be biased towards queries submitted as associated with certain tasks performed (or to be performed). It should also be noted that in some implementations, the generative model 152A is trained using training instances that include multiple different “types” in the training instance input. As described herein, this enables the generation of multi-task models that can produce multiple different types of variants and can be biased towards a specific type at runtime by applying the corresponding type values ​​as input.

[0059] Figure 3 Examples of using generative models to generate one or more variations of queries are shown. Figure 3 In this implementation, the user's original query and attributes are transmitted from client device 106 to variant engine 112. In some other implementations, client device 106 may not transmit one or more (e.g., all) attributes with the query, or client device 106 may not transmit one or more (e.g., all) attributes at all. For example, user attributes may be stored remotely relative to the client device. For example, attributes may be stored remotely and based on the user's past interactions (e.g., via other client devices), and accessed by variant engine 112 from the remote storage.

[0060] Variant engine 112 utilizes at least one of generative models 152 to generate one or more variants of the original query. When generating one or more variants, variant engine 112 may utilize attributes to select one of the generative models 152 and / or may apply one or more attributes as input to one of the generative models. Variant engine 112 may further apply tokens of the original query to the generative models and / or other features (e.g., previously generated variants, where multiple variants are generated iteratively).

[0061] In some implementations, variant engine 112 transmits variants as output based on the original query to client device 106. In some implementations, variant engine 112 additionally or alternatively provides one or more variants to search system 140, which determines one or more responses to the variants (e.g., a single response search result or multiple search results) and transmits the responses as output based on the original query to client device.

[0062] Figure 4 Another example is shown that utilizes a generative model to generate one or more variations of the query. Specifically, Figure 4 An example is shown where a control model is used to control the generation of variants.

[0063] exist Figure 4 In this process, the user's original query and attributes are transmitted from the client device 106 to the controller engine 114. (And...) Figure 3 Similarly, in some other implementations, client device 106 may not transmit one or more (e.g., all) attributes with the query, or client device 106 may not even transmit one or more (e.g., all) attributes at all.

[0064] In some implementations, controller engine 114 utilizes one or more control models 154 to determine whether to generate a variant of the original query. For example, controller engine 114 may apply a token of the original query, the search system response to the original query, and / or user attributes to one of the control models 154 to determine whether to generate a variant. In some other implementations, controller engine 114 may, by default, determine that at least one variant or the original query should be generated.

[0065] The controller engine 114 provides the variant engine 112 with a reward signal determined based on the output on one or more control models 154, and also provides the current state. The current state may include, for example, the original query, user attributes and / or feature vectors based on one or both of them—wherein the feature vectors are also based on the output on one or more control models 154.

[0066] The variant engine utilizes at least one of the generative models 152 to generate one or more variants of the original query. When generating variants, the variant engine 112 may utilize the provided state and optionally a reward signal. For example, the variant engine 112 may apply the reward signal to a learned reward function to determine the reward in generating the query variant. The variant engine 112 provides the variants to the search system 140. In response, the search system 140 generates one or more responses and provides the responses to the controller engine 114.

[0067] Controller engine 114 uses one or more variants generated so far and / or their corresponding responses to determine whether variant engine 112 should generate another variant. For example, controller engine 114 may apply the tokens and / or characteristics of the corresponding responses of one or more variants generated so far as input to one of the control models 154, generate an output on the control model based on that input, and use that output to determine whether another variant should be generated. In some implementations, controller engine 114 further applies the token of the original query, the search system response to the original query, and / or user attributes as part of the input.

[0068] If controller engine 114 determines that additional variants should be generated, it can provide (e.g., updated based on one or more variants generated so far and / or corresponding responses to one or more variants) an updated reward signal and an updated current state. Variant engine 112 can then generate one or more additional variants, provide the variants to search system 140, and again provide corresponding responses. Controller engine 114 can then determine again whether additional variants should be generated based on the additional variants and corresponding responses.

[0069] If, at a given iteration, controller engine 114 determines that no further variants should be generated, it transmits one or more search system responses and / or one or more generated variants to client device 106 as output based on the original query. For example, controller engine 114 may store all provided responses and provide only one as the response output (e.g., the highest quality response or the highest quality response confirmed by other responses). As another example, controller engine 114 may provide multiple responses (e.g., N best responses, various sets of responses).

[0070] In some implementations, reinforcement learning can be used to train the control model 154, the generative model 152, the controller engine 114, and / or the variant engine 112. In some of those implementations, other techniques can be initially used to train the control model 154 and / or the generative model 152, and then refined through reinforcement learning. For example, the generative model 152 can be as follows: Figure 2 They were initially trained as described, and further trained through reinforcement learning.

[0071] In some implementations of the participant-commentator algorithm, the controller engine 114 and control model 154 can be considered "commentators," while the variant engine 112 and generative model 152 are considered "participants." Typically, participants generate variants and use them to probe the environment. This environment can be, for example, a search system 140. Typically, the commentator collects evidence from this environment (e.g., responses such as response strings or their sorted lists) to generate global behavior / decision d to maintain the global state s and provide the participants with reward signals r and context c.

[0072] The behavior of participants and commentators can be driven by reinforcement at two different timescales. Participants can operate at a finer timescale (indexed by t′). At each step, the participant generates the next variant based on the context. The commentator accumulates evidence from that environment into a global state s. In some cases, the state will contain at least the original query, the generated variant, and observations (e.g., the search system response to the generated variant), as well as a vector summary h, s = ({q t o t} 1..T h t Given a global state, the commentator makes a global decision d at each time step to either issue a response or continue the cycle of variant generation and accumulating more evidence. The commentator also provides participants with context and reward signals, which are used to modulate variant generation. The commentator directly interacts with the state-action pair "Q-function" Q(s). t d t The value of the Q-function is used for modeling. This value is passed to the participants as a reward signal. The Q-function is trained using a global reward defined on the response (e.g., the response to the original query) and the sequence of decisions made, d. The separation of time scales allows for separate modeling of the two tasks, variant generation and global decision making, but they are trained jointly to optimize end-to-end performance.

[0073] The final state is reached when commenters respond instead of continuing the cycle of variant generation and further evidence accumulation. The participant's action space can be defined as: A := {(a, ...} <w>):a∈{question,answer}, <w>∈Strings}, where a can be used to probe the environment or emit a response using variants. This action is related to the string. <w>The pair, where the string defines the variant or response (the emitted response). In some implementations, the "probe the environment with a variant" action does not receive a reward, while the "emit a response" action receives a reward proportional to the quality of the response. Commentators can learn the Q-function, which converts an action (a, b, c) into a string. <w>Mapping the current state to the expected reward E[Gs]. In the case where only "issuing a response action" is rewarded, the expected reward can be represented as E[γ]. κ R], where [0, 1] is the discount, k is the number of iterations to the final state, and R is the final reward.

[0074] Monte Carlo Q-learning can be used to train the Q-function. Variants can be sampled until the final state is reached and the reward determined, and all intermediate predictions of the Q-function can be updated to γ. κ Participants generate variants and may include, for example, sequence-to-sequence models that take the original query, the latest variant, and potentially additional features modulated based on variants and response history as input, and return one or more additional variants. Participants can be trained using the Monte Carlo policy gradient method. The set of responses received from the environment acts as a memory of the responses seen so far. This can be used to feature the participants and the Q-function, and / or allow commenters to return responses seen in any intermediate iterations.

[0075] Turn now Figure 5 A flowchart illustrating method 500 for training a generative model according to various implementations disclosed herein is provided. For convenience, the operations of the flowchart are described with reference to a system performing the operations. This system may include one or more components, such as one or more processors (e.g., one or more CPUs, one or more GPUs, and / or one or more TPUs). Although the operations of method 500 are shown in a specific order, this is not intended to be limiting. One or more operations may be reordered, omitted, or added.

[0076] In box 552, the system selects a set of training instances. For example, when training the generative model as a multi-task model in method 500, the system may select this set such that it includes training instances demonstrating multiple types of variant generation. Furthermore, for example, if the generative model is additionally or alternatively trained for a specific user group, the system may select this set such that the training instances include only or a large number (e.g., more than half, more than 70%) of the training instances based on queries previously submitted by users conforming to said specific group. Moreover, for example, if the generative model is additionally or alternatively trained for a specific task, the system may select this set such that the training instances include only or a large number (e.g., more than half, more than 70%) of the training instances based on queries previously submitted associated with that specific task.

[0077] In box 554, the system selects a training instance from the group.

[0078] At box 556, the system applies the training instance input of the training instance as input to the generative model. The training instance input may include, for example, terms of the original query, attributes (e.g., attributes of the user who submitted the original query), and type values ​​(which indicate the type of variant of the original query).

[0079] In box 558, the system generates variants on the generative model based on the applied training instance inputs.

[0080] At box 560, the system determines the error of the training instance based on a comparison between the generated variant and the training instance output (i.e., the variant indicated in the training instance output).

[0081] At box 562, the system updates the generative model based on the error. For example, the error could be backpropagated on the generative model to update its gradient.

[0082] At box 564, the system determines whether there are any other unprocessed training instances in the group. If so, the system proceeds to box 554 and selects an additional training instance. Then, the system executes boxes 556, 558, 560, and 562 based on the additional training instance.

[0083] If at the iteration in box 564 the system determines that there are no other unprocessed training instances in the group (or other training criteria have been met), the system proceeds to box 566, where training ends.

[0084] Although Figure 5 Specific non-batch training methods are illustrated, but it should be understood that batch training can be used alternatively during training (e.g., identifying errors based on a batch of training instances and backpropagating those errors). Furthermore, it should be understood that in various implementations, the generative model trained based on method 500 can be further trained according to the techniques disclosed herein. For example, reinforcement learning techniques can be used to further train the generative model, and it can be further trained separately from but in conjunction with a separate control model. Moreover, in the case of generating multiple generative models, method 500 can be repeated in box 552 with different selection criteria to generate additional models.

[0085] Turn now Figure 6 A flowchart is provided illustrating method 600 for generating one or more variations of a query using a generative model, according to various implementations disclosed herein. For convenience, the operations of the flowchart are described with reference to a system performing the operations. This system may include one or more components, such as one or more processors (e.g., one or more CPUs, one or more GPUs, and / or one or more TPUs). Although the operations of method 600 are shown in a specific order, this is not intended to be limiting. One or more operations may be reordered, omitted, or added.

[0086] At box 652, the system receives the query.

[0087] In box 654, the system selects a generative model from multiple candidate generative models. In some implementations, the system selects the generative model based on one or more attributes of the user who submitted the query in box 652. For example, the system may select the generative model based on storing generative models associated with attributes that match one or more attributes of the user. For example, it may be based on training instances trained on past query submissions of users with those attributes to be stored in association with such attributes. In some implementations, box 654 may be omitted (e.g., only a single generative model is available).

[0088] At box 656, the system applies the query token and additional values ​​as input to the generative model. Various additional values ​​can be applied, such as attributes of the user submitting the query, time attributes, and / or attributes of the search system's response to the received query. As a specific example, additional values ​​may include predictive task attributes of the user submitting the query. For example, these predictive task attributes can be predicted based on content recently viewed on a computing device, stored calendar entries of the user, and / or the user's electronic communications.

[0089] In box 658, the system generates one or more variants on the generative model based on the applied input.

[0090] At box 660, the system determines whether to generate an additional variant. In some implementations, the system determines whether to generate an additional variant based on the characteristics of the variants generated so far and / or based on the search system's response to the variants generated so far. For example, the system may determine whether to generate an additional variant based on whether a response to one or more variants generated so far was found through a quality metric of the search system and / or the response. For example, if no response was found and / or if the quality metric failed to meet one or more quality criteria, the system may generate an additional variant.

[0091] If the system determines that an additional variant is generated during an iteration of box 660, the system proceeds to box 662 and updates one or more additional values, which will be used as input to the generative model in subsequent iterations of box 656. For example, the system may update the additional values ​​to reflect one or more variants generated in the latest iteration of box 658, to reflect the response to one or more variants, and / or to change the type value used for the next iteration of box 658. The system then performs another iteration of box 656 with the updated additional values, and then proceeds to boxes 658 and 660.

[0092] If the system determines in the iteration of box 660 that no additional variants are generated, the system proceeds to box 664 and provides output based on one or more of the generated variants. The output may include one or more variants and / or the system response to the search for one or more variants.

[0093] Figure 7 This is a flowchart illustrating method 700 for generating one or more variations of a query using a generative model, wherein a control model is used to control the generation of variations. For convenience, the operations of the flowchart are described with reference to a system performing the operations. This system may include one or more components, such as one or more processors (e.g., one or more CPUs, one or more GPUs, and / or one or more TPUs). Although the operations of method 700 are shown in a specific order, this is not intended to be limiting. One or more operations may be reordered, omitted, or added.

[0094] In box 752, the system receives a query.

[0095] At box 754, the system generates control outputs on the control model based on the current state. For example, the current state may be based on the token of the current query, the search system response to the current query, and / or other features.

[0096] In box 756, the system determines whether to generate a variant of the received query based on the control output. In some implementations, the initial iterations in boxes 754 and 756 can be omitted. In other words, in those implementations, the system can determine that a variant should always be generated (e.g., to check the validity of the search system's response to the received query).

[0097] If the system determines that no variants are generated in the iteration of box 756, the system proceeds to box 766 and provides output based on the current search system response and / or the variants generated.

[0098] If the system determines to generate a variant during the iteration of box 756, the system proceeds to box 758.

[0099] In box 758, the system determines a reward signal and / or context based on the control output generated in the latest iteration in box 754. The reward signal may be based on the Q-function learned herein, and the context may include, for example, the current state and / or a vector summary of the current state.

[0100] In box 760, the system generates variants on the generative model based on the received query and the reward signal and / or context of box 758.

[0101] In box 762, the system determines the response to the variant generated in box 760. For example, the system may submit the variant to a search system and receive responses to that variant from the search system. In some cases, the search system may not return a response and / or generate "empty", each of which indicates that no response (e.g., an answer) is available.

[0102] In box 764, the system updates the current state based on the variant and the response to the variant. The system then returns to box 754 and generates a control output on the control model based on the current state, which includes the update from box 764. In this way, in subsequent iterations of box 764, previously generated variants and responses (i.e., variants and responses generated in previous iterations of boxes 760 and 762) can be considered in the next iteration of box 754. The system then returns to box 756 and determines, based on the control output, whether to generate another variant of the received query. When the system determines to generate another variant, note that the reward signal and context provided at the next iteration of box 758 can similarly depend on the previously generated variants and responses (i.e., variants and responses generated in previous iterations of boxes 760 and 762). In this way, variant generation in the next iteration of box 760 is thus influenced by previously generated variants and responses.

[0103] Turn now Figure 8A and Figure 8B Exemplary graphical user interfaces 800A and 800B are shown for providing output based on variations of the implementation disclosed herein. Graphical user interfaces 800A and 800B may be presented at client device 106 (e.g., in a browser executing at client device 106 and / or in another application executing at client device 106).

[0104] exist Figure 8A In the example, the user provides the query 891A, "Did Leonardo da Vinci paint the Mona Lisa?". In response, output is provided, including response 892A and two variants 893A. The two variants 893A can be generated according to the implementation disclosed herein. In some implementations, each variant is optional, and in response to selection, the corresponding variant is submitted as a new query. In some implementations, response 892A is also based on the variants generated according to the implementation disclosed herein. For example, in some cases, response 892A may be a response to a variant of query 891A (a variant different from variant 893A), and / or, response 892A may be a response to query 891A, but validated based on the responses to the variants of the query (e.g., by ensuring that these variants also generate positive responses).

[0105] exist Figure 8B In the query 891B, the user has provided the query "Did Michelangelo paint the Mona Lisa?". In response, output 892B is provided, including a "No" response. Figure 8B Box 895B may optionally not be provided for display, but instead be presented as an example of a variant that can be generated according to the techniques described herein to produce a "No" response 892B. Box 895B displays the original query (represented by "O") and includes "Y" in parentheses to indicate that the search system generated a response in response to the original query. For example, the response might be "Yes, Michelangelo did paint the Mona Lisa." However, instead of providing a response, multiple variants are generated as "follow-up" variants to verify the accuracy of the response to the original query. In a specific variant VI, V2 and V3 are generated. As indicated by "N" in parentheses, the search system generates a "No Response" response in response to each of those follow-up variants. Given that no response is available for those multiple follow-ups, the controller engine can determine that the "response" to the original query is incorrect (because the follow-ups did not result in any response). As a result, the controller engine can provide a "No" response 892B.

[0106] Although Figure 8A and 8B Examples of graphical interfaces are presented, but it should be understood that queries may be received alternatively or in lieu of verbal input from the user, and / or variations and / or responses may be provided alternatively or in lieu of verbal input from the user to be audibly presented to the user via the client device.

[0107] Where the systems described herein collect or may utilize personal information about users, they may provide users with the following opportunities: control whether programs or features collect user information (e.g., information about a user's social networks, social behaviors or activities, occupation, user preferences, or the user's current geographic location), or control whether and / or how content more relevant to the user is received from content servers. Furthermore, some data may be processed in one or more ways before storage or use to remove personally identifiable information. For example, a user's identity may be processed to the point that no personally identifiable information can be determined for that user, or the user's geographic location may be generalized where geographic location information is obtained (e.g., to the level of city, ZIP code, or state), making it impossible to determine the user's specific geographic location. Therefore, users can control how information about themselves is collected and / or how that information is used.

[0108] Figure 9 This is a block diagram of an example computing device 910, which may optionally be used to perform one or more aspects of the techniques described herein. The computing device 910 includes at least one processor 914 (e.g., CPU, GPU, and / or TPU) that communicates with a plurality of peripheral devices via a bus subsystem 912. These peripheral devices may include a storage subsystem 924, including, for example, a memory subsystem 925 and a file storage subsystem 926, a user interface output device 920, a user interface input device 922, and a network interface subsystem 915. The input and output devices allow users to interact with the computing device 910. The network interface subsystem 915 provides an interface to an external network and is coupled to corresponding interface devices in other computing devices.

[0109] User interface input device 922 may include a keyboard, pointing devices such as a mouse, trackball, touchpad, or graphics tablet, a scanner, a touchscreen included in a display, audio input devices such as a voice recognition system, a microphone, and / or other types of input devices. Generally, the term "input device" is used to include all possible types of devices and the manner in which information is input into computing device 910 or a communication network.

[0110] User interface output device 920 may include a display subsystem, a printer, a fax machine, or a non-visual display such as an audio output device. The display subsystem may include a cathode ray tube (CRT), a flat panel device such as a liquid crystal display (LCD), a projection device, or other mechanisms for creating conventional images. The display subsystem may also provide non-visual displays, such as via an audio output device. Generally, the term "output device" is used to encompass all possible types of devices and the manner in which information is output from computing device 910 to a user or another machine or computing device.

[0111] Storage subsystem 924 stores program and data constructs that provide the functionality of some or all of the modules described herein. For example, storage subsystem 924 may include logic for performing selected aspects of the methods described herein.

[0112] These software modules are typically executed by processor 914 alone or in combination with other processors. The memory 925 used in storage subsystem 924 may include multiple memories, including main random access memory (RAM) 930 for storing instructions and data during program execution and read-only memory (ROM) 932 for storing fixed instructions. File storage subsystem 926 can provide persistent storage for program and data files and may include hard disk drives, solid-state drives, floppy disk drives, and associated removable media, CD-ROM drives, optical disk drives, or removable media cassettes. Modules implementing certain functions may be stored by file storage subsystem 926 within storage subsystem 924 or in other machines accessible to processor 914.

[0113] The bus subsystem 912 provides a mechanism for enabling the various components and subsystems of the computing device 910 to communicate with each other as intended. Although the bus subsystem 912 is schematically shown as a single bus, alternative implementations of the bus subsystem may use multiple buses.

[0114] The computing device 910 can be of various types, including workstations, servers, computing clusters, blade servers, server groups, or any other data processing system or computing device. Due to the constantly evolving nature of computers and networks, therefore... Figure 9 The description of the computing device 910 depicted herein is intended only as a specific example to illustrate some implementations. Many other configurations of the computing device 910 may have... Figure 9 The computing device depicted in the text has more or fewer components.< / w> < / w> < / w> < / w>

Claims

1. A method for generating query variants using a trained generative model, the method comprising: Receive the original query, which is generated based on user input via the user interface of the client device; The token of the original query is used as input to a trained generative model, which is trained to generate multiple types of query variants. Based on applying the token of the original query to the trained generative model, multiple variants of the original query are generated, wherein generating the variants includes generating the variants based on the learned parameters of the trained generative model, wherein each of the generated variants is different from the original query, and wherein the generated variants include a first variant as a first type of multiple query variants and a second variant as a second type of multiple query variants; The output is generated based on at least one of the plurality of variants and / or at least one search system response to at least one of the plurality of variants; In response to the original query, the output is provided to be presented via the client device; and Before generating the second variant, it is determined whether to generate the second variant based on the first search system response to the first variant.

2. The method according to claim 1, in, The first type is one of the following: equivalent query, follow-up query, generalized query, canonicalized query, implication query, normalized query, clarifying query, and language translation query; and The second type is another of the following: equivalent query, follow-up query, generalized query, canonicalized query, implication query, normalized query, clarification query, and language translation query.

3. The method according to claim 2, wherein, The first type value is applied as part of the input to the trained generative model, and the first variant is generated as the first type on the trained generative model. The second variant is generated on the trained generative model by applying a second type value as part of the input to the trained generative model.

4. The method of claim 1, wherein, Generating the second variant depends on determining whether to generate the second variant in place of providing a variant response.

5. The method of claim 4, wherein, Further, based on the first variant, determine whether to generate the second variant.

6. The method according to claim 1, wherein, The input further includes one or more attributes associated with the user.

7. The method of claim 1, further comprising: The original request, including the original query, is transmitted to the search system via the network. In response to the original request, receive the original response from the search system; The token from the original query is used as input to the trained generative model, which is based on the original response from the search system.

8. A method for generating query variants using a trained generative model, the method comprising: Receive the original query, which is generated based on user input via the user interface of the client device; The trained generative model is trained based on past query submissions from a set of users who share one or more attributes with the user, and the trained generative model is selected from multiple trained generative models. The token from the original query is used as input to the selected trained generative model; Based on applying the token of the original query to the trained generative model, at least one variant of the original query is generated; as well as The output is generated based on at least one variant and / or at least one search system response to at least one variant. The output is generated by comparing at least one search system response to the at least one variant with the original search system response to the original query to verify the accuracy of the original search system response. as well as In response to the original query, the output is provided to be presented via the client device.

9. A method for generating query variants using a trained generative model, the method comprising: Receive the original query, which is generated based on user input via the user interface of the client device; The token from the original query is used as input to the trained generative model; Based on the input, a variant of the original query is generated on the trained generative model; The variant response to the variant of the query is determined based on the variant of the query submitted to the search system; Additional inputs are applied to the trained generative model, and the additional inputs applied to the trained generative model include: the token of the original query, the variant token of the variant of the original query, and the variant response of the variant of the query; Based on the additional input, additional variants of the original query are generated on the trained generative model; The additional variant response for the additional variant of the original query is determined based on the additional variant of the original query submitted to the search system; Output is generated based on the variant response and / or the additional variant response; and In response to the original query, the output is provided to be presented via the client device.

10. A method for generating query variants using a trained generative model, the method comprising: Receive the original query, which is generated based on user input via the user interface of the client device; The prediction task for the user is determined based on non-query interactions with the user via the client device or an attached client device. The following are taken as input to the trained generative model: the token of the original query, and one or more task attributes for the prediction task of the user; At least one variant of the original query is generated by applying the token and one or more task attributes to the trained generative model; The output is generated based on the at least one variant and / or the response of at least one search system to the at least one variant; as well as In response to the original query, the output is provided to be presented via the client device.

11. The method of claim 10, wherein, The prediction task for the user is determined based on one or more interactions with the user via the client device or an additional client device.

12. The method of claim 11, wherein, The one or more interactions on which the prediction task is based include: electronic communications sent by the user or calendar entries created by the user.

13. A method for generating query variants using a trained generative model, the method comprising: Receive the original query, which is generated based on user input via the user interface of the client device; The prediction task for the user is determined based on non-query interactions with the user via the client device or an attached client device. The trained generative model is trained based on past query submissions associated with the prediction task, and the trained generative model is selected from multiple trained generative models. The token from the original query is used as input to the selected trained generative model; Based on applying the token of the original query to the trained generative model, at least one variant of the original query is generated; The output is generated based on the at least one variant and / or the response of at least one search system to the at least one variant; as well as In response to the original query, the output is provided.

14. The method of claim 13, wherein, The one or more interactions on which the prediction task is based include: electronic communications sent by the user or calendar entries created by the user.

15. A method for generating query variants using a trained generative model, the method comprising: Receive the original query, which is generated based on user input via the user interface of the client device; The original query is then applied to the trained generative model; Based on applying the original query to the trained generative model, a clarification variant with suggested clarification is generated; The clarification variant provides a hint for clarification in response to the presentation of the original query; In response to providing the aforementioned clarification variant: Receive clarification input, which is in response to the clarification variant and is generated based on further user interface input from the user via the client device; In response to receiving the clarified user interface input: Generate a context vector based on the clarified input; The original query and the clarified input are applied to the trained generative model or an additional trained generative model; as well as Further generative output is generated by applying the original query and the clarified input to the trained generative model or the additional trained generative model. as well as The output is generated based on the further generated output; as well as In response to the original query, the output is provided to be presented via the client device. Applying the clarification input to the trained generative model or the additional trained generative model includes applying the context vector generated based on the clarification input.

16. The method according to claim 15, wherein, Applying the clarifying input to the trained generative model or the additional trained generative model includes: applying the clarifying input to the trained generative model.

17. The method according to claim 15, wherein, Applying the clarifying input to the trained generative model or the additional trained generative model includes: applying the clarifying input to the additional trained generative model.

18. The method of claim 15, further comprising: The user's location, the task the user is currently performing, and / or the weather at the user's location are applied together with the original query to the trained generative model; The generation of the clarification variant is further based on applying the location, the task, and / or the weather to the trained generative model.

19. The method of claim 15, further comprising: The content recently viewed by the user on the client device is applied to the trained generative model along with the original query; The generation of the clarification variant is further based on applying the content to the trained generative model.

20. The method of claim 15, further comprising: The user's location and the task the user is currently performing are applied together with the original query to the trained generative model; The generation of the clarification variant is further based on applying the location and the task to the trained generative model.

21. The method of claim 15, wherein, The trained generative model is a sequence-to-sequence deep neural network that includes encoder and decoder layers.

22. A method for generating query variants using a trained generative model, the method comprising: Receive the original query, which is generated based on user input via the user interface of the client device; Obtain one or more search system responses to the original query from the search system; Apply the following to the trained generative model: Based on one or more attributes of the user determined before receiving the original query, The original query, and The one or more search systems respond to the original query; Generative output is generated by applying the original query, the one or more search system responses, and the one or more attributes to the trained generative model; After generating the output: By applying further input to the trained generative model, further generative output is generated. The further inputs therein are based on the generated output; The output is generated based on the further generated output; as well as In response to the original query, the output is provided to be presented via the client device.

23. The method of claim 22, further comprising: The following items are applied together with the original query and the one or more search system responses to the trained generative model: The user's location. The task the user is currently performing, and / or Weather at the user's location; The generation of the output is further based on applying the location, the task, and / or the weather to the trained generative model.

24. The method of claim 22, further comprising: The following items are applied together with the original query and the one or more search system responses to the trained generative model: Content recently viewed by the user on the client device; The generation of the output is further based on applying the content to the trained generative model.

25. The method of claim 22, further comprising: The trained generative model is selected from multiple candidate generative models; Applying the original query and the one or more search system responses to the trained generative model is in response to selecting the trained generative model from the plurality of candidate generative models.

26. The method according to claim 25, in, The selection of the trained generative model from the plurality of candidate generative models is based on one or more user attributes of the user who generated the original query.

27. The method according to claim 25, in, The selection of the trained generative model from the plurality of candidate generative models is based on the task currently being performed by the user.

28. The method of claim 22, further comprising: Before generating the output: Determining whether to generate the generated output for the original query using one or more trained control models includes: The original query is applied to the one or more trained control models; Generate a controller output on the one or more trained control models that indicates whether the generated output will be produced; and The generated output is determined based on the controller output.

29. A system for generating query variants using a trained generative model, comprising: One or more processors; A memory in communication with the one or more processors, wherein the memory stores instructions that, in response to execution by the one or more processors, cause the one or more processors to perform the method of any one of claims 1-28.

Images