Query replay for personalized responses in llm-driven assistants

Abstract

A method (500) for personalizing a response includes: receiving a natural language query (116) from a user requesting a response from an Assistant Large Language Model (LLM) (160); and processing the natural language query to identify preferred optimized queries related to the natural language query from a log of optimized queries (250), wherein each optimized query in the log of optimized queries was previously input by the user to instruct the Assistant LLM to optimize a corresponding previously generated response by the Assistant LLM. The method further includes: using the natural language query and the preferred optimized query identified as related to the natural language query to prompt the Assistant LLM to generate a personalized response to the natural language query (400). The method further includes: providing the personalized response to the natural language query for output from a user device (110).

Description

Technical Field

[0001] This disclosure relates to query replay for personalized responses in an LLM-driven assistant. Background Technology

[0002] Users frequently interact with conversational assistant apps on smart devices such as phones, watches, in-vehicle infotainment systems, and smart speakers. These apps enable users to accomplish tasks and find answers to their questions entirely through natural, conversational interactions. A key capability of conversational assistant apps involves personalization. For example, users may have certain tastes / preferences, and the assistant should understand these tastes / preferences and take them into account when providing responses to the user. While digital assistants driven by large language models (LLMs) offer the opportunity to generate responses that are more meaningfully aligned with a user's individual preferences, making clear decisions about which preferences are permanent, which may change over time, and which are context-dependent remains very difficult. Summary of the Invention

[0003] One aspect of this disclosure provides a computer-implemented method executed on data processing hardware, causing the data processing hardware to perform operations including: receiving from a user a natural language query requesting a response from an assistant large language model (LLM); processing the natural language query to identify preferred optimized queries related to the natural language query from a log of optimized queries, wherein each optimized query in the log of optimized queries was previously input by the user to instruct the assistant LLM to optimize a corresponding previously generated response by the assistant LLM; using the natural language query and the preferred optimized queries identified as related to the natural language query to prompt the assistant LLM to generate a personalized response to the natural language query; and providing the personalized response to the natural language query for output from a user device.

[0004] Implementations of this disclosure may include one or more of the following optional features. In some implementations, receiving a natural language query includes: receiving audio data representing the utterance of the natural language query spoken by a user and captured by a user device; and performing speech recognition on the audio data to generate a text representation of the natural language query spoken by the user. In some examples, prompting an assistant LLM to generate a personalized response to a natural language query includes: during a first round trip, issuing the natural language query as a non-personalized prompt to the assistant LLM to cause the assistant LLM to generate a non-personalized response to the natural language query; and during a second round trip in response to the assistant LLM generating the non-personalized response, issuing a preferred optimized query as a personalized prompt to the assistant LLM to cause the assistant LLM to generate a personalized response to the natural language query. In these examples, providing a personalized response to a natural language query may include: providing response content including both a personalized and non-personalized response to the natural language query for output from the user device.

[0005] In some implementations, the prompting assistant LLM generates a personalized response to a natural language query by: constructing a preferred optimized query into an optimized prompt; constructing a composite prompt by linking the natural language query with the optimized prompt; and issuing the composite prompt to the assistant LLM for input, so that the assistant LLM generates a basic response to the natural language query that is not personalized for the user and enhances the basic response to convey the personal preferences specified by the optimized prompt. Here, the personalized response includes an enhanced response, and the optimized prompt is specifically designed for the assistant LLM to personalize the basic response generated by the assistant LLM. In these implementations, constructing the preferred optimized query into an optimized prompt may include: constructing the preferred optimized query into the optimized prompt by appending a natural language personalized response instruction to the text representation of the preferred optimized query. Additionally or alternatively, enhancing the basic response may include at least one of the following: highlighting information or results in the basic response, annotating the text in the basic response, underlining or modifying the text in the basic response, or adjusting the format of the basic response.

[0006] In some examples, each corresponding optimized query in the optimized query log includes a corresponding confidence value assigned to that optimized query, and prompting the assistant LLM to generate a personalized response to the natural language query includes: determining that the corresponding confidence value threshold assigned to the preferred optimized query meets the confidence threshold; based on determining that the corresponding confidence value assigned to the preferred optimized query meets the confidence value threshold, constructing a natural language composite prompt from the natural language query and the preferred optimized query, the natural language composite prompt being specifically designed to instruct the assistant LLM to generate the personalized response to the natural language query; and issuing the natural language composite prompt for input into the assistant LLM to enable the assistant LLM to generate the personalized response to the natural language query. It is worth noting that the natural language composite prompt may include a single sentence.

[0007] In some implementations, each corresponding optimized query in the optimized query log is paired with a corresponding previous query embedding input by the user, which prompts the assistant LLM to generate a corresponding previous response optimized by the corresponding optimized query. Here, the corresponding previous query embedding of each corresponding previous query is projected into an embedding space. In these implementations, processing a natural language query to identify a preferred optimized query associated with the natural language query includes: embedding the natural language query into a query embedding using a neural network; identifying the previous query embedding that is closest to the query embedding in the embedding space from the previous query embeddings projected into the embedding space; and selecting the optimized query in the optimized query log that is paired with the closest previous query embedding identified in the previous query embedding as the preferred optimized query.

[0008] In some examples, the operation further includes: receiving a local context associated with the natural language query; and enhancing the natural language query by linking the natural language query to the local context. Here, prompting the assistant LLM to generate a personalized response to the natural language query includes: prompting the assistant LLM to generate the personalized response to the natural language query using the natural language query linked to the local context and a preferred optimized query. In these examples, the natural language query may include text, and the local context may be linked to the natural language query in plain text. Furthermore, the local context may include at least one of the following: recent activity history, including previous queries entered by the user during a conversational session and corresponding responses generated by the assistant LLM; geolocation data; website visits; recent documents from a private corpus; or recent user history information associated with the natural language query.

[0009] Another aspect of this disclosure provides a system including data processing hardware and memory hardware in communication with the data processing hardware. The memory hardware stores instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations including: receiving from a user a natural language query requesting a response from an Assistant Large Language Model (LLM); processing the natural language query to identify preferred optimized queries related to the natural language query from a log of optimized queries, wherein each optimized query in the log of optimized queries was previously input by the user to instruct the Assistant LLM to optimize a corresponding previously generated response by the Assistant LLM; using the natural language query and the preferred optimized query identified as related to the natural language query to prompt the Assistant LLM to generate a personalized response to the natural language query; and providing the personalized response to the natural language query for output from a user device.

[0010] This aspect may include one or more of the following optional features. In some implementations, receiving a natural language query includes: receiving audio data representing the utterance of the natural language query spoken by a user and captured by a user device; and performing speech recognition on the audio data to generate a text representation of the natural language query spoken by the user. In some examples, prompting an assistant LLM to generate a personalized response to a natural language query includes: during a first round trip, issuing the natural language query as a non-personalized prompt to the assistant LLM to cause the assistant LLM to generate a non-personalized response to the natural language query; and during a second round trip in response to the assistant LLM generating the non-personalized response, issuing a preferred optimized query as a personalized prompt to the assistant LLM to cause the assistant LLM to generate a personalized response to the natural language query. In these examples, providing a personalized response to a natural language query may include: providing response content that includes both a personalized response and a non-personalized response to the natural language query for output from the user device.

[0011] In some implementations, the prompting assistant LLM generates a personalized response to a natural language query by: constructing a preferred optimized query into an optimized prompt; constructing a composite prompt by linking the natural language query with the optimized prompt; and issuing the composite prompt to the assistant LLM for input, so that the assistant LLM generates a basic response to the natural language query that is not personalized for the user and enhances the basic response to convey the personal preferences specified by the optimized prompt. Here, the personalized response includes an enhanced response, and the optimized prompt is specifically designed for the assistant LLM to personalize the basic response generated by the assistant LLM. In these implementations, constructing the preferred optimized query into an optimized prompt may include: constructing the preferred optimized query into the optimized prompt by appending a natural language personalized response instruction to the text representation of the preferred optimized query. Additionally or alternatively, enhancing the basic response may include at least one of the following: highlighting information or results in the basic response, annotating the text in the basic response, underlining or modifying the text in the basic response, or adjusting the format of the basic response.

[0012] In some examples, each corresponding optimized query in the optimized query log includes a corresponding confidence value assigned to that optimized query, and prompting the assistant LLM to generate a personalized response to the natural language query includes: determining that the corresponding confidence value threshold assigned to the preferred optimized query meets the confidence threshold; based on determining that the corresponding confidence value assigned to the preferred optimized query meets the confidence value threshold, constructing a natural language composite prompt from the natural language query and the preferred optimized query, the natural language composite prompt being specifically designed to instruct the assistant LLM to generate the personalized response to the natural language query; and issuing the natural language composite prompt for input into the assistant LLM to enable the assistant LLM to generate the personalized response to the natural language query. It is worth noting that the natural language composite prompt may include a single sentence.

[0013] In some implementations, each corresponding optimized query in the optimized query log is paired with a corresponding previous query embedding input by the user, which prompts the assistant LLM to generate a corresponding previous response optimized by the corresponding optimized query. Here, the corresponding previous query embedding of each corresponding previous query is projected into an embedding space. In these implementations, processing a natural language query to identify a preferred optimized query associated with the natural language query includes: embedding the natural language query into a query embedding using a neural network; identifying the previous query embedding that is closest to the query embedding in the embedding space from the previous query embeddings projected into the embedding space; and selecting the optimized query in the optimized query log that is paired with the closest previous query embedding identified in the previous query embedding as the preferred optimized query.

[0014] In some examples, the operation further includes: receiving a local context associated with the natural language query; and enhancing the natural language query by linking the natural language query to the local context. Here, prompting the assistant LLM to generate a personalized response to the natural language query includes: prompting the assistant LLM to generate the personalized response to the natural language query using the natural language query linked to the local context and a preferred optimized query. In these examples, the natural language query may include text, and the local context may be linked to the natural language query in plain text. Furthermore, the local context may include at least one of the following: recent activity history, including previous queries entered by the user during a conversational session and corresponding responses generated by the assistant LLM; geolocation data; website visits; recent documents from a private corpus; or recent user history information associated with the natural language query.

[0015] Details of one or more implementations of this disclosure are set forth in the accompanying drawings and the following description. Other aspects, features, and advantages will become apparent from the specification, drawings, and claims. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of a system for applying optimized queries to personalize responses generated by an Assistant Large Language Model (LLM).

[0017] Figure 2 This is a schematic diagram of an example optimized query recognizer used to identify optimized queries previously entered by a user that are related to the current natural language query entered by that user.

[0018] Figures 3A to 3C These are example tips built by the tip builder for personalizing responses generated by LLM.

[0019] Figures 4A to 4C This is an example of response content generated by the Assistant LLM that conveys a personalized response.

[0020] Figure 5 This is a flowchart of an example operation arrangement for applying preferred optimized queries to personalize the response generated by the assistant LLM.

[0021] Figure 6 This is a schematic diagram of an example computing device that can be used to implement the systems and methods described herein.

[0022] In the various figures, the same reference numerals indicate the same elements. Detailed Implementation

[0023] Humans can engage in human-computer dialogues via various computing devices using interactive software applications known as "chatbots," "voice bots," "automated assistants," "interactive personal assistants," "intelligent personal assistants," and "conversational agents." As an example, these chatbots can correspond to machine learning models or combinations of different machine learning models and can be used to perform various tasks on behalf of users.

[0024] Chatbots employing large language models (LLMs) are currently opening up a wide range of applications due to their powerful understanding and generation capabilities, which can operate on text, images, and / or audio input. These models are also extended to have actuation capabilities through integration mechanisms with various service providers.

[0025] Users frequently interact with conversation assistant apps on smart devices such as phones, watches, in-vehicle infotainment systems, and smart speakers. These apps enable users to complete tasks and find answers to their questions entirely through natural, conversational interactions. Large Language Models (LLMs) demonstrate great potential as a key technological component in next-generation conversation assistants due to their versatility and performance across a wide range of tasks, including summarizing, writing, and tool use.

[0026] A key capability of conversational assistant applications (whether LLM-based or not) involves personalization. Specifically, users may have certain tastes / preferences, and the assistant should understand these tastes / preferences and take them into account when providing responses to the user. For example, a user might enjoy Italian food, might prefer an assistant response with a certain writing style in certain situations or even generally, and might have certain personal preferences such as health or dietary restrictions. To provide an optimal user experience, these preferences / tastes should be understood by the conversational assistant and taken into account when conversing with the user.

[0027] While digital assistants powered by large language models (LLMs) offer the opportunity to generate responses that are more meaningfully aligned with individual user preferences, making clear decisions about which preferences are permanent, which may change over time, and which are context-dependent remains extremely difficult. For example, if a user asks a conversational assistant for a list of vegetarian restaurants, the user may be vegetarian, or this could be a context-dependent preference, such as based on who the user is dining with. Therefore, it can be difficult to infer that certain preferences are hard preferences for future conversations with the user. Similarly, some user preferences may change over time.

[0028] This paper describes an implementation that personalizes responses generated by an assistant LLM for a user during a conversation between the user and the assistant LLM. Specifically, the implementation involves a query replay technique that automatically applies preferred optimized queries related to a natural language query entered by the user into the assistant LLM, causing the assistant LLM to generate a personalized response to the current query. The disclosed technique includes processing the natural language query from the user to identify preferred optimized queries related to the natural language query from a log of optimized queries, wherein each optimized query in the log was previously entered by the user to instruct the assistant LLM to optimize a corresponding previously generated response. The disclosed implementation then includes a query replay technique that uses the natural language query and the preferred optimized query to prompt the assistant LLM to generate a personalized response to the natural language query. This personalized response can be provided for output from a user device associated with the user. For example, the results associated with the personalized response can be graphically presented in a user interface executed on the user device for display. Alternatively, in addition to graphically presenting the personalized response, or instead of graphically presenting the personalized response, the personalized response can also be audibly output as synthesized speech from the user device. In some examples, a basic or non-personalized response is also generated by the assistant LLM and presented simultaneously with the personalized response. Although the non-personalized response generated by the assistant LLM responds to the natural language query, the non-personalized response is not optimized for any relevant query that conveys the user's preferences.

[0029] As will become apparent, by applying an optimized query relevant to the current query, this query replay technology automatically applies the optimized query without user input, eliminating the need for the user to repeatedly restate their preferences by manually entering the optimized query. Thus, the user experience of interacting with the assistant LLM is improved because the response generated by the assistant LLM is personalized based on the user's preferences conveyed by the relevant optimized query, and it saves the user time, as they are not required to repeatedly restate their preferences by issuing optimized queries after the assistant LLM generates an initial, non-personalized response.

[0030] Figure 1 An example system 100 is illustrated for allowing verbal conversation between user 10 and assistant LLM 160. A conversation assistant application 105 can execute on a user device 110 associated with user 10, enabling user 10 and assistant LLM 160 to interact with each other through verbal conversation. The conversation assistant application 105 can access various components for facilitating natural verbal conversation between user 10 and assistant LLM 160. For example, through the use of application programming interfaces (APIs) or other types of plug-ins, the conversation assistant application 105 can access an automated speech recognition (ASR) system 140, a playback recognizer 180, a prompt builder 150, assistant LLM 160, and a user interface 170.

[0031] During user turns of a verbal conversation between user 10 and assistant LLM 160, user device 110 captures audio data 102 representing uttered query 116 by user 10, which is directed to assistant LLM 160 to request a response. Query 116 may specify a particular task that user 10 requests assistant LLM 160 to perform on behalf of the user. For example, query 116 may request assistant LLM 160 to obtain information on a specific topic, query 116 may request assistant LLM 160 to generate text on a specific topic, or query 116 may include a question that user 10 hopes assistant LLM 160 will answer.

[0032] In some examples, user 10 speaks query 116 in natural language, and ASR system 140 performs speech recognition on audio data 102 representing the spoken query 116 to generate a text representation of query 116 spoken by user 10. This text representation of query 116 may be simply referred to as natural language query 116. In other examples, instead of verbal input, user 10 inputs the text representation of natural language query 116 via user interface 170 executed on user device 110. For example, the text representation of natural language query 116 may be input by the user via a keyboard or other input device communicating with user interface 170. Alternatively, the text representation of natural language query 116 may be one of a plurality of suggested queries displayed on screen 112 of user device 110 that the user can select.

[0033] Subsequently, the replay recognizer 180 processes the natural language query 116 to identify preferred optimization queries 250P associated with the natural language query 116 from the log 260 of optimization queries 250 (also referred to as the "optimization query log 260" or "RQ log 260"), where each optimization query in the log was previously input by the user 10 to instruct the assistant LLM 160 to optimize the corresponding previous response generated by the assistant LLM 160. The replay recognizer 180 can process the text representation of the natural language query 116 by embedding the text representation of the natural language query 116 into a corresponding query embedding 216, and use the corresponding query embedding 216 to retrieve one or more related optimization queries 250 from the optimization query log 260.

[0034] Figure 2 A schematic diagram 200 illustrates a replay recognizer 180 identifying a preferred optimized query 250P from an optimized query log 260. The replay recognizer 180 includes an embedder 210 configured to embed a natural language query 116 into a query embedding 216. This embedder may include a neural network trained to convert natural language text into corresponding query embeddings. For two natural language queries 116 conveying similar linguistic content, the neural network 210 is trained to generate query embeddings 216 that have values ​​close to each other in the embedding space. Conversely, for two natural language queries 116 conveying different linguistic content or meanings, the neural network 210 is trained to generate query embeddings 216 that have values ​​far apart from each other in the embedding space.

[0035] The optimization query log 260 stores multiple optimization queries 250, 250a-n, each of which was previously input by user 10 to optimize a corresponding previous response generated by assistant LLM 160. That is, each corresponding previous response generated by assistant LLM 160 responds to a corresponding previous query input by user 10. Each corresponding optimization query 250 stored in the optimization query log 260 is paired with a corresponding previous query embedding 216P of a corresponding previous user query input by user 10, which prompted assistant LLM 160 to generate a corresponding previous response optimized by the corresponding optimization query 250. As a non-limiting example, the previous query input by user 10 could include "Show me Restaurants in San Francisco," and the previous response generated by assistant LLM 160 could include a list of results containing the names and addresses of all restaurants in San Francisco. Continuing the example, an optimization query 250, previously entered by the user to instruct the assistant LLM to optimize and list restaurants in San Francisco, could include “Can you show only vegetarian ones?” Thus, optimization query 250 is paired with the previous query embedding 216P of the previous user query “Show me Restaurants in San Francisco.” Each previous query embedding 216P can be projected into the same embedding space as the query embedding 216 generated for the current natural language query 116. Furthermore, each optimization query 250 stored in the RQ log 260 can include optimization query (RQ) text 251 representing optimization query 250 (e.g., can you show only vegetarian ones), the previous query embedding 216P paired with optimization query 250, a confidence value 252 assigned to optimization query 250 by replay recognizer 180, and / or a date / time 253 indicating when the previous query embedding 216P was last used. Each RQ 250 may also include metadata 254, which indicates additional information about the optimized query 250 that may help determine whether the optimized query 250 is relevant to the current natural language query 116.For example, metadata 254 may include the activities that user 10 was performing when user 10 entered optimization query 250 (e.g., driving, walking, etc.), the modality in which user 10 entered optimization query 250 (e.g., by verbal input or typing), the type of user device that user 10 used to interact with the assistant LLM when entering optimization query 250 (e.g., smartphone, desktop / laptop, tablet, smart speaker, in-vehicle infotainment system, etc.), or any other contextual information, such as the history of previous conversations in which the user entered optimization query 250.

[0036] RQ log 260 may store only the optimized queries 250 entered by user 10 within a predetermined time range. The confidence value 252 of each corresponding optimized query 250 may be dynamically updated by replay recognizer 180 based on how frequently user 10 has entered the corresponding optimized query 250 and how recent the corresponding optimized query 250 is relative to the time when user 10 last entered the current natural language query 116.

[0037] After embedding the natural language query 116 into a query embedding 116 projected into the embedding space using the embedder neural network 210, the replay recognizer 180 processes the natural language query 116 to identify preferred optimized queries 250P (or multiple preferred optimized queries 250P) associated with the natural language query 116 by: identifying one or more previous query embeddings 216P that are closest to the query embedding 216; and selecting optimized queries 250 from the optimized query log 260 that pair with the identified previous query embeddings 216P that are closest to the current query embedding 216P in the embedding space as candidate optimized queries 250C. In some examples, the replay recognizer 180 calculates the corresponding distance between the current query embedding 216 and each previous query embedding 216P, and identifies one or more previous query embeddings 216P that are closest to the current query embedding 216 as those previous query embeddings whose corresponding distances satisfy a distance threshold. For example, the corresponding distance between the current embedding 216 and each previous query embedding 216P within the embedding space may include a corresponding cosine distance, and the distance threshold may include a cosine distance threshold. A shorter distance between the current embedding 216 and the previous query embedding 216P within the embedding space indicates that the previous query embedding 216P is closer to the current query embedding 216, and therefore more similar / contextually relevant to the current natural language query 116. In some scenarios, the replay recognizer 180 simply selects the optimized query 250 that is paired with the previous query embedding 216P identified as being closest to the current query embedding 216 (e.g., having the shortest corresponding distance) within the embedding space as the preferred optimized query 250P provided to the prompt builder 150. Furthermore, the replay recognizer 180 may further adjust the confidence value 252 as a function of the corresponding distance within the embedding space between the current query embedding 216 and the previous query embeddings 216P paired with the corresponding optimized query 250. For example, the confidence value 252 can be boosted for those optimized queries 250 that are paired with a previous query embedding 216P that is closer to the current query embedding 216 in the embedding space.

[0038] In a scenario where the replay recognizer 180 selects multiple candidate optimization queries 250C as relevant to the current natural language query, the replay recognizer 180 may apply an RQ candidate sorter 220 to sort each candidate optimization query 250C. Here, the RQ candidate sorter 220 processes the RQ text 251 and confidence value 252 of each selected candidate optimization query 250, and sorts the candidate optimization queries 250C in the order in which they most accurately depict the user's current preferences and which candidate optimization queries the user 10 is most likely to restate during subsequent optimization of the basic response 400U generated by the assistant LLM 160 for the current natural language query 116. The RQ candidate sorter 220 may include an LLM pre-trained to process each candidate optimization query 250C (including the RQ text 251, confidence value 252, and optionally date / time 253 and metadata 254) and output a sorted list of candidate optimization queries 250C. The LLM-based RQ candidate sorter 220 can additionally process the context 20 associated with the natural language query 116, such as, but not limited to, the natural language query 116, the conversation history of the current conversation between user 10 and assistant LLM 160, the user's location, contacts, user profile information, the modality in which user 10 inputs query 116 (e.g., voice input or typing input), the activity that user 10 is performing when user 10 inputs query 116, or even the type of user device 110 that user 10 is using to interact with assistant LLM 160 (e.g., smartphone, desktop / laptop, tablet, smart speaker, in-vehicle infotainment system, etc.). In some implementations, the LLM-based RQ candidate sorter 220 includes a pre-trained reward model (RM) for reordering. In these implementations, candidate optimized queries 250C selected by the RQ recognizer 180 and associated with high confidence values ​​252 (i.e., confidence values ​​that meet the confidence value threshold) can be paired with the current natural language query 116 for use when fine-tuning the RM via reinforcement learning. For example, user feedback indicating whether a personalized response 400P generated by the LLM 150 using one or more preferred optimized queries 250C is positive or negative can be used to fine-tune the RM via reinforcement learning.

[0039] In some examples, the RQ candidate sorter 220 selects the highest-ranked candidate optimized query 250C as the preferred optimized query 250P fed to the prompt builder 150. In other examples, the RQ candidate sorter 220 provides the prompt builder 150 with a sorted list of candidate optimized queries 250 as the preferred optimized query 250P (including at least the RQ text 251 and confidence value 252) for use in the prompt assistant LLM 160. The RQ candidate sorter 220 may exclude lower-ranked candidate optimized queries 250C from being selected as the preferred optimized query 250P fed to the prompt builder 150. For example, for the current natural language query 116 “Show me a list of restaurants in Detroit”, the candidate optimized queries 250C identified by the replay recognizer 180 may include “show me ones on the ocean”, “can you show only vegetarian ones?”, and “which ones allow children”. Here, the RQ candidate sorter 220 can exclude optimized queries 250 that include the RQ text 251 “show me ones on the ocean” because the optimized query is not context-dependent on the current natural language query 116, since Detroit is not a coastal city near any ocean.

[0040] Return to reference Figure 1 After the replay recognizer 180 identifies a preferred optimized query 250P (or multiple preferred optimized queries 250P related to a natural language query) associated with the natural language query 116, the prompt builder uses the natural language query 116 and the preferred optimized query 250P to construct a personalized prompt 300 for the prompt assistant LLM 160 to generate a personalized response 400P to the natural language query 116. For simplicity, a single preferred optimized query 250P is identified and fed to the prompt builder 150. However, this disclosure is non-limiting and will be understood that one or more preferred optimized queries 250P may be identified and fed to the prompt builder 150 to construct a prompt for personalizing the response generated by the assistant LLM 160.

[0041] The suggestion builder 150 can use a preferred optimized query 250P based on the confidence value 252 of the preferred optimized query 250P to construct different types of personalized suggestions 300. For example, and continuing with this example, when the confidence value 252 fails to meet a first confidence value threshold, the suggestion builder 150 can construct two parts of personalized suggestions 300, 300a, which simply include a natural language query 116 as a non-personalized suggestion (“Show me restaurants in Detroit”) and a preferred optimized query 250P as a personalized suggestion (“Can you show only vegetarian ones?”), such as Figure 3A As shown. Here, the conversation assistant application 105 issues a natural language query 116 as a non-personalized prompt as input to the assistant LLM 160, causing the assistant LLM 160 to generate a non-personalized response 400U to the natural language query 116 during a first round trip. During a second round trip in response to the assistant LLM 160 generating a non-personalized response, the conversation assistant application 105 issues a preferred optimized query 250P as a personalized prompt input to the assistant LLM 160, causing the assistant LLM to generate a personalized response 400P to the natural language query. It is worth noting that the preferred optimized query 250P is automatically issued during the second round trip without requiring the user to provide any additional input after the natural language query 116. The two-part personalized prompt 300a causes the assistant LLM 160 to generate response content 400 in two parts by outputting the non-personalized response 400U generated during the first round trip and separately outputting the personalized response 400P generated during the second round trip. Figure 4A An example of response content 400a is shown, which includes a non-personalized response 400U output to be displayed on the user's device screen and a personalized response 400P output to be displayed on the user's device screen. Advantageously, since the confidence value 252 of the preferred optimized query 250P does not meet the first confidence threshold, the two-part personalized prompt 300a results in the user being provided with two separate responses 400U and 400P, thereby indicating the user's preference for vegetarian restaurants without excluding other non-vegetarian restaurants in case the user is also interested in non-vegetarian restaurants. It is worth noting that if the user issues a follow-up prompt to the personalized response 400P, or otherwise interacts with the personalized results presented in the personalized response 400P, the replay recognizer 180 can increase the confidence value 252 associated with the preferred optimized query 250P (or decrease the confidence value 252 if the user interacts with the non-personalized response 400U).

[0042] In another implementation, when the confidence value 252 associated with the preferred optimized query 250P meets a first confidence value threshold but does not meet a second confidence value threshold greater than the first confidence value threshold, the suggestion builder 150 constructs the preferred optimized query 250P into an optimized suggestion 350, and then constructs composite suggestions 300 and 300b by concatenating the natural language query 116 with the optimized suggestion 350, such as... Figure 3B As shown. The composite suggestion 300b may include a natural language query 116 as the first sentence and an optimized suggestion 350 as the second sentence. The optimized suggestion 350, constructed by the suggestion builder 150, is specifically designed for the assistant LLM to personalize the basic response generated by the assistant LLM during a single round trip. That is, the suggestion builder 150 appends a natural language personalized response instruction (“Also mark those that are”) to the textual representation of the preferred optimized query 250P (“vegetarian”). For example, and continuing with that example, the suggestion builder 150 constructs the preferred optimized query 250P of “show me the vegetarian ones” into the optimized suggestion 350 of “Also mark those which are vegetarian”. Subsequently, the conversational assistant application 105 issues a compound prompt 300b for input to the assistant LLM 160, enabling the assistant LLM 160 to: generate a basic response to the natural language query 116 (a non-personalized response 400U); and enhance the basic response to convey the personal preferences specified by the optimized prompt 350 (e.g., vegetarian restaurant). Here, the enhanced basic response includes a personalized response 400P. Issuing the compound prompt 300b to the assistant LLM 160 causes the assistant LLM 160 to generate response content 400 by merging the non-personalized response 400U and the personalized response 400P into a single response. Notably, prompting the assistant LLM 160 with the compound prompt 300b reduces latency and computational cost compared to the two-part prompt 300a, which requires two round trips by the LLM 160.

[0043] Figure 4BAn example of response content 400b is shown, which combines the non-personalized response (i.e., the basic response) 400U and the personalized response 400B by enhancing the non-personalized basic response 400U to convey the user preferences specified by the optimization query 350. In the example shown, response content 400b is output to be displayed on the screen of user device 110, whereby the vegetarian restaurant in the personalized response 400P is enhanced by highlighting its name in bold text. Enhancements may additionally or alternatively include annotating the text in the basic response (e.g., adding asterisks), underlining or modifying the text in the basic response, adjusting the format of the basic response, or any other form of enhancing the basic response to convey user preferences. It is worth noting that if the user issues a follow-up prompt to the personalized response 400P, or otherwise interacts with the personalized results presented in the personalized response 400P, the replay recognizer 180 may increase the confidence value 252 associated with the preferred optimized query 250P (or decrease the confidence value 252 if the user interacts with the non-personalized response 400U).

[0044] In another implementation, when the confidence value 252 associated with the preferred optimized query 250P satisfies both a first confidence value threshold and a second confidence value threshold greater than the first confidence value threshold, the suggestion builder 150 constructs a natural language composite suggestion 300, 300c from the natural language query 116 and the preferred optimized query 250P. This natural language composite suggestion is specifically designed to instruct the assistant LLM 160 to generate a personalized response 400P. Continuing with the above example, Figure 3C An example natural language composite prompt 300c is shown, constructed by prompt builder 150 from preferred optimized query 250P and natural language query 116 based on a confidence value 252 assigned to preferred optimized query 250P. Here, prompt builder 150 constructs natural language composite prompt 300c as a new sentence instructing assistant LLM 160 to “show me a list of vegetarian restaurants in Detroit.” Issuing natural language composite prompt 300c to assistant LLM 160 causes assistant LLM 160 to generate response content 400, which includes a personalized response 400P conveying user preferences without including non-personalized results. The value of the second confidence value threshold can be selected such that the confidence value 252 satisfying the second confidence value threshold is a very high confidence indication that user 10 wants personalized search results conveying user preferences communicated by preferred optimized query 250P. It is worth noting that the prompt builder 150 generates natural language compound prompts 300c into a single sentence, and therefore uses more... Figure 3A and Figure 3BThe hints 300a and 300b indicate fewer word groups. Figure 4C An example of response content 400c is shown, which includes a personalized response 400P but does not include a non-personalized result.

[0045] Return to reference Figure 1 The prompt builder 150 and / or assistant LLM 160 may further consider additional context 20, such as, but not limited to, natural language query 116, conversation history of the current conversation between user 10 and assistant LLM 160, user location, contacts, user profile information, modality of user 10 inputting query 116 (e.g., voice input or typing input), activity that user 10 is performing when user 10 inputs query 116, or even the type of user device 110 that user 10 is using to interact with assistant LLM 160 (e.g., smartphone, desktop / laptop, tablet, smart speaker, in-vehicle infotainment system, etc.).

[0046] Figure 5 This is a flowchart of an example operational arrangement for method 500, which automatically applies a preferred optimized query 250 associated with a natural language query input by the user, causing the assistant LLM 160 to generate a personalized response 400P that conveys the user preferences specified by the optimized query 250. Method 500 may be based on memory hardware 620 (…). Figure 6 (e.g., instructions on the memory hardware 114 of user device 110 or the memory hardware 124 of remote server 120) to process data on data processing hardware 610. Figure 6 (e.g., the data processing hardware 113 of user device 110 or the data processing hardware 123 of remote server 120). At operation 510, method 500 includes receiving a natural language query 116 from user 10 requesting a response from assistant LLM 160.

[0047] At operation 520, method 500 includes processing natural language query 116 to identify preferred optimization query 250P associated with natural language query 116 from the log 260 of optimization queries 250. Each optimization query 250 in the log 260 of optimization queries 250 was previously input by user 10 to instruct assistant LLM 160 to optimize the corresponding previous response generated by assistant LLM 160.

[0048] At operation 530, method 500 includes prompting assistant LLM 160 to generate a personalized response 400P to the natural language query 116 using natural language query 116 and a preferred optimized query 250P identified as being related to the natural language query 116. At operation 540, method 500 also includes providing the personalized response 400P to the natural language query 116 for output from user device 110.

[0049] A software application (i.e., a software resource) can refer to computer software that instructs a computing device to perform a task. In some examples, a software application may be referred to as an "application," "app," or "program." Example applications include, but are not limited to, system diagnostic applications, system management applications, system maintenance applications, word processing applications, spreadsheet applications, messaging applications, media streaming applications, social networking applications, and game applications.

[0050] Non-transitory memory can be a physical device used for temporary or permanent storage of programs (e.g., instruction sequences) or data (e.g., program state information) for use by a computing device. Non-transitory memory can be volatile and / or non-volatile addressable semiconductor memory. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electronically erasable programmable read-only memory (EEPROM) (e.g., commonly used for firmware, such as boot programs). Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase-change memory (PCM), and magnetic disks or magnetic tapes.

[0051] Figure 6 This is a schematic diagram of an example computing device 600 that can be used to implement the systems and methods described in this document. The computing device 600 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. The components shown herein, their connections and relationships, and their functions are intended to be exemplary only and are not intended to limit the implementations of the invention described and / or claimed in this document.

[0052] Computing device 600 includes a processor 610, memory 620, storage device 630, a high-speed interface / controller 640 connected to memory 620 and high-speed expansion port 650, and a low-speed interface / controller 660 connected to low-speed bus 670 and storage device 630. Each of components 610, 620, 630, 640, 650, and 660 is interconnected using various buses and may be mounted on a common motherboard or otherwise. Processor 610 can process instructions for execution within computing device 600, including instructions stored in memory 620 or storage device 630, to display graphical information of a graphical user interface (GUI) on an external input / output device, such as a display 680 coupled to high-speed interface 640. In other implementations, multiple processors and / or multiple buses, as well as multiple memories and various types of memory, may be used as appropriate. Moreover, multiple computing devices 600 may be connected, with each device providing some of the necessary operations (e.g., as a server library, blade server group, or multiprocessor system).

[0053] Memory 620 stores information non-temporarily within computing device 600. Memory 620 may be a computer-readable medium, a volatile memory cell, or a non-volatile memory cell. Non-temporary memory 620 may be a physical means for temporarily or permanently storing programs (e.g., instruction sequences) or data (e.g., program state information) used by computing device 600. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electronically erasable programmable read-only memory (EEPROM) (e.g., commonly used in firmware, such as boot programs). Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase-change memory (PCM), and magnetic disks or magnetic tapes.

[0054] Storage device 630 provides mass storage for computing device 600. In some implementations, storage device 630 is a computer-readable medium. In various implementations, storage device 630 may be a floppy disk device, hard disk device, optical disk device, magnetic tape device, flash memory or other similar solid-state memory device, or device array (including devices arranged in a storage area network or other configuration). In additional implementations, a computer program product is tangibly embodied in an information carrier. The computer program product contains instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer or machine-readable medium, such as memory 620, storage device 630, or memory on processor 610.

[0055] High-speed controller 640 manages bandwidth-intensive operations of computing device 600, while low-speed controller 660 manages lower bandwidth-intensive operations. This allocation of responsibilities is merely exemplary. In some implementations, high-speed controller 640 (e.g., via a graphics processor or accelerator) is coupled to memory 620, display 680, and high-speed expansion port 650, which can accept various expansion cards (not shown). In some implementations, low-speed controller 660 is coupled to storage device 630 and low-speed expansion port 690. Low-speed expansion port 690, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, Wireless Ethernet), may be coupled to one or more input / output devices, such as keyboards, pointing devices, scanners, or networking devices, such as switches or routers, for example, via a network adapter.

[0056] As shown in the figure, the computing device 600 can be implemented in a variety of different forms. For example, it can be implemented as a standard server 600a or multiple implementations as a group of such servers 600a, as a laptop computer 600b, or as part of a rack server system 600c.

[0057] Various implementations of the systems and techniques described herein can be implemented in digital electronic and / or optical circuit systems, integrated circuit systems, specially designed ASICs (Application-Specific Integrated Circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system, which includes at least one programmable processor, which may be dedicated or general-purpose and is coupled to receive data and instructions from a storage system, at least one input device, and at least one output device, and to transfer data and instructions to the storage system, at least one input device, and at least one output device.

[0058] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages ​​and / or assembly / machine languages. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, non-transitory computer-readable medium, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term “machine-readable signal” refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0059] The processes and logic flows described in this specification can be executed by one or more programmable processors, also known as data processing hardware, which execute one or more computer programs to perform functions by manipulating input data and generating output. The processes and logic flows can also be executed by special-purpose logic circuit systems, such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits). For example, processors suitable for executing computer programs include both general-purpose microprocessors and special-purpose microprocessors, as well as any one or more processors of any type of digital computer. Typically, the processor receives instructions and data from read-only memory or random access memory, or both. The basic elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or operatively coupled to receive data from or transfer data to said mass storage device, or both. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry systems.

[0060] To provide interaction with the user, one or more aspects of this disclosure can be implemented on a computer having a display device for displaying information to the user (e.g., a CRT (cathode ray tube), LCD (liquid crystal display) monitor, or touchscreen) and possibly a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including sound, speech, or tactile input. Additionally, the computer can interact with the user by sending documents to and receiving documents from the device used by the user; for example, by sending a webpage to a web browser on the user's client device in response to a request received from a web browser.

[0061] Various implementations have been described. However, it should be understood that various modifications can be made without departing from the spirit and scope of this disclosure. Therefore, other implementations are within the scope of the appended claims.

Claims

1. A computer-implemented method (500), said computer-implemented method being executed on data processing hardware, said computer-implemented method causing said data processing hardware (610) to perform operations, said operations including: The user receives a natural language query (116) requesting a response from the assistant large language model LLM (160); The natural language query (116) is processed to identify preferred optimized queries (250) related to the natural language query (116) from the optimized query log, wherein each optimized query in the optimized query log was previously input by the user to instruct the assistant LLM (160) to optimize the corresponding previous response generated by the assistant LLM (160); The natural language query (116) and the preferred optimized query (250) identified as being related to the natural language query (116) are used to prompt the assistant LLM (160) to generate a personalized response (400) to the natural language query (116); and The personalized response (400) to the natural language query (116) is provided for output from the user device (110).

2. The computer-implemented method (500) according to claim 1, wherein receiving the natural language query (116) comprises: Receive audio data (102), the audio data representing the speech of the natural language query (116) spoken by the user and captured by the user device (110); as well as Speech recognition is performed on the audio data (102) to generate a text representation of the natural language query (116) spoken by the user.

3. The computer-implemented method (500) according to claim 1 or 2, wherein prompting the assistant LLM (160) to generate the personalized response (400) to the natural language query (116) comprises: During the first round trip, the natural language query (116) is issued as a non-personalized prompt for input to the assistant LLM (160), so that the assistant LLM (160) generates a non-personalized response to the natural language query (116); and During the second round trip in response to the assistant LLM (160) generating the non-personalized response, the preferred optimized query (250) is issued as a personalized prompt for input to the assistant LLM (160) so that the assistant LLM (160) generates the personalized response (400) to the natural language query (116).

4. The computer-implemented method (500) according to claim 3, wherein providing the personalized response (400) to the natural language query (116) comprises: A response content (400) comprising both the personalized response and the non-personalized response to the natural language query (116) is provided for output from the user device (110).

5. The computer-implemented method (500) according to any one of claims 1 to 4, wherein prompting the assistant LLM (160) to generate the personalized response (400) to the natural language query (116) comprises: The preferred optimization query (250) is constructed into an optimization hint (350), which is specifically designed for the assistant LLM (160) to personalize the basic response (400) generated by the assistant LLM (160); A composite suggestion (300) is constructed by concatenating the natural language query (116) with the optimization suggestion (350); and The compound prompt (300) is issued for input to the assistant LLM (160) so that the assistant LLM (160) can: Generate a basic response (400U) to the natural language query (116), the basic response (400U) being unpersonalized for the user; as well as The basic response (400U) is enhanced to convey the personal preferences specified by the optimization prompt (350), wherein the personalized response includes the enhanced basic response (400U).

6. The computer-implemented method (500) according to claim 5, wherein constructing the preferred optimization query (250) into the optimization hint (350) comprises: The preferred optimization query (250) is constructed into the optimization hint (350) by attaching a natural language personalized response instruction to the text representation of the preferred optimization query (250).

7. The computer-implemented method (500) according to claim 5 or 6, wherein enhancing the basic response (400U) comprises at least one of the following: Highlight the information or results in the basic response (400U); Annotate the text in the basic response (400U); Underline or modify the text in the basic response (400U); or Adjust the format of the basic response (400U).

8. The computer-implemented method (500) according to any one of claims 1 to 7, wherein: Each corresponding optimized query (250) in the log of optimized queries includes a corresponding confidence value (252) assigned to the corresponding optimized query (250); and The assistant LLM (160) is prompted to generate the personalized response to the natural language query (116) by including: The corresponding confidence value (252) threshold assigned to the preferred optimized query (250) is determined to satisfy the confidence threshold. Based on the determination that the corresponding confidence value (252) assigned to the preferred optimized query (250) satisfies the confidence value (252) threshold, the natural language query (116) and the preferred optimized query (250) are constructed into a natural language composite prompt (300), which is specifically designed to instruct the assistant LLM (160) to generate a personalized response to the natural language query (116); and The natural language compound prompt (300) is issued for input to the assistant LLM (160) so that the assistant LLM (160) generates the personalized response to the natural language query (116).

9. The computer-implemented method (500) according to claim 8, wherein the natural language compound prompt (300) comprises a single sentence.

10. The computer-implemented method (500) according to any one of claims 1 to 9, wherein: Each corresponding optimized query (250) in the log of the optimized query is paired with a corresponding previous query embedding (216) of the corresponding previous query input by the user, the corresponding previous query prompting the assistant LLM (160) to generate the corresponding previous response optimized by the corresponding optimized query (250), and the corresponding previous query embedding (216) of each corresponding previous query is projected into the embedding space. and Processing the natural language query (116) to identify the preferred optimized query (250) associated with the natural language query (116) includes: The natural language query (116) is embedded into the query embedding (216) using a neural network (210); From the previous query embeddings (216) projected into the embedding space, identify the previous query embedding (216) that is closest to the query embedding (216) within the embedding space; and The optimized query (250) that is paired with the closest previous query embedding identified in the previous query embedding (216) in the log of the optimized query is selected as the preferred optimized query (250).

11. The computer-implemented method (500) according to any one of claims 1 to 10, wherein the operation further comprises: Receive the local context associated with the natural language query (116); as well as The natural language query (116) is enhanced by linking it to the local context. The prompting the assistant LLM (160) to generate the personalized response to the natural language query (116) includes: using the natural language query (116) connected to the local context and the preferred optimized query (250) to prompt the assistant LLM (160) to generate the personalized response to the natural language query (116).

12. The computer-implemented method (500) according to claim 11, wherein: The natural language query (116) includes text; and The local context is linked to the natural language query (116) in plain text.

13. The computer-implemented method (500) according to claim 11 or 12, wherein the local environment includes at least one of the following: Recent activity history, which includes previous queries entered by the user during the conversation session and corresponding responses generated by the assistant LLM (160); Geographic location data; Website access; Recent documents from a private corpus; or Recent user history information associated with the natural language query (116).

14. A system (100) comprising: Data processing hardware (610); as well as A memory hardware (620) communicating with the data processing hardware (610), the memory hardware (620) storing instructions that, when executed on the data processing hardware (610), cause the data processing hardware (610) to perform operations, the operations including: The user receives a natural language query (116) requesting a response from the assistant large language model LLM (160); The natural language query (116) is processed to identify preferred optimized queries (250) related to the natural language query (116) from the optimized query log, wherein each optimized query in the optimized query log was previously input by the user to instruct the assistant LLM (160) to optimize the corresponding previous response generated by the assistant LLM (160); The natural language query (116) and the preferred optimized query (250) identified as being related to the natural language query (116) are used to prompt the assistant LLM (160) to generate a personalized response (400) to the natural language query (116); and The personalized response (400) to the natural language query (116) is provided for output from the user device (110).

15. The system (100) of claim 14, wherein receiving the natural language query (116) comprises: Receive audio data (102), the audio data representing the speech of the natural language query (116) spoken by the user and captured by the user device (110); as well as Speech recognition is performed on the audio data (102) to generate a text representation of the natural language query (116) spoken by the user.

16. The system (100) of claim 14 or 15, wherein prompting the assistant LLM (160) to generate the personalized response to the natural language query (116) comprises: During the first round trip, the natural language query (116) is issued as a non-personalized prompt for input to the assistant LLM (160), so that the assistant LLM (160) generates a non-personalized response to the natural language query (116); and During a second round trip in response to the assistant LLM (160) generating the non-personalized response, the preferred optimized query (250) is issued as a personalized prompt for input to the assistant LLM (160) so that the assistant LLM (160) generates the personalized response to the natural language query (116).

17. The system (100) of claim 16, wherein providing the personalized response to the natural language query (116) comprises: A response content (400) comprising both the personalized response and the non-personalized response to the natural language query (116) is provided for output from the user device (110).

18. The system (100) according to any one of claims 14 to 17, wherein prompting the assistant LLM (160) to generate the personalized response to the natural language query (116) comprises: The preferred optimization query (250) is constructed into an optimization hint (350), which is specifically designed for the assistant LLM (160) to personalize the basic response (400) generated by the assistant LLM (160); A composite suggestion (300) is constructed by concatenating the natural language query (116) with the optimization suggestion (350); and The compound prompt (300) is issued for input to the assistant LLM (160) so that the assistant LLM (160) can: Generate a basic response (400U) to the natural language query (116), the basic response (400U) being unpersonalized for the user; as well as The basic response (400U) is enhanced to convey the personal preferences specified by the optimization prompt (350), wherein the personalized response includes the enhanced basic response (400U).

19. The system (100) according to claim 18, wherein constructing the preferred optimization query (250) into the optimization hint (350) comprises: The preferred optimization query (250) is constructed into the optimization hint (350) by attaching a natural language personalized response instruction to the text representation of the preferred optimization query (250).

20. The system (100) according to claim 18 or 19, wherein enhancing the basic response (400U) comprises at least one of the following: Highlight the information or results in the basic response (400U); Annotate the text in the basic response (400U); Underline or modify the text in the basic response (400U); or Adjust the format of the basic response (400U).

21. The system (100) according to any one of claims 14 to 20, wherein: Each corresponding optimized query (250) in the log of optimized queries includes a corresponding confidence value (252) assigned to the corresponding optimized query (250); and The assistant LLM (160) is prompted to generate the personalized response to the natural language query (116) by including: The corresponding confidence value (252) threshold assigned to the preferred optimized query (250) is determined to satisfy the confidence threshold. Based on the determination that the corresponding confidence value (252) assigned to the preferred optimized query (250) satisfies the confidence value (252) threshold, the natural language query (116) and the preferred optimized query (250) are constructed into a natural language composite prompt (300), which is specifically designed to instruct the assistant LLM (160) to generate a personalized response to the natural language query (116); and The natural language compound prompt (300) is issued for input to the assistant LLM (160) so that the assistant LLM (160) generates the personalized response to the natural language query (116).

22. The system (100) of claim 21, wherein the natural language compound prompt (300) comprises a single sentence.

23. The system (100) according to any one of claims 14 to 22, wherein: Each corresponding optimized query (250) in the log of the optimized query is paired with a corresponding previous query embedding (216) of the corresponding previous query input by the user, the corresponding previous query prompting the assistant LLM (160) to generate the corresponding previous response optimized by the corresponding optimized query (250), and the corresponding previous query embedding (216) of each corresponding previous query is projected into the embedding space. and Processing the natural language query (116) to identify the preferred optimized query (250) associated with the natural language query (116) includes: The natural language query (116) is embedded into the query embedding (216) using a neural network; From the previous query embeddings (216) projected into the embedding space, identify the previous query embedding (216) that is closest to the query embedding (216) within the embedding space; and The optimized query (250) that is paired with the closest previous query embedding identified in the previous query embedding (216) in the log of the optimized query is selected as the preferred optimized query (250).

24. The system (100) according to any one of claims 14 to 23, wherein the operation further comprises: Receive the local context associated with the natural language query (116); as well as The natural language query (116) is enhanced by linking it to the local context. The prompting the assistant LLM (160) to generate the personalized response to the natural language query (116) includes: using the natural language query (116) connected to the local context and the preferred optimized query (250) to prompt the assistant LLM (160) to generate the personalized response to the natural language query (116).

25. The system (100) according to claim 24, wherein: The natural language query (116) includes text; and The local context is linked to the natural language query (116) in plain text.

26. The system (100) according to claim 24 or 25, wherein the local environment comprises at least one of the following: Recent activity history, which includes previous queries entered by the user during the conversation session and corresponding responses generated by the assistant LLM (160); Geographic location data; Website access; Recent documents from a private corpus; or Recent user history information associated with the natural language query (116).

Images