Query planning method and related apparatus

Abstract

A query planning method, comprising: acquiring historical query dialogues and a current query; using the historical query dialogues and the current query as inputs to a query rewriting and decomposition model, and outputting a sub-query list; and on the basis of the sub-query list, determining a query planning result, the query planning result indicating query planning for the current query, wherein the sub-query list comprises several sub-query tasks. In the present application, a query rewriting and decomposition model can further rewrite the content of a current query on the basis of the content of historical query dialogues; if the current query is a complex question or a long-tail question, the query rewriting and decomposition model can also decompose the current query; and after the query rewriting and decomposition model rewrites and decomposes the current query, a sub-query list is obtained, wherein the sub-query list is used for retrieval, thereby improving the retrieval accuracy of a retrieval module, and further improving the quality of answers outputted by a question-answering system.

Description

Query planning methods and related devices

[0001] This application claims priority to Chinese Patent Application No. 2024111789946, filed on August 26, 2024, entitled "Query Planning Method and Related Apparatus", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of artificial intelligence (AI) technology, and in particular to query planning methods and related apparatus. Background Technology

[0003] In the context of the big data era, fast and accurate information retrieval technology is crucial. With the explosive growth of information, users need more efficient intelligent search and question-answering platforms to manage and utilize big data. Retrieval-augmented generation (RAG) is one method for realizing intelligent question-answering platforms. In RAG technology, the user query is first searched in a knowledge base, and then the relevant documents found are returned to a large language model to generate an answer, mitigating the inaccuracy of answers when using a large language model alone. Specifically, when searching the knowledge base based on the user query, a search planning scheme is needed to improve the quality of search results. Summary of the Invention

[0004] This application provides a query planning method and related apparatus, which can obtain query planning results after processing the current query. Searching based on the query planning results can improve the quality of search results.

[0005] Firstly, this application provides a query planning method, comprising: obtaining historical query dialogues and the current query; using the historical query dialogues and the current query as input to a query rewriting decomposition model, and outputting a subquery list; and determining a query planning result based on the subquery list, wherein the query planning result indicates the query planning for the current query. The subquery list includes several subquery tasks.

[0006] In the technical solution of this application, the query rewriting and decomposition model can further rewrite the content of the current query based on the content of historical query dialogues. If the current query is a complex or long-tailed problem, the query rewriting and decomposition model can further decompose the current query. After rewriting and decomposing the current query, the query rewriting and decomposition model obtains a subquery list. Using the subquery list for retrieval improves the retrieval accuracy of the retrieval module, thereby improving the quality of the answers output by the question-answering system. For example, by rewriting the current query based on historical query dialogues, the subquery list solves the problem that the RAG system cannot understand and associate the contextual relationships between multiple rounds of queries in a single-round query; and the problem of poor retrieval results when the user's query contains omissions or references. By decomposing the current query based on historical query dialogues, the subquery list solves the problems of inaccuracy and incompleteness in complex long-tailed queries. The application scenarios of the technical solution of this application are not limited to search planning scenarios. For example, when this application is applied to search technologies such as RAG, the query planning result can be understood as a search planning result, which is used to indicate the search planning for the current query. When this application is applied to question-answering technologies such as intelligent agents, chatbots, and question-answering robots, the query planning result can be understood as a question-answering planning result, which is used to indicate the question-answering planning for the current query. It should be noted that in existing solutions, the multi-round rewriting and complex decomposition of queries are all achieved by generating large language models through prompt words. The method of this application solves the problems existing in existing solutions: the high latency and high cost caused by models with a large number of parameters; the increased development difficulty caused by the lack of a unified search planning integration scheme; the difficulty in iteratively updating large language models when errors occur on the inference side; the fact that large language models are generally external neural network models, not self-designed and trained models, thus leading to data leakage problems due to dependence on external neural network models; and the increased development costs due to the lack of a relatively universal classification system.

[0007] In one possible implementation, the number of parameters in the query-rewritten decomposition model is less than or equal to a first threshold.

[0008] In this implementation, by setting a first threshold, the number of parameters in the query rewriting decomposition model can be limited, making the query rewriting decomposition model a language model with a small number of parameters, thereby reducing the latency of the rewriting and / or decomposition process of the current query.

[0009] In one possible implementation, the query rewrite decomposition model is fine-tuned based on a first dataset, which includes the current query sample, historical query dialogue samples, and truth values ​​of the subquery list.

[0010] This implementation provides a fine-tuning method for a query rewriting decomposition model. The fine-tuned model can simultaneously perform query rewriting and decomposition. This means the user can input the current query into the model, which directly outputs the rewritten and decomposed subquery lists. Compared to methods that perform rewriting and decomposition sequentially, this model reduces latency and improves the accuracy of subsequent query result derivations. For example, the fine-tuning process of the query rewriting decomposition model includes: obtaining a first dataset, which includes the current query sample, historical query dialogue samples, and the truth values ​​of the subquery list; using the current query sample and historical query dialogue samples as input to the model, and outputting the predicted values ​​of the subquery list; obtaining a first loss value based on the truth values ​​and predicted values ​​of the subquery list; and adjusting the parameters of the query rewriting decomposition model with the goal of minimizing the first loss value.

[0011] In one possible implementation, the method further includes: detecting whether the subquery list is correct; correcting the subquery list when it is incorrect; updating the first dataset based on the corrected subquery list; and fine-tuning the query rewrite decomposition model based on the updated first dataset.

[0012] In this implementation, when the rewritten and / or decomposed subquery list is found to be incorrect, the fine-tuning training dataset can be updated. Through iterative fine-tuning, the performance of the query rewriting and decomposition model in handling the current query is efficiently optimized. Updating the first dataset can involve adding the corrected subquery list and the corresponding query. Alternatively, to further optimize training performance, updating the first dataset can also involve adding training samples belonging to the same technical field or related to the content of the query that caused the error. For example, an incorrect subquery list could be due to textual errors, content errors, or logical errors.

[0013] In one possible implementation, the truth values ​​of the subquery list are obtained by rewriting and / or decomposing the current query sample based on historical query dialogue samples using a large language model.

[0014] In this implementation, the truth values ​​of the subquery list are generated using a large language model. In other implementations, the truth values ​​of the subquery list can be obtained through annotation; alternatively, they can be obtained from an existing dataset.

[0015] In one possible implementation, the above-mentioned process of taking historical query dialogues and the current query as input to the query rewriting decomposition model and outputting a subquery list includes: processing the historical query dialogues and the current query according to the prompt word template to obtain a first prompt word, so that the format of the first prompt word is the same as the format of the second prompt word in the fine-tuning process of the query rewriting decomposition model; and taking the first prompt word as input to the query rewriting decomposition model and outputting a subquery list.

[0016] In this implementation, the query rewriting decomposition model can be a language model, so the input needs to be set in the form of prompt words to meet the requirements of the query rewriting decomposition model for input content. Specifically, the first prompt word on the inference side and the second prompt word on the fine-tuning training side have the same format, which facilitates the query rewriting decomposition model's understanding of the input query and improves the accuracy of the output subquery list.

[0017] In one possible implementation, the above-mentioned determination of query planning results based on the subquery list includes: taking the current query, the subquery list, and the intent tag database as input to the intent classification model, outputting the target intent set to which the current query belongs, the intent tag database representing multiple intent tags corresponding to different classification systems, and the target intent set including several target intents corresponding to the current query under different classification systems; and determining the query planning results based on the subquery list and the target intent set.

[0018] This implementation categorizes the intent of the current query and subquery list to obtain the target intent set corresponding to the current query. Based on this target intent set, it routes queries to different subsequent modules or performs database-specific queries. This solves the problem of weak relevance of subsequent search documents and poor answer quality in certain scenarios where query intents are not categorized, enabling diverse business logic in a custom routing process. For example, subsequent modules can perform searches based on the subquery list to obtain search results; the question-answering system matches the target intent set, so the answers output by the question-answering system based on search results, the current query, and historical query dialogues are more accurate.

[0019] In one possible implementation, the query planning result includes a set of query operations and query path information. The set of query operations represents multiple query operations performed on several subquery tasks in the subquery list. The query path information indicates the routing of the subquery list to a target downstream task among multiple downstream query tasks, and / or the requirements for the target downstream task. The aforementioned determination of the query planning result based on the subquery list and the target intent set includes: determining the set of query operations based on the subquery list; and determining the query path information based on the target intent set.

[0020] In this implementation, the set of query operations is determined based on the subquery list, which facilitates the downstream task's understanding of the input query and improves the accuracy of the output subquery list. The query path information is determined based on the target intent set, which facilitates routing from multiple downstream query tasks to the target downstream task, and / or, based on the requirements of the target downstream task, the specific configuration of the target downstream task is implemented, improving the adaptability between the current query and the target downstream task, thereby enabling the target downstream task to output higher quality final results.

[0021] In one possible implementation, the intent classification model includes N intent sub-models, each used to identify the target intent of the current query under one of the N classification systems. The intent tag database includes N intent tag sub-databases corresponding to the N intent sub-models, and the target intent set includes N target intents corresponding to the N intent sub-models, where N is an integer greater than or equal to 1. The process of using the current query, subquery list, and intent tag database as input to the intent classification model and outputting the target intent set to which the current query belongs includes: using the current query, subquery list, and corresponding intent tag sub-database as input to each intent sub-model and outputting the corresponding target intent; and obtaining the target intent set based on the N target intents output by the N intent sub-models.

[0022] In this implementation, the accuracy of user intent identification directly affects the accuracy of selecting the target downstream task, thus impacting the quality of the downstream task's output. This application's embodiment sets up N intent sub-models, each used to identify different intent categories, solving the problem of poor identification performance when using a single model to identify user intent, and enabling more accurate confirmation of the corresponding downstream task. Furthermore, this method can be understood as a search planning ensemble scheme, using different intent sub-models to identify the intent classification results of the current query. An intent classification sub-model is a neural network model with specific functions, achieving its function with a small number of parameters. Small-parameter models are characterized by ease of fine-tuning, small fine-tuning datasets, low inference latency, and the ability to be internalized (i.e., the model can be designed and trained independently).

[0023] In one possible implementation, the intent sub-model is configured based on the number of intent tags in the corresponding intent tag sub-database. When the number of intent tags in the corresponding intent tag sub-database is less than or equal to a second threshold, the intent sub-model includes a first fine-grained ranking model. The current query, subquery list, and corresponding intent tag sub-database are used as inputs to each intent sub-model, and the corresponding target intent is output. This includes: using the current query, subquery list, and corresponding intent tag sub-database as inputs to the first fine-grained ranking model, outputting a first intent tag sort; and determining the target intent based on the first intent tag sort.

[0024] In this implementation, when the number of intent tags in the intent tag sub-database is small (e.g., less than or equal to the second threshold), the intent sub-model can include a first fine-ranking model. Fine-ranking is a common ranking model that accurately sorts the input candidate tags to obtain the final intent tag ranking, and identifies the top-ranked intent tags as the target intent. Compared to a large language model, fine-ranking is a model with fewer parameters, effectively reducing the overall latency issue when using multiple intent sub-models. In this scenario, a recall model can be omitted, thus avoiding the latency generated by the recall model and reducing the overall latency of the intent sub-models.

[0025] In one possible implementation, when the number of intent tags in the corresponding intent tag sub-database exceeds a second threshold, the intent sub-model includes a recall model and a second ranking model. Each intent sub-model takes the current query, subquery list, and corresponding intent tag sub-database as input and outputs the corresponding target intent. This includes: taking the current query, subquery list, and intent tag sub-database as input to the recall model and outputting a set of recalled intent tags; taking the current query, subquery list, and recalled intent tag set as input to the second ranking model and outputting a second intent tag ranking; and determining the target intent based on the second intent tag ranking.

[0026] In this implementation, the recall model is optional. Whether to set a recall model for a given intent sub-model can be determined based on the number of intent tags in the intent tag sub-database for each intent category. That is, when the intent tag sub-database for an intent category is small (e.g., less than or equal to the second threshold), the first ranking model can directly sort the intent tags in the sub-database due to the small number of intent tags. However, when the intent tag sub-database for an intent category is large (e.g., greater than the second threshold), the workload is significant when the first ranking model directly sorts the intent tags in the sub-database due to the large number of intent tags. Therefore, a recall model can be set up to first recall the intent tags in the sub-database, and then the second ranking model can be used to sort the intent tags in the sub-database.

[0027] In one possible implementation, the intent classification model is fine-tuned based on a second dataset, which includes query samples, intent label database samples, target category labels, and non-target category labels.

[0028] This implementation provides a fine-tuning method for an intent classification model. The model takes query samples and intent label database samples as input and outputs intent prediction values. It trains the intent classification model using target category labels as positive samples and non-target category labels as negative samples. For example, the intent classification model is fine-tuned. The fine-tuning process includes: obtaining a second dataset, which includes query samples, intent label set samples, target category labels, and non-target category labels; taking the query samples and intent label set samples as input and outputting intent prediction values; obtaining a second loss value based on the intent prediction value and the target category label; obtaining a third loss value based on the intent prediction value and the non-target category labels; and adjusting the parameters of the intent classification model with the goal of minimizing the second loss value and maximizing the third loss value.

[0029] In one possible implementation, the method further includes: detecting whether the target intent set is correct; correcting the target intent set when it is incorrect; updating the second dataset based on the corrected target intent set; and fine-tuning the intent classification model based on the updated second dataset.

[0030] In this implementation, when an error is detected in the target intent set, the fine-tuned training dataset can be updated. Through iterative fine-tuning, the intent classification model's ability to recognize the intent of the current query is efficiently optimized. Updating the second dataset can involve adding corrected intent labels and corresponding queries. Optionally, to further optimize training performance, updating the second dataset can also involve adding query samples and their intent label sets that belong to the same technical field or are content-related to the query that caused the error. Alternatively, the erroneous target intent set can be used as a non-target category label to fine-tune the intent classification model, improving its intent classification ability and reducing the likelihood of the same type of error recurring.

[0031] In one possible implementation, the target category label and the non-target category label are obtained by inputting the query sample into a large language model and then outputting the results.

[0032] In this implementation, the target category label and non-target category label can be generated based on the query sample using a large language model. For example, prompt words can be obtained from the query sample using a prompt word template; the prompt words are then input into the large language model, and the target category label and non-target category label for the query sample are output. In other implementations, the target category label and non-target category label can be obtained through annotation; or they can be obtained by acquiring an existing dataset.

[0033] In one possible implementation, the intent classification model includes three intent sub-models: a topic-based intent sub-model, a time-sensitive intent sub-model, and a computational intent sub-model. The topic-based intent sub-model is used to identify user intents categorized by topic, the time-sensitive intent sub-model is used to identify user intents categorized by time, and the computational intent sub-model is used to identify user intents categorized by computation. The intent tag database includes a topic-based intent tag database, a time-sensitive intent tag database, and a computational intent tag database. The target intent set includes topic-based intents, time-sensitive intents, and computational intents. The above-mentioned method of taking the current query, subquery list, and intent tag database as input to the intent classification model and outputting the target intent set of the current query includes: taking the current query, subquery list, and topic-based intent tag database as input to the topic-based intent sub-model and outputting topic-based intents; taking the current query, subquery list, and timeliness-based intent tag database as input to the timeliness-based intent sub-model and outputting timeliness-based intents; and taking the current query, subquery list, and computational intent tag database as input to the computational intent sub-model and outputting computational intents.

[0034] In this implementation, the intent classification model includes three intent sub-models. The first intent sub-model identifies intent tags in the topic category corresponding to the input query. The second intent sub-model identifies intent tags in the timeliness category corresponding to the input query. The third intent sub-model identifies intent tags in the computational category corresponding to the input query. An illustrative explanation of the intent classification model is provided. Based on different classification dimensions, user intents are divided into three categories: main category (i.e., topic category), timeliness category, and computational category. The main category identifies the attribute category to which the current query belongs; that is, it distinguishes which category the current query belongs to, thereby determining the corresponding downstream task. The timeliness category identifies whether the current query is time-sensitive, determining whether the corresponding downstream task requires related processing, such as retrieving web pages or sorting by time. The computational category identifies whether the current query is computational, determining whether the corresponding downstream task requires related processing, such as adding a calculator. A calculator refers to any module, software, or tool with technical functions. Identifying the user intent corresponding to the current query from different dimensions can improve the accuracy of user intent recognition, more accurately identify the corresponding downstream tasks, thereby improving the accuracy of downstream task recognition results and solving the problem of poor user intent recognition when user intent is not classified.

[0035] In one possible implementation, the topic-based intent indicates whether the subquery list should be routed to the target downstream task among multiple downstream query tasks; the timeliness-based intent indicates whether the processing results of the current query output by the target downstream task need to be sorted by time; and the computational intent indicates whether a calculator should be configured for the target downstream task.

[0036] In this implementation, the application scenarios of topic-based intents, time-sensitive intents, and computational intents are illustrated by examples, but this does not constitute a limitation on the embodiments of this application.

[0037] In one possible implementation, the intent tags in the topic category include at least one of the following: general knowledge, industry knowledge, casual conversation, language task, and persona; the intent tags in the timeliness category include timeliness and non-timeliness; and the intent tags in the computational category include computational and non-computational.

[0038] In this implementation, the various intent categories are described by way of example and do not constitute a limitation on the intent tags of the embodiments of this application. In other implementations, each intent category may include more or fewer intent tags.

[0039] In one possible implementation, the method further includes: configuring intent sub-models in the intent model based on the user's model selection information, each intent sub-model being used to identify a target intent of a classification system.

[0040] This implementation provides a method for determining the intent sub-model, ensuring that the target intent set of the current query meets the diverse needs of users and improves the user experience.

[0041] Secondly, this application provides a query planning apparatus, comprising: an acquisition module for acquiring historical query dialogues and the current query; a processing module for taking the historical query dialogues and the current query as input to a query rewriting decomposition model and outputting a subquery list; and, based on the subquery list, determining a query planning result, wherein the query planning result indicates the query planning for the current query. The subquery list includes several subquery tasks.

[0042] In one possible implementation, the number of parameters in the query-rewritten decomposition model is less than or equal to a first threshold.

[0043] In one possible implementation, the query rewrite decomposition model is fine-tuned based on a first dataset, which includes the current query sample, historical query dialogue samples, and truth values ​​of the subquery list.

[0044] In one possible implementation, the processing module is further configured to: detect whether the subquery list is correct; correct the subquery list when it is incorrect; update the first dataset based on the corrected subquery list; and fine-tune the query rewrite decomposition model based on the updated first dataset.

[0045] In one possible implementation, the truth values ​​of the subquery list are obtained by rewriting and / or decomposing the current query sample based on historical query dialogue samples using a large language model.

[0046] In one possible implementation, the processing module is specifically used to: process historical query dialogues and the current query according to the prompt word template to obtain a first prompt word, so that the format of the first prompt word is the same as the format of the second prompt word in the fine-tuning process of the query rewriting decomposition model; and use the first prompt word as input to the query rewriting decomposition model to output a list of subqueries.

[0047] In one possible implementation, the processing module is specifically used to: take the current query, subquery list, and intent tag database as input to the intent classification model, output the target intent set to which the current query belongs, the intent tag database represents multiple intent tags corresponding to different classification systems, and the target intent set includes several target intents corresponding to the current query under different classification systems; and determine the query planning result based on the subquery list and the target intent set.

[0048] In one possible implementation, the query planning result includes a set of query operations and query path information. The set of query operations represents multiple query operations performed on several subquery tasks in the subquery list. The query path information indicates the routing of the subquery list to a target downstream task among multiple downstream query tasks, and / or the requirements for the target downstream task. Specifically, this processing module is used to: determine the set of query operations based on the subquery list; and determine the query path information based on the target intent set.

[0049] In one possible implementation, the intent classification model includes N intent sub-models, each used to identify the target intent of the current query under one of the N classification systems. The intent tag database includes N intent tag sub-databases corresponding to the N intent sub-models, and the target intent set includes N target intents corresponding to the N intent sub-models, where N is an integer greater than or equal to 1. Specifically, this processing module is used to: take the current query, the subquery list, and the corresponding intent tag sub-database as input to each intent sub-model, and output the corresponding target intent; based on the N target intents output by the N intent sub-models, obtain the target intent set.

[0050] In one possible implementation, the intent sub-model is configured based on the number of intent tags in the corresponding intent tag sub-database. When the number of intent tags in the corresponding intent tag sub-database is less than or equal to a second threshold, the intent sub-model includes a first fine-grained ranking model. This processing module is specifically used to: take the current query, the subquery list, and the corresponding intent tag sub-database as input to the first fine-grained ranking model, and output a first intent tag sort; based on the first intent tag sort, determine the target intent.

[0051] In one possible implementation, when the number of intent tags in the corresponding intent tag sub-database exceeds a second threshold, the intent sub-model includes a recall model and a second ranking model. Specifically, this processing module is used to: take the current query, subquery list, and intent tag sub-database as input to the recall model and output a set of recalled intent tags; take the current query, subquery list, and the set of recalled intent tags as input to the second ranking model and output a second intent tag ranking; and determine the target intent based on the second intent tag ranking.

[0052] In one possible implementation, the intent classification model is fine-tuned based on a second dataset, which includes query samples, intent label database samples, target category labels, and non-target category labels.

[0053] In one possible implementation, the processing module is further configured to: detect whether the target intent set is correct; when the target intent set is incorrect, correct the target intent set; update the second dataset based on the corrected target intent set; and fine-tune the intent classification model based on the updated second dataset.

[0054] In one possible implementation, the target category label and the non-target category label are obtained by inputting the query sample into a large language model and then outputting the results.

[0055] In one possible implementation, the intent classification model includes three intent sub-models: a topic-based intent sub-model, a time-sensitive intent sub-model, and a computational intent sub-model. The topic-based intent sub-model identifies user intents categorized by topic; the time-sensitive intent sub-model identifies user intents categorized by time sensitivity; and the computational intent sub-model identifies user intents categorized by computation. The intent tag database includes a topic-based intent tag database, a time-sensitive intent tag database, and a computational intent tag database. The target intent set includes topic-based intents, time-sensitive intents, and computational intents. Specifically, this processing module is used to: take the current query, subquery list, and topic-based intent tag database as input to the topic-based intent sub-model and output topic-based intents; take the current query, subquery list, and time-sensitive intent tag database as input to the time-sensitive intent sub-model and output time-sensitive intents; and take the current query, subquery list, and computational intent tag database as input to the computational intent sub-model and output computational intents.

[0056] In one possible implementation, the topic-based intent indicates whether the subquery list should be routed to the target downstream task among multiple downstream query tasks; the timeliness-based intent indicates whether the processing results of the current query output by the target downstream task need to be sorted by time; and the computational intent indicates whether a calculator should be configured for the target downstream task.

[0057] In one possible implementation, the intent tags in the topic category include at least one of the following: general knowledge, industry knowledge, casual conversation, language task, and persona; the intent tags in the timeliness category include timeliness and non-timeliness; and the intent tags in the computational category include computational and non-computational.

[0058] In one possible implementation, the processing module is further configured to: configure intent sub-models in the intent model based on the user's model selection information, each intent sub-model being used to identify the target intent of a classification system.

[0059] Thirdly, this application provides a chip system including a processor and a power supply circuit. The power supply circuit supplies power to the processor, and the processor executes the method described in any possible implementation of the first aspect and / or the algorithm function implemented by the apparatus described in any possible implementation of the second aspect.

[0060] Fourthly, this application provides a search question-answering system, characterized in that it includes: a query planning module, configured to execute the algorithm function implemented by the method described in any possible implementation of the first aspect and / or the apparatus described in any possible implementation of the second aspect, to generate a query planning result for the current query; and a query processing module, configured to generate a query processing result for the current query under the instruction of the query planning result.

[0061] Fifthly, this application provides a computing device including a processor and a memory. The processor is configured to execute instructions stored in the memory, causing the computing device to perform the methods described in any possible implementation of the first aspect and / or the algorithmic functions implemented by the means described in any possible implementation of the second aspect.

[0062] In a sixth aspect, this application provides a computing device cluster, including at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, such that the computing device cluster performs the method described in any possible implementation of the first aspect and / or the algorithm function implemented by the apparatus described in any possible implementation of the second aspect.

[0063] In a seventh aspect, this application provides a computer-readable storage medium including computer program instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method described in any possible implementation of the first aspect and / or the algorithmic function implemented by the apparatus described in any possible implementation of the second aspect, wherein the cluster of computing devices includes at least one computing device.

[0064] Eighthly, this application provides a computer program product containing instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method described in any possible implementation of the first aspect and / or the algorithmic function implemented by the apparatus described in any possible implementation of the second aspect, wherein the cluster of computing devices includes at least one computing device.

[0065] It is understood that the beneficial effects of aspects two through eight above can be found in the relevant descriptions in aspect one above, and will not be repeated here. Attached Figure Description

[0066] The accompanying drawings used in the embodiments or technical description are briefly introduced below.

[0067] Figure 1 is a schematic diagram of the architecture of a search question-answering system provided in an embodiment of this application;

[0068] Figure 2 is a schematic diagram of the composition of a search question-answering system provided in an embodiment of this application;

[0069] Figure 3 is a schematic diagram of the composition of a query planning module provided in an embodiment of this application;

[0070] Figure 4 is a flowchart illustrating a query planning method provided in an embodiment of this application;

[0071] Figure 5 is a schematic diagram of the composition of an intent classification model provided in an embodiment of this application;

[0072] Figure 6 is a schematic diagram of the composition of a query planning device provided in an embodiment of this application;

[0073] Figure 7 is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0074] Figure 8 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of this application;

[0075] Figure 9 is a schematic diagram of another computing device cluster provided in an embodiment of this application. Detailed Implementation

[0076] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0077] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0078] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0079] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0080] To facilitate understanding of the solutions provided in the embodiments of this application, a brief introduction to some of the terms involved in this solution will be given first.

[0081] A query is a question posed by a user. For example, in the question-and-answer domain, a user inputs a query into a question-and-answer device, which then outputs the query results based on that query. Queries can take various forms, including text and voice data.

[0082] Large Language Models (LLMs) typically refer to neural network models containing an extremely large number of parameters (usually over a billion). They possess the following characteristics: First, they have a massive parameter scale, containing billions of parameters and reaching sizes of hundreds of gigabytes or even larger. This enormous scale provides them with powerful expressive and learning capabilities. Second, they can learn multiple tasks simultaneously. Large language models often learn various NLP (Neuro-Linguistic Programming) tasks together, such as machine translation, text summarization, and question-answering systems. Multi-task learning allows the model to learn broader and more generalized language understanding abilities. Third, they require powerful computing resources. Training large language models typically requires hundreds or even thousands of GPUs (graphics processing units) and a significant amount of time, usually ranging from weeks to months. Powerful computing resources can accelerate the training process while preserving the capabilities of the large language model. Fourth, they require abundant data. Large language models require vast amounts of data for training; only with a large amount of data can the advantages of their parameter scale be fully realized. Furthermore, large language models are widely used in the field of natural language processing (NLP) and are fundamentally changing the landscape of NLP tasks, giving rise to more powerful and intelligent language technologies. Large language models are one of the important directions in the development of AI (Artificial Intelligence). At the same time, large language models also possess the ability to perform exceptionally well in various NLP tasks, such as text classification, sentiment analysis, summarization, and translation. Large language models can be used in multiple application areas, including automatic writing, chatbots, virtual assistants, voice assistants, and automatic translation.

[0083] Retrieval-augmented Generation (RAG) is a technique based on LLM (Language, Mechanics, and Information) technology. RAG models combine LLM and information retrieval techniques. Specifically, when an LLM needs to generate text or answer a question, the RAG model first retrieves relevant information from a large document collection. This retrieved information is then used to guide text generation, improving the quality and accuracy of predictions. RAG generates answers or content by referencing information from external knowledge bases, offering strong interpretability and customizability. It is suitable for various natural language processing tasks such as question-answering systems, document generation, and intelligent assistants. The advantages of RAG models lie in their versatility, ability to update knowledge instantly, and the provision of more efficient and accurate information services through end-to-end evaluation methods. Retrieval-augmented Generation (RAG) is one method for implementing intelligent question-answering platforms. The main idea of ​​RAG is to first search the knowledge base for the user's query, and then return the relevant documents found to the LLM to generate the answer, mitigating problems such as illusions, outdated knowledge, and lack of industry knowledge that occur when using LLM alone. However, RAG not only needs a retrieval module to search for relevant documents and generate models to provide answers, but also a query planning module that can understand and handle users' complex query needs in order to provide higher quality search results and answers.

[0084] Prompt: The prompt word input into the large language model, enabling the large language model to generate an answer based on the prompt word.

[0085] Multi-turn query rewriting: In multi-turn dialogues, the user's query is rewritten to improve the retrieval performance of the rewritten query.

[0086] Complex query decomposition: Decompose the user's complex query into smaller parts, so that the retrieval effect of the decomposed query is better.

[0087] Intent Classification: Classify user queries to obtain a set of target intents, which facilitates subsequent routing to different processes.

[0088] Long-tail queries refer to queries targeting long-tail keywords. Long-tail keywords are relatively niche phrases that users only search for a few times per month. These keywords are called long-tail because they form a long and narrow portion of the keyword search volume graph. While individual long-tail queries may not bring in much traffic, they can contribute to website traffic overall.

[0089] Recall models are used to reduce data from tens of thousands, hundreds of thousands, or even millions of records to a scale of hundreds to thousands. Because recall requires processing massive amounts of data in a short time, the number of features should not be excessive, the model structure should be relatively simple, and the inference time should be short. Its characteristics can be summarized as a large candidate pool, fast computation speed, few features, and a simple model, with the goal of ensuring high recall.

[0090] Fine-grained ranking models are used to score and accurately rank small-scale recalled data. These models typically employ relatively complex structures, utilizing multiple features to achieve precise ranking. Because this stage requires significant computational resources—directly performing fine-grained ranking on a massive data pool would be unpredictable—the division between recall and fine-grained ranking is essentially a balance between efficiency and performance. Its characteristics can be summarized as a small candidate pool, slow computation speed, numerous features, and complex models, with the goal of ensuring accurate ranking.

[0091] In the field of big data retrieval, fast and accurate information retrieval technology is crucial. With the explosive growth of information volume, users need more efficient intelligent search and question-answering platforms to manage and utilize big data. Retrieval-augmented generation (RAG) can provide information retrieved from certain data sources to guide the generation of answers by large language models. Specifically, when an LLM needs to generate text or answer questions, the RAG model first retrieves relevant information from a large document collection, and then uses this retrieved information to guide text generation, thereby improving the quality and accuracy of predictions.

[0092] Please refer to Figure 1, which shows a schematic diagram of the architecture of a search question-answering system provided in an embodiment of this application. As shown in Figure 1, this embodiment of the application provides a search question-answering system, which may include a terminal 1 and / or a server 2. The server 2 may include one or more servers (Figure 1 illustrates this with one server as an example), and the server 2 may provide the methods and / or apparatus provided in this embodiment of the application to one or more terminals.

[0093] Optionally, a relevant application may be installed on terminal 1. This application can receive user-input queries through terminal 1 and send the queries to server 2. Server 2 processes the queries, obtains the corresponding query results, and sends the query results back to terminal 1 to display the query results to the user.

[0094] For example, as shown in Figure 1, the user interface of terminal 1 displays: "Search Question and Answer System", which indicates the name of the relevant application installed on terminal 1; "Please enter your query", which indicates a prompt for the user to enter a query; "What is the difference between douzhi and doujiang?" in the text box indicates the content of the query entered by the user; "Start Search" button, which indicates the start of the search or retrieval; "View Answer" button, which indicates that the user can view the search answer.

[0095] It should be understood that in some optional implementations, terminal 1 can also implement the methods and / or apparatus of the embodiments of this application independently. That is, terminal 1 can complete the question-and-answer process on its own and obtain the query result corresponding to the query without the cooperation of server 2. The embodiments of this application are not limited in this respect. In some optional implementations, server 2 can also implement the query based on the query received from other devices outside the terminal or from local storage without the cooperation of terminal 1. The embodiments of this application are not limited in this respect.

[0096] The product form of terminal 1 in Figure 1 will be described next. In this application embodiment, terminal 1 can be a mobile phone, a speaker, a robot, a watch with voice function, a tablet computer, a wearable device, an in-vehicle device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc. This application embodiment does not impose any limitations on this.

[0097] The product form of server 2 in Figure 1 will be described next. It can be further understood that server 2 can be various types of servers, such as x86 architecture servers, specifically rack servers, blade servers, high-density servers, platform servers, or high-performance servers. In other words, this application embodiment does not specifically limit the specific category of the server. Furthermore, it can be understood that the server structure shown in Figure 1 does not constitute a limitation on the server structure; a server may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0098] Furthermore, server 2 can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server or cloud server cluster that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The cloud server cluster is deployed in several cloud data centers. The software can be an application that implements object control methods, but is not limited to the above forms. Optionally, the method or apparatus of this application embodiment can be deployed on server 2 equipped with a GPU (Graphics Processing Unit) or an NPU (Neural-network Processing Unit).

[0099] Next, the communication connection method between terminal 1 and server 2 is described. For example, terminal 1 and server 2 are connected via a network, enabling terminal 1 to access the cloud management platform deployed on the cloud server cluster. The network can be a wired network or a wireless network. For example, a wired network can be a cable network, a fiber optic network, a Digital Data Network (DDN), etc., while a wireless network can be a telecommunications network, an internal network, the Internet, a Local Area Network (LAN), a Wide Area Network (WAN), a Wireless Local Area Network (WLAN), a Metropolitan Area Network (MAN), a Public Service Telephone Network (PSTN), a Bluetooth network, a ZigBee network, a Global System for Mobile Communications (GSM), a CDMA (Code Division Multiple Access) network, a CPRS (General Packet Radio Service) network, etc., or any combination thereof.

[0100] Understandably, a network can use any known network communication protocol to enable communication between different terminal layers and gateways. These network communication protocols can be various wired or wireless communication protocols, such as Ethernet, Universal Serial Bus (USB), FireWire, Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), New Radio (NR), Bluetooth, Wireless Fidelity (Wi-Fi), and other communication protocols.

[0101] In one possible scenario, server 2 can function as a cloud platform (a software platform employing application virtualization technology, integrating multiple functions such as software search, download, use, management, and backup). In practical use, server 2 can deploy a cloud management platform and a data center, with terminal 1 interacting with the cloud through the cloud management platform. Additionally, the data center can deploy nodes, which can be virtual machine instances, container instances, physical servers, etc.

[0102] In another possible scenario, the method provided in this application embodiment can be implemented by software. The software has a terminal and a server; terminal 1 is the terminal running the software, and server 2 is the server running the software. During the operation of the software on terminal 1, it can call the server running on server 2 to implement the method provided in this application embodiment.

[0103] In other words, the method provided in this application embodiment can be applied to terminal 1 or server 2. In specific implementation, it can run as software on terminal 1 or server 2; for example, the software can be a service or an application. This application embodiment can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application embodiment can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can reside in local and remote computer storage media, including storage devices.

[0104] Referring to Figure 2, Figure 2 is a schematic diagram of the composition of a search question-answering system provided in an embodiment of this application. As shown in Figure 2, this embodiment of the application provides a search question-answering system, including: a query planning module and a query processing module. The query planning module is used to generate the query planning result of the current query; the query processing module is used to generate the final query processing result of the current query based on the query planning result. Exemplarily, the query processing module includes a retrieval module, a prompt word template, and a generation model. The query planning module is used to process the query plan to improve the retrieval accuracy of the retrieval module, such as multi-round query rewriting, complex query decomposition, intent classification, etc.; the retrieval module is used to retrieve or search the query processed by the query planning module to obtain one or more retrieval results; the prompt word template is used to convert the user-input query (including the current query and historical query dialogue) and retrieval results into prompt words that meet the input requirements of the question-answering model; the prompt words are used as input to the generation model, and the generation model outputs the final query result. Exemplarily, the generation model can be implemented based on LLM, and the implementation method of the generation model will not be described in detail in this application. For example, the retrieval module can be implemented using a retriver. The main function of the retriver is to perform a search based on an inverted index formed by the input keywords, simultaneously evaluate the relevance between the page and the search results, sort the output results, and implement a user relevance feedback mechanism. This application will not elaborate on the implementation method of the retriver.

[0105] In this application embodiment, one application scenario is a Retrieval Enhancement Generation (RAG) system. For example, the functional device corresponding to the method in this application embodiment can be used as a module to enhance the effect of the RAG system. In practical applications, the RAG system is mainly used in enterprise intelligent search and question-answering platforms, where users can engage in multiple rounds of question-and-answer sessions with the platform to obtain answers to questions related to the enterprise's knowledge base.

[0106] Optionally, the method or apparatus of this application embodiment can be deployed independently as a general search planning module for RAG systems; or it can be integrated into an enterprise's own RAG system to work together with an end-to-end RAG system.

[0107] It should be noted that the search question-answering system or RAG system is merely an exemplary application scenario of the method or apparatus in this application embodiment. For example, in other application scenarios, such as some instant messaging scenarios, the query planning results output by the query planning module can obtain prompt words based on the prompt word template, and the generation model can generate dialogue content based on the prompt words, without needing to execute the retrieval method.

[0108] The following is an illustrative description of the functions of the query planning module.

[0109] For example, multi-turn query rewriting refers to the process where, when a user's query uses ellipsis or pronouns, the query planning module needs to be able to rewrite the user's query into a complete, independent, and searchable query by incorporating historical dialogue. For instance, the query might be "Why is this?", which needs to be rewritten by combining the demonstrative pronoun "this" with the context to "Why is ***?", thus making the query a complete, independent, and searchable query.

[0110] For example, complex query decomposition refers to the situation where, when a user's query involves multiple knowledge items, direct retrieval is often ineffective. The query planning module needs to break down the long-tail query into multiple simpler queries to improve search quality. For instance, a query might be "What is the difference between douzhi (fermented mung bean juice) and doujiang (soy milk)?". A better approach is to decompose this query into three queries: the first could be "What is douzhi?", the second "What is doujiang?", and the third "What is the difference between douzhi (fermented mung bean juice) and doujiang?". This decomposes the long-tail query into multiple simpler queries, thereby improving the search quality of the retrieval module.

[0111] For example, intent classification refers to determining the intent category of a user's input query, and then inputting that query into the downstream branch corresponding to that intent category. Each downstream branch of an intent category can output better query results for queries belonging to that intent category, thereby improving the accuracy of the query results.

[0112] Therefore, the accuracy of the query results output by the retrieval module is highly dependent on the query planning module. Consequently, how to design a query planning module to further improve the accuracy of query results has been a key focus in the RAG technology field.

[0113] One solution involves rewriting user queries based on an LLM (Local Management Module). Taking multi-turn query rewriting as an example, a pre-defined prompt template is created. The conversation history and the user's latest query are then populated into the prompt template to obtain a complete prompt. This complete prompt is then passed to a callable LLM. The LLM's results are retrieved, parsed, and the rewritten multi-turn query is obtained. Finally, a search is performed based on this rewritten query to retrieve the results.

[0114] In this solution, query rewriting based on LLM presents several problems: LLM typically involves a large number of parameters, long processing time, and high cost; it can only complete one or more subtasks in a search planning task at a time, making joint development and testing of various subtasks difficult and increasing development complexity; when the search planning results returned by LLM are incorrect or do not meet expectations, the optimization prompt method is usually used to optimize the search planning results, but the effect is limited, and there is a reluctance to iteratively optimize erroneous results; to improve the effectiveness of search planning, LLM is generally an external AI model, which can easily lead to data leakage; the lack of a relatively universal classification system increases development costs, requiring developers to define their own classification system according to project requirements, but in RAG tasks, universal classification standards have poor universality and low usability.

[0115] In another solution, the retrieval module does not perform search planning, i.e., it does not have a query planning module.

[0116] The problems with this solution include: In single-round query planning, without using search planning rules, only single-round queries can be processed. This means each user query is processed independently, and the platform cannot understand or relate the contextual relationships between multiple round queries. When user queries contain omissions or references, the retrieval results are greatly affected. It cannot effectively handle complex long-tail queries. The system typically performs semantic retrieval directly on user queries or relies on fixed rules to process queries. When facing complex long-tail queries, this method lacks accuracy and flexibility, making it difficult to cope with diverse and complex user query needs, resulting in inaccurate and incomplete query results. It cannot customize subsequent routing processes and does not categorize intents. For example, there is no unified module to categorize user query intents based on different dimensions such as whether a search is needed, whether the latest information is needed, or whether calculations are needed. This leads to weak relevance of subsequent search documents in specific scenarios, thus affecting the quality of answers and preventing the implementation of diverse business logic in custom routing processes.

[0117] In view of this, this application provides a query planning module. Referring to Figure 3, Figure 3 is a schematic diagram of the composition of a query planning module provided in this application embodiment. As shown in Figure 3, this application embodiment provides a query planning module including: a query rewriting and decomposition model and an intent classification model. The query rewriting and decomposition model is used to rewrite and decompose the current query based on historical query dialogues, and output a subquery list. The intent classification model is used to determine the intent tag of the user of the current query based on the current query and the subquery list, so as to determine the corresponding analysis module according to the user intent.

[0118] For example, as shown in Figure 3, the analysis modules (examples of downstream tasks) routed to according to user intent include: a retrieval module, which can match the knowledge base of the retrieval module according to user intent, such as a persona knowledge base, web page, industry knowledge base, etc.; a chat model, which can directly call a neural network model with dialogue processing function when the user intent is a chat dialogue; and a text generation model, which can directly call a neural network model with text generation function when the user intent is to generate relevant text.

[0119] For example, in the query rewriting and decomposition model, a module capable of simultaneously performing multiple rounds of query rewriting and complex query decomposition is implemented based on a model generated by fine-tuning small parameters. Furthermore, the user's query is rewritten in multiple rounds or decomposed into a complex query so that subqueries can be used for subsequent searches, thus optimizing retrieval quality.

[0120] For example, the intent classification model employs a two-stage intent classification architecture of embedding vector recall and fine-tuning, using a mutually exclusive main classification and a single-classification system with separate judgments, and designing classification criteria. This allows user queries to be classified across different dimensions to route them to subsequent processes.

[0121] In the embodiments of this application and their optional implementations, the high latency and high cost problems caused by large parameter models are solved by using a small parameter model for query rewriting and decomposition. By integrating sub-modules such as a query rewriting and decomposition model for multi-round rewriting and complex decomposition, and an intent classification model for identifying the current query intent, the problem of increased development difficulty caused by the lack of a unified search planning integration scheme is solved. When errors occur on the inference side, the training dataset can be updated, and the query rewriting and decomposition model and / or intent classification model can be fine-tuned again, solving the problem of difficulty in iteration when errors occur on the inference side of a large language model with large parameters. The problem of data leakage caused by reliance on external AI is solved. By designing a classification system with different classification standards corresponding to different intent categories, the problem of increased development costs caused by the lack of a more universal classification system is solved.

[0122] Furthermore, the methods or apparatus of this application embodiment can be applied in public cloud / hybrid cloud / private cloud, and the classification system of intent classification, the scope supported by rewriting decomposition, and the output preferences can all be customized according to actual business scenarios. Only the corresponding training data needs to be prepared and the model fine-tuned. In addition to the RAG system, the embodiments of this application can also be used in a wider range of AI applications, such as chatbots and AI agents. Similarly, it is necessary to optimize the design of the classification system of intent classification, the scope supported by rewriting decomposition, and the output preferences, prepare the corresponding dataset, and fine-tune the model.

[0123] Furthermore, the embodiments of this application solve the core problems of high latency, high cost, data leakage, and high development cost caused by the current LLM+prompt solution. Exemplary examples show that the technical solutions of the embodiments of this application, after adaptive modifications, should fall within the scope of this application's embodiments. Possible modifications include: using a generative model with larger parameters for fine-tuning to achieve multi-round query rewriting and complex query decomposition, or an integrated device that performs multi-round query rewriting first and then complex query decomposition, or adding more functions such as query expansion and query recommendation based on these two functions. The intent classification model is implemented using different architectures, such as a decoder + classification layer scheme, or directly using the LLM+prompt scheme. Only mutually exclusive primary classifications are used, or other different classification criteria are used. The query rewriting and decomposition model and the intent classification model are merged into the same generative model to implement the technical solutions of the embodiments of this application.

[0124] For example, a major application of this application is a search planning scheme in a Retrieval Enhancement Generation (RAG) scenario. This scheme utilizes a fine-tuned small-parameter generation model (i.e., a query rewriting and decomposition model) to simultaneously perform multi-round query rewriting and complex query decomposition, addressing issues such as query omission, referencing, and long tails with low latency, and resolving the difficulty in iterating on rewriting and decomposition effects. The intent classification model is implemented based on a two-stage classification architecture of embedding recall and fine-tuning. This model requires only a small amount of data for fine-tuning and supports custom and extended classification criteria, solving the problems of difficulty in iterating on classification effects and prolonged inference time. Furthermore, it employs a mutually exclusive primary classification and a single-classification system with separate judgments, and designs classification criteria to address the difficulties in defining multi-dimensional classification criteria and the increased development costs due to the lack of a universal classification system. The query planning module design includes using different model schemes to construct sub-modules for multi-round query rewriting and complex decomposition, intent classification, etc., optimizing the quality of subsequent retrieval and responses with low latency.

[0125] Referring to Figure 4, Figure 4 is a flowchart illustrating a query planning method provided in an embodiment of this application. As shown in Figure 4, this embodiment of the application provides a query planning method, which mainly includes the following steps:

[0126] Step S410: Obtain historical query dialogues and the current query.

[0127] Step S420: Take the historical query dialogue and the current query as input to the query rewriting decomposition model, and output a subquery list, which includes several subquery tasks.

[0128] Step S430: Based on the subquery list, determine the query plan result, which indicates the query plan for the current query.

[0129] In this embodiment, the content of the current query can be further understood based on the content of historical query dialogues. For example, the current query is "What about Monday? Where is the nearest restaurant?", while historical query dialogues are "What's the weather like this weekend?" and "The weather is sunny this weekend." Therefore, the current query needs to be rewritten as "What's the weather like on Monday?" based on the content of historical query dialogues. Simultaneously, the current query is a complex problem or a long-tail problem containing multiple query questions, so it needs to be decomposed. After rewriting and decomposing the current query, a subquery list is obtained. The subquery list includes a first subquery and a second subquery. The first subquery is "What's the weather like on Monday?"; the second subquery is "Where is the nearest restaurant?". A search is performed based on the subquery list to obtain search results; or, further, based on the search results, the user's current query, and historical query dialogues, the question-answering system outputs an answer.

[0130] In this embodiment, the intent classification of the current query input by the user can obtain the target intent set corresponding to the current query. Then, based on the target intent set, the query can be routed to different subsequent modules or a database query can be performed. For example, if the current query is "I'm so tired from taking the train", the corresponding target intent set is "leisure chat". Then, the query can be routed to a large language model that is good at chat and dialogue to perform subsequent tasks, such as outputting "At this time, you can have a good sleep".

[0131] In this embodiment, a query rewrite decomposition model is used to rewrite and decompose the current query based on historical query dialogues, resulting in a subquery list. Using this subquery list for retrieval improves the accuracy of the retrieval module, thereby enhancing the quality of the answers output by the question-answering system.

[0132] In this embodiment, the current query is rewritten based on the historical query dialogue. Therefore, the subquery list solves the problem that the RAG system cannot understand and associate the contextual relationships between multiple queries in a single query; and the problem that the retrieval effect is poor when the user's query contains omissions and references.

[0133] In this embodiment, the current query is decomposed based on historical query dialogues. Therefore, the subquery list solves the problems of inaccuracy and incompleteness in complex long-tail queries. For example, a long-tail query problem is that directly performing semantic retrieval on user queries or relying on fixed rules to process queries results in insufficient accuracy and flexibility when facing complex long-tail queries, making it difficult to cope with diverse and complex user query needs, leading to inaccuracy and incompleteness.

[0134] In one possible implementation, the current query, subquery list, and intent tag database are used as input to the intent classification model, and the output is the target intent set to which the current query belongs. The intent tag database represents multiple intent tags corresponding to different classification systems, and the target intent set includes several target intents corresponding to the current query under different classification systems. Based on the subquery list and the target intent set, the query planning result is determined.

[0135] This implementation categorizes the intent of the current query and subquery list to obtain the target intent set corresponding to the current query. Based on this target intent set, it routes queries to different subsequent modules or performs database-specific queries. This solves the problem of weak relevance of subsequent search documents and poor answer quality in certain scenarios where query intents are not categorized, enabling diverse business logic in a custom routing process. For example, subsequent modules can perform searches based on the subquery list to obtain search results; the question-answering system matches the target intent set, so the answers output by the question-answering system based on search results, the current query, and historical query dialogues are more accurate.

[0136] In one possible implementation, the query planning result includes a set of query operations and query path information. The set of query operations represents multiple query operations performed on several subquery tasks in the subquery list. The query path information indicates the routing of the subquery list to a target downstream task among multiple downstream query tasks, and / or the specific requirements for the target downstream task. Further, the set of query operations is determined based on the subquery list; the query path information is determined based on the target intent set.

[0137] In this implementation, the set of query operations is determined based on the subquery list, which facilitates the downstream task's understanding of the input query and improves the accuracy of the output subquery list. The query path information is determined based on the target intent set, which facilitates routing from multiple downstream query tasks to the target downstream task, and / or, based on the requirements of the target downstream task, the specific configuration of the target downstream task is implemented, improving the adaptability between the current query and the target downstream task, thereby enabling the target downstream task to output higher quality final results.

[0138] In this embodiment, the method can be understood as a search planning integration scheme, using different models to achieve multi-round query rewriting and complex decomposition, intent classification, and other objectives. The query rewriting decomposition model and the intent classification model are neural network models with specific functions, capable of achieving their respective functions with a relatively small number of parameters. Therefore, the query rewriting decomposition model and the intent classification model can be small-parameter models. Small-parameter models have advantages such as ease of fine-tuning, small fine-tuning datasets, low inference latency, and the ability to be internalized (i.e., the models can be designed and trained independently).

[0139] Furthermore, in other query planning schemes, multi-round query rewriting and complex decomposition, intent classification, etc., are all achieved by generating large language models through prompt words. In this embodiment, the method solves the problems existing in other query planning schemes (based on the "large language model + prompt words" technical solution): high latency and high cost caused by models with a large number of parameters; increased development difficulty due to the lack of a unified search planning integration scheme; difficulty in iteratively updating the large language model when errors occur on the inference side; large language models are generally external AI models, not self-designed and trained models, thus leading to data leakage problems due to dependence on external AI models; and increased development costs due to the lack of a relatively universal classification system.

[0140] Furthermore, by using the current query and the list of subqueries as input to the intent classification model, and since the list of subqueries is derived from the rewriting and decomposition of the current query and contains more accurate information than the current query, the intent classification model can more accurately understand the intent of the current query based on the list of subqueries when analyzing the intent of the current query.

[0141] In one example, if the current query does not require rewriting, decomposition, or topic shifting, the subquery list is also resistant to the dialogue history, maintaining the original sample of the current query for output.

[0142] In one possible implementation, the number of parameters in the query-rewritten decomposition model is less than or equal to a first threshold.

[0143] In this implementation, a first threshold limits the number of parameters in the query rewriting decomposition model, making it a language model with a smaller number of parameters and reducing the latency of the query rewriting decomposition process. For example, using a large language model for query rewriting and decomposition results in a latency of approximately 1-2 seconds, while the inference latency of the query rewriting decomposition model in this embodiment can be shortened to approximately 100-300 ms. Because the query rewriting decomposition model has a smaller number of parameters, when errors are found in the rewriting and / or decomposition results in practical applications, the training data can be updated, and the query rewriting decomposition model can be iteratively trained to efficiently optimize its inference performance. For example, assuming the allowed number of parameters for the query rewriting decomposition model is 1B to 10B, and the inference latency can be shortened to 100-300 ms, then the parameter threshold is 10B, and the latency threshold is 300 ms.

[0144] In one possible implementation, the query rewriting decomposition model is fine-tuned. The fine-tuning process of the query rewriting decomposition model includes: obtaining a first dataset, which includes the current query sample, historical query dialogue samples, and the truth values ​​of the subquery list; using the current query sample and historical query dialogue samples as input to the query rewriting decomposition model, and outputting the predicted values ​​of the subquery list; obtaining a first loss value based on the truth values ​​and predicted values ​​of the subquery list; and adjusting the parameters of the query rewriting decomposition model with the goal of minimizing the first loss value.

[0145] This implementation provides a fine-tuning method for the query rewrite decomposition model. The finely tuned model can simultaneously perform query rewrite and query decomposition. In other words, the user can input the current query into the model, which will directly output the rewritten and decomposed subquery list. Compared to methods that perform rewrite and decomposition sequentially, this model reduces latency and improves the accuracy of subsequent query result derivations.

[0146] In one possible implementation, the method further includes: detecting whether there are errors in the subquery list; when there are errors in the subquery list, updating the first dataset, which is used to fine-tune the query rewrite decomposition model.

[0147] In this implementation, when an error is found in the rewritten or decomposed subquery list, the fine-tuning training dataset can be updated. Through iterative fine-tuning, the performance of the query rewrite / decomposition model in handling the current query is efficiently optimized. Updating the first dataset can involve adding the corrected subquery list and the corresponding query. Alternatively, to further optimize training performance, updating the first dataset can also include samples of queries and their subquery lists that belong to the same technical field or are related to the query that caused the error.

[0148] In one possible implementation, the truth value of the subquery list is obtained by inputting the current query sample and historical query dialogue samples into a large language model and then outputting the results.

[0149] In this implementation, the truth values ​​of the subquery list are generated using a large language model. In other implementations, the truth values ​​of the subquery list can be obtained through annotation; alternatively, they can be obtained from an existing dataset.

[0150] In one possible implementation, the current query and historical query dialogues are processed based on the first prompt word template to obtain the first prompt word; the first prompt word is used as input to the query rewriting decomposition model to output a list of subqueries.

[0151] In this implementation, the query rewriting decomposition model can be a language model, so the input needs to be set as prompt words to meet the requirements of the query rewriting decomposition model for input content.

[0152] In one possible implementation, the intent classification model comprises N intent sub-models, where N is an integer greater than or equal to 1. Each intent sub-model identifies the user intent within an intent category corresponding to the input query. The current query and a list of subqueries are used as input to each intent sub-model, which outputs the target intent for the corresponding intent category. Based on the target intents output by the N intent sub-models, a set of target intents is obtained.

[0153] In this implementation, during the process of determining the user intent of the current query, the identification results of user intents of different intent categories directly affect the quality of the intent label (i.e., the target intent), thereby affecting the determination of downstream tasks and the quality of output results. This application embodiment sets up N intent sub-models, each used to identify different intent categories, solving the problem of poor identification performance when identifying user intent based on a single model, and can more accurately identify the corresponding downstream tasks. In this application embodiment, the target intent is also referred to as the intent label.

[0154] For example, when identifying user intent based on a model, if intent tags in intent category A are ranked higher than those in intent category B, then when the output result shows the first few intent tags, it might only output the intent tags in intent category A, ignoring the intent tags in intent category B. However, in this embodiment, different intent sub-models are used to identify different intent categories. For example, intent sub-model 1 is used to identify intent category A, and intent sub-model 2 is used to identify intent category B. Therefore, when the output result shows the first few intent tags, intent sub-model 1 outputs the first few intent tags in intent category A, and intent sub-model 2 outputs the first few intent tags in intent category B, without ignoring intent tags in any intent category, thus more accurately identifying downstream tasks.

[0155] Referring to Figure 5, which is a schematic diagram of the composition of an intent classification model provided in an embodiment of this application, the intent classification model includes N intent sub-models, such as Intent Sub-model 1, Intent Sub-model 2, etc., as shown in Figure 5. Each intent sub-model outputs its target intent based on the current query and the subquery list, such as Intent Label 1 outputted by Intent Sub-model 1, Intent Label 2 outputted by Intent Sub-model 2, etc. The corresponding downstream task can be determined based on the N target intents.

[0156] In one possible implementation, the intent sub-model includes a fine-ranking model. Candidate tags for each intent category are obtained; the current query, subquery list, and candidate tags are used as input to the fine-ranking model of each intent sub-model, and the output is the target intent in the corresponding intent category.

[0157] In this implementation, the fine-ranking model is a common ranking model that can accurately sort the input candidate tags to obtain the final ranking of intent tags. The top-ranked intent tags are then identified as the target intent and output. Compared to large language models, the fine-ranking model has fewer parameters and can effectively reduce the overall latency problem when using multiple intent sub-models.

[0158] In one possible implementation, the intent sub-model also includes a recall model. The method further includes taking the current query, the list of subqueries, and the intent tag set (i.e., the intent tag sub-database) as input to the recall model of each intent sub-model, and outputting candidate tags from the corresponding intent category.

[0159] In this implementation, the recall model is optional. Whether to set a recall model for the corresponding intent sub-model can be determined based on the number of intent tags in the intent tag set for each intent category. That is, when the intent tag set for an intent category is small, the ranking model can directly sort the intent tags in the set due to the small number of tags. In this scenario, a recall model can be omitted, thus avoiding the latency introduced by the recall model and reducing the overall latency of the intent sub-model. In this scenario, candidate tags are all intent tags in the intent tag set.

[0160] For example, when the set of intent tags for an intent category is large—for instance, the number of intent tags in the set is in the tens of thousands, hundreds of thousands, or even millions—the ranking model experiences significant latency when sorting massive amounts of data due to the large number of intent tags. Therefore, a recall model is needed to retrieve the top m candidate tags from the intent tag set, where m is an integer greater than or equal to 1. These top m candidate tags are then processed by the ranking model to obtain the final target intent. Conversely, when the set of intent tags for an intent category is small but the number of intent tags is large, a recall model is also needed to retrieve the top m intent tags (i.e., candidate tags) from the intent tag set. These top m candidate tags are then processed by the ranking model to obtain the final target intent.

[0161] For example, when the set of intent tags for an intent category is small, such as a few hundred to a few thousand intent tags, the ranking model can directly sort the intent tags in the set due to the small number of intent tags. In this scenario, a recall model can be omitted, thus avoiding the latency caused by the recall model and reducing the overall latency of the intent sub-model. In this scenario, the candidate tags are all intent tags in the intent tag set.

[0162] In one possible implementation, the intent classification model is fine-tuned. The fine-tuning process of the intent classification model includes: obtaining a second dataset, which includes query samples, intent label set samples, target category labels, and non-target category labels; using the query samples and intent label set samples as input to the intent classification model and outputting intent prediction values; obtaining a second loss value based on the intent prediction value and the target category label; obtaining a third loss value based on the intent prediction value and the non-target category label; and adjusting the parameters of the intent classification model with the goal of minimizing the second loss value and maximizing the third loss value.

[0163] This implementation provides a method for fine-tuning an intent classification model. The query samples and intent label set samples are used as input to the intent classification model, which outputs intent prediction values. The intent classification model is trained using target category labels as positive samples and non-target category labels as negative samples.

[0164] It should be noted that in the second dataset used for fine-tuning, the query samples are generally complete queries and do not usually need to be rewritten or decomposed, so subquery list samples are not needed; that is, if the query samples are incomplete queries, they need to be rewritten and decomposed, so subquery list samples are needed.

[0165] In one possible implementation, the method further includes: detecting whether there is an error in the target intent set of the current query; if there is an error in the target intent set of the current query, updating a second dataset, the updated second dataset being used to fine-tune the intent classification model.

[0166] In this implementation, when an error is detected in the target intent set, the fine-tuned training dataset can be updated. Through iterative fine-tuning, the intent classification model's ability to recognize the intent of the current query is efficiently optimized. Updating the second dataset can involve adding corrected intent labels and corresponding queries. Optionally, to further optimize training performance, updating the second dataset can also involve adding query samples and their intent label sets that belong to the same technical field or are content-related to the query that caused the error. Alternatively, the erroneous target intent set can be used as a non-target category label to fine-tune the intent classification model, improving its intent classification ability and reducing the likelihood of the same type of error recurring.

[0167] In one possible implementation, the target category label and the non-target category label are obtained by inputting the query sample into a large language model and then outputting the results.

[0168] In this implementation, the target category label and non-target category label can be generated based on the query sample using a large language model. For example, prompt words can be obtained from the query sample using a prompt word template; the prompt words are then input into the large language model, and the target category label and non-target category label for the query sample are output. In other implementations, the target category label and non-target category label can be obtained through annotation; or they can be obtained by acquiring an existing dataset.

[0169] In one possible implementation, the intent classification model includes three intent sub-models: the first intent sub-model is used to identify intent tags in the topic category corresponding to the input query; the second intent sub-model is used to identify intent tags in the timeliness category corresponding to the input query; and the third intent sub-model is used to identify intent tags in the computational category corresponding to the input query.

[0170] This implementation provides an illustrative example of the intent classification model. User intents are categorized into three types based on different classification dimensions: main category (i.e., topic category), timeliness category, and computational category. The main category identifies the attribute category to which the current query belongs; that is, it distinguishes which category the current query belongs to, thereby determining the corresponding downstream task. The timeliness category identifies whether the current query is time-sensitive, determining whether the corresponding downstream task requires relevant processing, such as retrieving web pages or sorting by time. The computational category identifies whether the current query is computational, determining whether the corresponding downstream task requires relevant processing, such as adding a calculator or setting up a calculation module. Identifying the user intent corresponding to the current query from different dimensions improves the accuracy of user intent recognition and allows for more precise identification of the corresponding downstream task, thereby improving the accuracy of downstream task recognition results and solving the problem of poor user intent recognition performance when user intents are not classified.

[0171] It should be noted that the exemplary description of this intent category does not constitute a limitation on the intent categories of the embodiments of this application. In other implementations, each intent category may include more or fewer intent labels.

[0172] In one possible implementation, the intent tags in the topic category include at least one of the following: general knowledge, industry knowledge, casual conversation, language task, and persona; the intent tags in the timeliness category include timeliness and non-timeliness; and the intent tags in the computational category include computational and non-computational.

[0173] In this implementation, the various intent categories are described by way of example and do not constitute a limitation on the intent tags of the embodiments of this application. In other implementations, each intent category may include more or fewer intent tags.

[0174] In one possible implementation, the number of intent categories is determined based on the user's input information; and / or, the number of intent sub-models is determined based on the user's input information. These are used to identify different intent categories, solving the problem of poor recognition performance when using a single model to identify user intents, and allowing for more accurate identification of the corresponding downstream tasks.

[0175] This implementation provides several methods for determining the intent sub-model, ensuring that the target intent set of the current query meets the diverse needs of users and improves the user experience.

[0176] For example, taking the search and question answering system shown in Figure 1, terminal 1 provides a user interface for customizing intent categories. Users can configure these categories through the user interface, such as by adding or removing intent categories. Server 2 then determines the corresponding intent sub-model based on the intent category. Terminal 1 also provides a user interface for customizing intent sub-models. Users can configure these sub-models through the user interface, such as by selecting intent sub-models. Server 2 then determines the corresponding intent sub-model. Specifically, when a user adds a new intent category, server 2 needs to fine-tune the corresponding intent sub-model based on the new intent category.

[0177] In other implementations, to ensure the integrity and security of system functions, the default execution of the pre-set intent sub-model in the background can be used to identify the corresponding intent category and thus route it to the corresponding downstream task.

[0178] This application's embodiments address the issues of referential resolution and information omission in multi-turn scenarios by replacing complete content with referential information in the query or supplementing omitted content. A single query is decomposed into a subquery list of multiple subqueries, resolving the problems of inaccurate or incomplete results in complex or long-tail queries. Furthermore, when no rewriting, decomposition, or topic shift is required, the subquery list is also resistant to interference from the dialogue history, maintaining the original sample of the current query for output. The query rewriting and decomposition model automatically determines whether to output one or multiple queries, completing rewriting and decomposition simultaneously in a single call, reducing latency. The rewritten or decomposed subquery list is used as a subsequent retrieval query. Compared to the common "LLM+prompt" solution with a latency of 1-2 seconds, the technical solution of this application's embodiments can shorten the inference latency to approximately 100-300ms. When errors are found in the rewritten / decomposed subquery list in practical applications, the query rewriting and decomposition model can be fine-tuned through iterative training data to efficiently optimize the rewriting and decomposition effect.

[0179] In this embodiment, the query planning module can be understood as a search planning module. Different model schemes are used to construct sub-modules such as query multi-round rewriting and complex decomposition, and intent classification. With low latency, the problem of extended end-to-end search planning inference time is solved, as well as the problems of lack of a unified search planning integration scheme and data leakage caused by reliance on external AI are also solved.

[0180] This document provides an exemplary description of the query rewriting decomposition model. For instance, an instruction fine-tuning dataset can be constructed using an LLM (Low-Level Model). This dataset contains instruction data for multi-turn query rewriting and complex query decomposition, and should maintain diversity to cover as many rewriting and decomposition scenarios as possible. Fine-tuning is performed based on a small-parameter generation model (i.e., the query rewriting decomposition model), which can be, for example, a model with 1B to 10B parameters. The fine-tuned small-parameter model is then deployed. Preprocessing is performed on the user's dialogue history and latest questions, including limiting the number of dialogue turns, limiting the total number of characters, and handling redundant punctuation. This data is then concatenated into a rewriting decomposition prompt consistent with that used during training, serving as input to the small-parameter generation model. Multiple subqueries output by the small-parameter generation model are obtained, and post-processing is performed on these subqueries to obtain a list of subqueries. This post-processing includes regenerating subqueries when they cannot be parsed and forcing them into a list format. In other words, this embodiment of the application solves problems such as query omission, referencing, and long tails, and addresses the difficulty in iterating the rewriting decomposition effect, all while maintaining low latency.

[0181] This document provides an exemplary description of the intent classification model. Specifically, this embodiment primarily focuses on the technical architecture of the intent classification model within the search planning module, providing the fundamental technical capability for multi-dimensional intent classification of queries. This example uses an intent sub-model obtained from three classification categories. In intent classification, multiple classification results under multiple classification systems are often required. This embodiment uses different models to distinguish between different classification systems, obtaining classification results for different systems through multiple model calls. Each individual classification model employs a two-stage classification architecture: embedding retrieval and fine-ranking. First, the top N1 intent tags with the highest scores are retrieved via embedding, where N1 is an integer greater than or equal to 1. Then, the fine-ranking model obtains the scores after deep interaction between the query and each intent tag, and outputs them in order. When the number of candidate tags is small, the embedding retrieval module can be omitted, and only the fine-ranking module can be used for classification to optimize the overall latency under different candidate tag count scenarios. The intent classification model is derived by fine-tuning the embedding recall and ranking models. During fine-tuning training, a single-round query is used as input, with the target category as positive samples and non-target categories as negative samples. The fine-tuning training data can be constructed using an LLM or by using queries from real-world business scenarios, requiring only a few hundred samples per category. Compared to general classification models, the recall and ranking models have fewer parameters, facilitating fine-tuning. Therefore, if classification errors are found in application, the training set can be optimized to fine-tune the intent classification model, thereby efficiently correcting the errors. In other words, this embodiment solves the problems of difficulty in iterating classification results and prolonged inference time.

[0182] The classification system used by the intent classification model is illustrated by way of example. For example, embodiments of this application employ a mutually exclusive primary classification (i.e., classification by topic) and a single-classification system based on individual judgment, and design a universal classification standard. In other words, embodiments of this application solve the problems of difficulty in defining multi-dimensional classification standards and increased development costs due to the lack of a universal classification system.

[0183] It should be noted that in actual RAG scenarios, query intents often need to be categorized to route to different subsequent modules or perform database-specific queries. For intents like timeliness and computation, this does not contradict the main category. A query may be categorized into one of the main categories, or it may be categorized as either time-sensitive or computational; that is, the classification criteria for different intents are not orthogonal. Therefore, this embodiment separates the timeliness and computational categories from the main category, forming two single categories. This ensures that the main category and the two single categories are mutually exclusive, and each single category can be judged independently, outputting its respective intent label.

[0184] In this example, intent classification is divided into three dimensions: primary classification, timeliness classification, and computational classification. The three intent sub-models in the intent classification model are used to identify intents in primary classification, timeliness classification, and computational classification, respectively.

[0185] The main category categorizes queries into intent tags such as general knowledge, industry knowledge, casual conversation, language tasks, and persona. Different categories can be routed to different modules in subsequent processes. The boundaries between the main categories are relatively clear, and the entire set can cover all common queries in the RAG system. For common database sharding queries and industry segmentation requirements, categories such as industry knowledge can be further broken down to achieve more refined intent classification. Table 1 illustrates the different intent tags in the main category and their uses.

[0186] Table 1 Examples of different intent labels in the main category

[0187] The timeliness classification categorizes queries into time-sensitive and non-time-sensitive categories, which can serve as a basis for determining whether to sort the retrieved documents by time. For example, when a query is classified as time-sensitive, multiple web pages can be sorted in reverse chronological order to obtain the final result; when a query is classified as non-time-sensitive, multiple web pages can be sorted according to criteria such as the number of queries or comments to obtain the final result. Table 2 illustrates the different intent tags in the timeliness classification and their uses.

[0188] Table 2 Examples of different intent labels in the timeliness category

[0189] The computational classification categorizes queries into computational and non-computational categories, which can serve as a basis for determining whether subsequent large-scale models need to invoke computational tools such as calculators. Table 3 provides an illustrative example of the different intent labels and their uses in the computational classification.

[0190] Table 3 Examples of different intent labels in computational classification

[0191] Based on the methods in the above embodiments, this application also provides a query planning device.

[0192] For example, Figure 6 shows a schematic diagram of the composition of a query planning device provided in an embodiment of this application. As shown in Figure 6, this application provides a query planning device 600, including: an acquisition module 610, used to acquire historical query dialogues and the current query; a processing module 620, used to take the historical query dialogues and the current query as input to a query rewriting decomposition model and output a subquery list; and, based on the subquery list, to determine a query planning result, wherein the query planning result indicates the query planning for the current query. The subquery list includes several subquery tasks.

[0193] In one possible implementation, the number of parameters in the query-rewritten decomposition model is less than or equal to a first threshold.

[0194] In one possible implementation, the query rewrite decomposition model is fine-tuned based on a first dataset, which includes the current query sample, historical query dialogue samples, and truth values ​​of the subquery list.

[0195] In one possible implementation, the processing module 620 is further configured to: detect whether the subquery list is correct; when the subquery list is incorrect, correct the subquery list; update the first dataset based on the corrected subquery list; and fine-tune the query rewrite decomposition model based on the updated first dataset.

[0196] In one possible implementation, the truth values ​​of the subquery list are obtained by rewriting and / or decomposing the current query sample based on historical query dialogue samples using a large language model.

[0197] In one possible implementation, the processing module 620 is specifically used to: process the historical query dialogue and the current query according to the prompt word template to obtain a first prompt word, so that the format of the first prompt word is the same as the format of the second prompt word in the fine-tuning process of the query rewriting decomposition model; and use the first prompt word as input to the query rewriting decomposition model to output a list of subqueries.

[0198] In one possible implementation, the processing module 620 is specifically used to: take the current query, subquery list, and intent tag database as input to the intent classification model, output the target intent set to which the current query belongs, the intent tag database represents multiple intent tags corresponding to different classification systems, and the target intent set includes several target intents corresponding to the current query under different classification systems; and determine the query planning result based on the subquery list and the target intent set.

[0199] In one possible implementation, the query planning result includes a set of query operations and query path information. The set of query operations represents multiple query operations performed on several subquery tasks in the subquery list. The query path information indicates the routing of the subquery list to a target downstream task among multiple downstream query tasks, and / or the requirements for the target downstream task. Specifically, the processing module 620 is used to: determine the set of query operations based on the subquery list; and determine the query path information based on the target intent set.

[0200] In one possible implementation, the intent classification model includes N intent sub-models, each used to identify the target intent of the current query under one of the N classification systems. The intent tag database includes N intent tag sub-databases corresponding to the N intent sub-models, and the target intent set includes N target intents corresponding to the N intent sub-models, where N is an integer greater than or equal to 1. The processing module 620 is specifically used to: take the current query, the subquery list, and the corresponding intent tag sub-database as input to each intent sub-model, and output the corresponding target intent; based on the N target intents output by the N intent sub-models, obtain the target intent set.

[0201] In one possible implementation, the intent sub-model is configured based on the number of intent tags in the corresponding intent tag sub-database. When the number of intent tags in the corresponding intent tag sub-database is less than or equal to a second threshold, the intent sub-model includes a first fine-ranking model. The processing module 620 is specifically used to: take the current query, the subquery list, and the corresponding intent tag sub-database as input to the first fine-ranking model, and output a first intent tag sort; and determine the target intent based on the first intent tag sort.

[0202] In one possible implementation, when the number of intent tags in the corresponding intent tag sub-database exceeds a second threshold, the intent sub-model includes a recall model and a second ranking model. The processing module 620 is specifically used to: take the current query, subquery list, and intent tag sub-database as input to the recall model and output a set of recalled intent tags; take the current query, subquery list, and the set of recalled intent tags as input to the second ranking model and output a second intent tag ranking; and determine the target intent based on the second intent tag ranking.

[0203] In one possible implementation, the intent classification model is fine-tuned based on a second dataset, which includes query samples, intent label database samples, target category labels, and non-target category labels.

[0204] In one possible implementation, the processing module 620 is further configured to: detect whether the target intent set is correct; when the target intent set is incorrect, correct the target intent set; update the second dataset based on the corrected target intent set; and fine-tune the intent classification model based on the updated second dataset.

[0205] In one possible implementation, the target category label and the non-target category label are obtained by inputting the query sample into a large language model and then outputting the results.

[0206] In one possible implementation, the intent classification model includes three intent sub-models: a topic-based intent sub-model, a time-sensitive intent sub-model, and a computational intent sub-model. The topic-based intent sub-model identifies user intents categorized by topic; the time-sensitive intent sub-model identifies user intents categorized by time sensitivity; and the computational intent sub-model identifies user intents categorized by computation. The intent tag database includes a topic-based intent tag database, a time-sensitive intent tag database, and a computational intent tag database. The target intent set includes topic-based intents, time-sensitive intents, and computational intents. Specifically, the processing module 620 is used to: take the current query, subquery list, and topic-based intent tag database as input to the topic-based intent sub-model and output topic-based intents; take the current query, subquery list, and time-sensitive intent tag database as input to the time-sensitive intent sub-model and output time-sensitive intents; and take the current query, subquery list, and computational intent tag database as input to the computational intent sub-model and output computational intents.

[0207] In one possible implementation, the topic-based intent indicates whether the subquery list should be routed to the target downstream task among multiple downstream query tasks; the timeliness-based intent indicates whether the processing results of the current query output by the target downstream task need to be sorted by time; and the computational intent indicates whether a calculator should be configured for the target downstream task.

[0208] In one possible implementation, the intent tags in the topic category include at least one of the following: general knowledge, industry knowledge, casual conversation, language task, and persona; the intent tags in the timeliness category include timeliness and non-timeliness; and the intent tags in the computational category include computational and non-computational.

[0209] In one possible implementation, the processing module 620 is further configured to: configure the intent sub-models in the intent model according to the user's model selection information, each intent sub-model being used to identify the target intent of a classification system.

[0210] The implementation of each module in the query planning device shown in Figure 6 will be described below as an example.

[0211] In some possible implementations, multiple modules in the query planning device can be implemented in software or hardware. For example, the implementation of module 610 will be described below. Similarly, the implementation of the other modules can refer to the implementation of module 610.

[0212] As an example of a software functional unit, module 610 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, or a container. Further, the aforementioned computing instance may be one or more. For example, module 610 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed within the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed within the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0213] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0214] As an example of a hardware functional unit, the acquisition module 610 may include at least one computing device, such as a server. Alternatively, the acquisition module 610 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0215] The multiple computing devices included in the acquisition module 610 can be distributed in the same region or in different regions. Similarly, the multiple computing devices included in the module can be distributed in the same Availability Zone (AZ) or in different AZs. Likewise, the multiple computing devices included in the module can be distributed in the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0216] It should be noted that in other embodiments, the query planning device additionally provides one or more modules for performing any of the steps included in the above implementation. The steps implemented by one or more modules in the query planning device can be specified as needed. The functionality of the query planning device can also be implemented using more or fewer modules than in the embodiments of this application. One or more modules in the query planning device are used to implement different steps in the above methods, thereby realizing all the functions of the query planning device.

[0217] This application also provides a computing device 700. As shown in FIG7, the computing device 700 includes: a bus 702, a processor 704, a memory 706, and a communication interface 708. The processor 704, the memory 706, and the communication interface 708 communicate with each other via the bus 702. The computing device 700 can be a server, such as a central server, an edge server, or a local server in a local data center, or it can be an electronic device such as a desktop computer, a laptop computer, or a smartphone. It should be understood that this application does not limit the number of processors and memories in the computing device 700.

[0218] Bus 702 can be a Peripheral Component Interconnect (PCI) bus, a Peripheral Component Interconnect Express (PCIe) bus, an Extended Industry Standard Architecture (EISA) bus, a Unified Bus (Ubus or UB), a Compute Express Link (CXL), a Cache Coherent Interconnect for Accelerators (CCIX), etc. The Unified Bus is also known as the Lingqu Bus. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 7 uses only one line, but this does not imply that there is only one bus or one type of bus. Bus 704 can include pathways for transmitting information between various components of computing device 700 (e.g., memory 706, processor 704, communication interface 708).

[0219] The processor 704 may include any one or more computing devices such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP) or a digital signal processor (DSP), an ASIC, an FPGA, a CPLD, an NPU, a SoC, an offload card, or an accelerator card.

[0220] Memory 706 may include volatile memory, such as random access memory (RAM). Processor 704 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD). Furthermore, memory 706 may also be implemented using storage class memory (SCM), phase change memory (PCM), or other types of storage media.

[0221] It is worth noting that the same type of storage medium can be configured in the same computing device to realize the function of memory 706, or two or more types of storage media can be configured to realize the function of memory 706. This application does not limit this.

[0222] The memory 706 stores executable program code, and the processor 704 executes the executable program code to implement the functions of one or more modules of the query planning device, thereby implementing the method described in the above embodiments. That is, the memory 706 stores instructions for executing the method described in the above embodiments.

[0223] Alternatively, the memory 706 stores executable code, which the processor 704 executes to implement the functions of the query planning device, thereby implementing the method described in the above embodiments. That is, the memory 706 stores instructions for executing the method described in the above embodiments.

[0224] The communication interface 708 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 700 and other devices or communication networks.

[0225] As one possible implementation, the computing device 700 may also include a chip system, or this application embodiment further provides a chip system. The chip system includes a processor and a power supply circuit, the power supply circuit being used to supply power to the processor, and the processor being used to execute the operation steps corresponding to the method of this application embodiment. For simplicity, further details are omitted here. The processor can be implemented using a GPU, or it can be implemented using computing devices or AI chips such as a DPU, NPU, XPU, SoC, offloading card, or accelerator card.

[0226] As one possible implementation, the computing device 700 may include various types of processors 704, meaning the computing device 700 is a heterogeneous device. For example, the computing device 700 may include a CPU and a GPU, and at least one of the processors 704 may execute the operation steps corresponding to the methods of the embodiments of this application. For the sake of brevity, further details are omitted here.

[0227] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be an electronic device such as a desktop computer, a laptop computer, or a smartphone.

[0228] As shown in Figure 8, the computing device cluster includes at least one computing device 700. The memory 706 of one or more computing devices 700 in the computing device cluster may store the same instructions for performing the methods described in the above embodiments.

[0229] In some possible implementations, the memory 706 of one or more computing devices 700 in the computing device cluster may also store partial instructions for executing the methods described in the above embodiments. In other words, a combination of one or more computing devices 700 can jointly execute instructions for executing the methods described in the above embodiments.

[0230] It should be noted that the memory 706 in different computing devices 700 within the computing device cluster can store different instructions, each used to execute a portion of the functions of the query planning device. That is, the instructions stored in the memory 706 of different computing devices 700 can implement the functions of one or more modules of the query planning device.

[0231] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 9 illustrates one possible implementation. As shown in Figure 9, two computing devices 700A and 700B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this type of possible implementation, the memory 706 in computing device 700A stores instructions for querying the functions of one or more modules in the planning device. Simultaneously, the memory 706 in computing device 700B stores instructions for querying the functions of another one or more modules in the planning device.

[0232] It should be understood that the functions of computing device 700A shown in Figure 9 can also be performed by multiple computing devices 700. Similarly, the functions of computing device 700B can also be performed by multiple computing devices 700.

[0233] This application also provides another computing device cluster. The connection relationship between the computing devices in this computing device cluster can be similarly referred to the connection method of the computing device cluster described in Figures 8 and 9. The difference is that the memory 706 of one or more computing devices 700 in this computing device cluster can store the same instructions for executing the methods in the above embodiments.

[0234] In some possible implementations, the memory 706 of one or more computing devices 700 in the computing device cluster may also store partial instructions for executing the aforementioned data processing method. In other words, a combination of one or more computing devices 700 can jointly execute the instructions for performing the aforementioned method.

[0235] Based on the methods in the above embodiments, this application provides a computer-readable storage medium including computer program instructions. When executed by a cluster of computing devices including at least one computing device, the computer program instructions cause the cluster of computing devices to perform the methods in the above embodiments. Exemplarily, the computer-readable storage medium can be any available medium capable of being stored in the cluster of computing devices or a data storage device such as a data center containing one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives).

[0236] Based on the methods in the above embodiments, this application provides a computer program product containing instructions. The computer program product may be software or program products containing instructions, capable of running on a computing device or stored on any available medium. When the instructions are executed by a cluster of computing devices containing at least one computing device, at least one computing device in the cluster of computing devices performs the methods in the above embodiments.

[0237] It is understood that the processor in the embodiments of this application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor.

[0238] The method steps in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0239] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0240] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.

[0241] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A query planning method, characterized in that, include: Retrieve historical query conversations and the current query; The historical query dialogue and the current query are used as inputs to the query rewriting decomposition model, and a subquery list is output, which includes several subquery tasks. Based on the subquery list, a query planning result is determined, which indicates the query plan for the current query.

2. The method according to claim 1, characterized in that, The number of parameters in the query rewrite decomposition model is less than or equal to the first threshold.

3. The method according to claim 1 or 2, characterized in that, The query rewriting decomposition model is fine-tuned based on a first dataset, which includes current query samples, historical query dialogue samples, and truth values ​​of the subquery list.

4. The method according to claim 3, characterized in that, Also includes: Check if the subquery list is correct; If the subquery list is incorrect, the subquery list shall be corrected. Update the first dataset based on the revised list of subqueries; Based on the updated first dataset, the query rewrite decomposition model is fine-tuned.

5. The method according to any one of claims 1-4, characterized in that, The process of taking the historical query dialogues and the current query as input to the query rewriting decomposition model and outputting a list of subqueries includes: Based on the prompt word template, the historical query dialogue and the current query are processed to obtain the first prompt word, so that the format of the first prompt word is the same as the format of the second prompt word in the fine-tuning process of the query rewriting decomposition model; The first prompt word is used as input to the query rewrite decomposition model, and the subquery list is output.

6. The method according to any one of claims 1-5, characterized in that, The process of determining the query plan result based on the subquery list includes: The current query, the subquery list, and the intent tag database are used as inputs to the intent classification model, and the target intent set to which the current query belongs is output. The intent tag database represents multiple intent tags corresponding to different classification systems, and the target intent set includes several target intents corresponding to the current query under different classification systems. The query planning result is determined based on the subquery list and the target intent set.

7. The method according to claim 6, characterized in that, The query planning result includes a set of query operations and query path information. The set of query operations represents multiple query operations performed on several subquery tasks in the subquery list. The query path information indicates the routing of the subquery list to a target downstream task among multiple downstream query tasks, and / or the requirements for the target downstream task. The step of determining the query planning result based on the subquery list and the target intent set includes: Based on the subquery list, determine the set of query operations; The query path information is determined based on the set of target intents.

8. The method according to claim 6 or 7, characterized in that, The intent classification model includes N intent sub-models, each of which is used to identify the target intent of the current query under N classification systems. The intent tag database includes N intent tag sub-databases corresponding to the N intent sub-models. The target intent set includes N target intents corresponding to the N intent sub-models, where N is an integer greater than or equal to 1. The step of taking the current query, the subquery list, and the intent tag database as input to the intent classification model and outputting the target intent set to which the current query belongs includes: The current query, the sub-query list, and the corresponding intent tag sub-database are used as inputs to each intent sub-model, and the corresponding target intent is output. The target intent set is obtained based on the N target intents output by the N intent sub-models.

9. The method according to claim 8, characterized in that, The intent sub-model is configured based on the number of intent tags in the corresponding intent tag sub-database. When the number of intent tags in the corresponding intent tag sub-database is less than or equal to a second threshold, the intent sub-model includes a first fine-ranking model. The step of taking the current query, the sub-query list, and the corresponding intent tag sub-database as input to each intent sub-model and outputting the corresponding target intent includes: The current query, the subquery list, and the corresponding intent tag sub-database are used as inputs to the first fine ranking model, and the first intent tag sorting is output. The target intent is determined based on the sorting of the first intent tags.

10. The method according to claim 9, characterized in that, When the number of intent tags in the corresponding intent tag sub-database is greater than the second threshold, the intent sub-model includes a recall model and a second fine ranking model; The step of taking the current query, the sub-query list, and the corresponding intent tag sub-database as input to each intent sub-model and outputting the corresponding target intent includes: The current query, the subquery list, and the intent tag sub-database are used as inputs to the recall model, and the output is a set of recalled intent tags. The current query, the subquery list, and the recall intent tag set are used as inputs to the second fine-ranking model, and the output is the second intent tag ranking. The target intent is determined based on the sorting of the second intent tags.

11. The method according to any one of claims 6-10, characterized in that, The intent classification model is fine-tuned based on a second dataset, which includes query samples, intent label database samples, target category labels, and non-target category labels.

12. The method according to claim 11, characterized in that, Also includes: Check whether the target intent set is correct; When the target intent set is incorrect, the target intent set is corrected; Update the second dataset based on the revised set of target intents; Based on the updated second dataset, the intent classification model is fine-tuned.

13. The method according to any one of claims 6-12, characterized in that, The intent classification model includes three intent sub-models: a topic-based intent sub-model, a time-sensitive intent sub-model, and a computational intent sub-model. The topic-based intent sub-model is used to identify user intents categorized by topic, the time-sensitive intent sub-model is used to identify user intents categorized by time sensitivity, and the computational intent sub-model is used to identify user intents categorized by computation. The intent tag database includes a topic-based intent tag database, a time-sensitive intent tag database, and a computational intent tag database. The target intent set includes topic-based intents, time-sensitive intents, and computational intents. The step of taking the current query, the subquery list, and the intent tag database as input to the intent classification model and outputting the target intent set of the current query includes: The current query, the subquery list, and the topic-type intent tag database are used as inputs to the topic-type intent sub-model, and the topic-type intent is output. The current query, the subquery list, and the timeliness-related intent tag database are used as inputs to the timeliness-related intent sub-model, and the timeliness-related intent is output. The current query, the subquery list, and the computational intent tag database are used as inputs to the computational intent sub-model, and the computational intent is output.

14. The method according to claim 13, characterized in that, The topic-based intent indicates whether the subquery list should be routed to a target downstream task among multiple downstream query tasks; the timeliness-based intent indicates whether the processing result of the current query output by the target downstream task needs to be sorted by time; and the computational intent indicates whether a calculator should be configured for the target downstream task.

15. The method according to claim 13 or 14, characterized in that, The topic categories include at least one of the following: general knowledge, industry knowledge, casual conversation, language task, and persona; the timeliness categories include time-sensitive and non-time-sensitive categories; and the computational categories include computational and non-computational categories.

16. The method according to any one of claims 6-15, characterized in that, Also includes: Based on the user's model selection information, the intent sub-models in the intent model are configured, and each intent sub-model is used to identify the target intent of a classification system.

17. A query planning device, characterized in that, include: The retrieval module is used to retrieve historical query conversations and the current query; The processing module is used to take the historical query dialogue and the current query as input to the query rewriting decomposition model and output a subquery list, which includes several subquery tasks. Furthermore, based on the subquery list, a query planning result is determined, the query planning result indicating the query plan for the current query.

18. A search and question-answering system, characterized in that, include: A query planning module is configured to execute the method described in any one of claims 1-16 to generate a query planning result for the current query; The query processing module is used to generate the query processing result of the current query under the guidance of the query planning result.

19. A chip system, characterized in that, The chip system includes a processor and a power supply circuit, the power supply circuit being used to supply power to the processor, the processor being used to perform the method as described in any one of claims 1-16.

20. A computing device, characterized in that, The computing device includes a processor and memory; The processor is configured to execute instructions stored in the memory to cause the computing device to perform the method as described in any one of claims 1-16.

21. A computing device cluster, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1-16.

22. A computer-readable storage medium, characterized in that, The method includes computer program instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method as described in any one of claims 1-16, wherein the cluster of computing devices includes at least one computing device.

23. A computer program product containing instructions, characterized in that, When the instruction is executed by the computing device cluster, the computing device cluster causes the computing device cluster to perform the method as described in any one of claims 1-16, wherein the computing device cluster includes at least one computing device.

Images