Task planning method for large model retrieval enhanced generation

Abstract

The application discloses a task planning method for large model retrieval enhanced generation, and belongs to the technical field of natural language processing. The method comprises the following steps: inputting a user query into a first classifier, a second classifier and a third classifier in parallel; performing intent recognition on the user query through the first classifier, the second classifier and the third classifier respectively, and determining the feature scores of the user query in multiple preset dimensions; fusing the scores of the multiple preset dimensions into a retrieval trigger probability; obtaining a dynamic threshold value, comparing the retrieval trigger probability with the dynamic threshold value, and obtaining a determination result of whether to trigger retrieval enhanced generation; in the case that the determination result is to trigger retrieval enhanced generation, calling an external knowledge base and a large language model to generate a final answer; and in the case that the determination result is not to trigger retrieval enhanced generation, calling a large language model to generate a final answer. The method effectively overcomes the efficiency and accuracy imbalance problem caused by the "one-size-fits-all" retrieval strategy of the traditional method.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing technology, specifically relating to a task planning method for enhanced generation of large-scale model retrieval. Background Technology

[0002] In recent years, large language models have demonstrated significant advantages in tasks such as open-domain question answering, dialogue systems, and decision support due to their powerful general reasoning and text generation capabilities. However, the knowledge of large models is limited by the static and closed nature of the training data, making it difficult to handle problems with high timeliness, specialized domains, or scarce knowledge. To address this, Retrieval-Enhanced Generation (RAG) technology has been proposed. By retrieving external knowledge bases during generation tasks, it dynamically supplements the information lacking in the model, thereby improving the accuracy and reliability of the answers, and has gradually become a means to enhance the practicality of large models.

[0003] While RAG technology has improved the reasoning capabilities of large models to some extent, it generally employs a fixed retrieval strategy, applying the retrieval enhancement process indiscriminately to all types of user queries. This indiscriminate approach, when faced with simple queries (such as factual questions), results in unnecessary retrieval operations that not only increase response latency and computational resource consumption but may also introduce irrelevant information that interferes with the generation process, reducing answer quality. Summary of the Invention

[0004] The purpose of this invention is to provide a task planning method for enhanced generation of large model retrieval, which can solve the problems existing in the background technology.

[0005] To solve the above-mentioned technical problems, the present invention is implemented as follows: This invention provides a task planning method for enhancing the generation of large model retrieval, comprising: User queries are input in parallel to a first classifier, a second classifier, and a third classifier. The first, second, and third classifiers respectively perform intent recognition on the user query, determining feature scores for the user query across multiple preset dimensions. These feature scores include at least a task type score, an information completeness score, and a complexity score. The task type score indicates the degree of dependence of the user query on external knowledge, the information completeness score indicates the degree of supplementation of external knowledge required for the user query, and the complexity score indicates the reasoning difficulty of the user query. The first, second, and third classifiers are pre-trained using a labeled dataset to fine-tune a pre-trained language model. The labeled dataset is constructed based on task type, information completeness, and complexity. The scores from the multiple preset dimensions are merged into a search trigger probability; A dynamic threshold is obtained, and the retrieval trigger probability is compared with the dynamic threshold to obtain a determination result on whether retrieval enhancement generation is triggered; If the determination result indicates that the search enhancement generation has been triggered, an external knowledge base and a large language model are invoked to generate the final answer. If the determination result is that the retrieval enhancement generation is not triggered, the large language model is invoked to generate the final answer.

[0006] Optionally, the user query is input in parallel to a first classifier, a second classifier, and a third classifier. The first classifier, the second classifier, and the third classifier, respectively, perform intent recognition on the user query to determine the feature scores of the user query on multiple preset dimensions, including: The user query is split into multiple words to obtain a query character sequence; The query character sequence is input into a pre-trained first classifier to determine the degree of dependence of the user query on external knowledge, and the task type score is obtained. The task type includes factual query, inference query and open query. The query character sequence is input into a pre-trained second classifier. Based on privacy, sensitivity, real-time performance, availability, and contextual relevance, the degree of supplementation of external knowledge required by the user query is evaluated to obtain the information completeness score. The query character sequence is input into a pre-trained third classifier to determine the reasoning difficulty of the user query and obtain the complexity score. The complexity level includes basic, intermediate and expert levels.

[0007] The technical solutions provided by the embodiments of the present invention bring at least the following beneficial effects: This invention utilizes parallelized, lightweight first, second, and third classifiers to perform multi-dimensional feature scoring and fusion of user queries, effectively overcoming the efficiency and accuracy imbalance caused by the "one-size-fits-all" retrieval strategy in traditional retrieval enhancement generation techniques. It can offload simple queries to a direct generation path, significantly reducing response latency and computational overhead; simultaneously, it ensures that complex queries reliably trigger the retrieval enhancement process, guaranteeing the accuracy and depth of the answer by introducing precise external knowledge, thereby suppressing model illusions. Furthermore, the task planning proposed in this invention does not depend on a specific model architecture or internal state, possessing good versatility and scalability, and can be adapted to various large language models and external knowledge bases, providing an efficient and reliable solution for differentiated query processing. Attached Figure Description

[0008] Figure 1This is a schematic diagram illustrating the steps of a task planning method for enhanced generation of large model retrieval, provided in one embodiment of the present invention. Figure 2 This is an overall flowchart of the task planning method for enhanced retrieval generation in one embodiment of the present invention; Figure 3 This is a schematic diagram of the multi-dimensional feature scoring and fusion process in one embodiment of the present invention; Figure 4 This is a schematic diagram of the dynamic threshold determination process in one embodiment of the present invention. Detailed Implementation

[0009] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0010] The terms "first," "second," etc., used in the specification and claims of this invention are used to distinguish similar objects and are not used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0011] The following description, in conjunction with the accompanying drawings, details a task planning method for enhanced generation of large model retrieval provided by the present invention through specific embodiments and application scenarios.

[0012] Figure 1 This is a schematic diagram illustrating the steps of a task planning method for enhanced generation of large model retrieval, provided in an embodiment of the present invention. Please refer to [link / reference]. Figure 1 ,include: Step S11: The user query is input in parallel to the first classifier, the second classifier, and the third classifier. The first classifier, the second classifier, and the third classifier respectively perform intent recognition on the user query to determine the feature scores of the user query on multiple preset dimensions. The feature scores of the user query on multiple preset dimensions include at least a task type score, an information completeness score, and a complexity score. The task type score is used to indicate the degree of dependence of the user query on external knowledge. The information completeness score is used to indicate the degree of supplementation of external knowledge required by the user query. The complexity score is used to indicate the reasoning difficulty of the user query. The first classifier, the second classifier, and the third classifier are pre-trained by fine-tuning a pre-trained language model using a labeled dataset. The labeled dataset is constructed based on task type, information completeness, and complexity.

[0013] Figure 2 This is an overall flowchart of the task planning method for retrieval enhancement generation in one embodiment of the present invention. Please refer to [link / reference]. Figure 2 Receiving user queries (i.e.) Figure 2 The query shown is a user-submitted query text, processed using a parallel architecture. The user query is simultaneously input into three independent lightweight classifiers: a first classifier, a second classifier, and a third classifier. These three classifiers are lightweight models fine-tuned from pre-trained language models. The pre-trained language models can be RoBERTa, BERT, LLaMA, or other Transformer architecture models. Each classifier is responsible for semantic analysis and intent recognition of the user query from different dimensions, outputting corresponding feature scores, including task type score, information completeness score, and complexity score, which together constitute a comprehensive quantitative evaluation of the user query.

[0014] Task type scoring measures the degree to which user queries rely on external knowledge resources. Specifically, the first classifier categorizes user queries into various task types, such as factual queries, inferential queries, and open-ended queries. Factual queries involve objective and verifiable information, such as data queries or event descriptions, and have a high task type score, indicating a need for supplementation from external knowledge bases. Inferential queries require logical deduction or causal analysis and have a medium task type score, indicating a need for some external support. Open-ended queries focus on creative or subjective content, such as opinion generation or creative writing, and have a low task type score, indicating that they can be completed primarily by relying on the internal knowledge of the large language model.

[0015] The information completeness score assesses the completeness of the information carried in a user query and whether additional information from an external knowledge base is needed to complete the answer. The second classifier performs a fine-grained evaluation of the query based on multiple sub-dimensions, including privacy, sensitivity, real-time performance, availability, and contextual relevance. The privacy dimension determines whether the query involves personal or confidential information and whether it requires external, non-public knowledge; the sensitivity dimension assesses whether the query involves topics or areas that the large language model itself cannot answer; the real-time performance dimension measures whether the query requires the latest or dynamically updated information, such as news or policy changes; the availability dimension determines whether the required information is easily obtained from an authorized external knowledge base, which refers to a third-party knowledge source that has undergone content security review, complies with cooperation agreements, and is authorized for use by this system; and the contextual relevance analyzes the degree of relevance between the query and the current dialogue history or scenario to determine whether supplementary contextual information is needed. Each sub-dimension is scored using an independent lightweight discriminant model, and then weighted and fused according to preset weights to form a comprehensive information completeness score. A higher information completeness score indicates that the user query relies more on external knowledge supplementation.

[0016] Complexity scores quantify the reasoning difficulty of user queries, including the length of the required logical reasoning chain, the breadth of information integration, and the depth of analysis. A third classifier categorizes queries into multiple complexity levels, such as basic, intermediate, and expert. Basic queries are typically simple, straightforward questions requiring fewer reasoning steps; intermediate queries involve multi-step reasoning or partial information integration; and expert queries involve cross-domain knowledge fusion, deep logical analysis, or complex computation. The classifier learns the characteristics of queries of different complexities through a progressive training strategy and outputs corresponding scores. Higher complexity scores indicate more complex queries and greater processing difficulty.

[0017] In an optional implementation, step S11 specifically includes steps S111 to S114: Step S111: The user query is split into multiple words to obtain a query character sequence.

[0018] Figure 3 This is a schematic diagram of the multi-dimensional feature scoring and fusion process in one embodiment of the present invention. Please refer to [link / reference]. Figure 3 Preprocessing user queries (i.e.) Figure 2 The general routing stage (shown) transforms the user query into a character sequence that the model can process. Specifically, the user query is a text string input in natural language, such as "Please explain the relationship between corporate tax refund policies and social security contribution years". To facilitate deeper semantic understanding and feature extraction by the subsequent classifier, the query is first segmented into words.

[0019] The word segmentation process relies on a predefined tokenizer, which is constructed based on the vocabulary of the selected pre-trained language model. The tokenizer segments continuous user query text into a series of discrete lexical or subword units, which are referred to as tokens. For example, the above query is segmented into the following token sequence: "please", "explain", "enterprise", "tax rebate", "policy", "and", "social security", "payment", "years", "of", "correlation". Each token corresponds to a unique identifier in the model, and the entire query is thus converted into a sequence composed of token identifiers, that is, a query character sequence.

[0020] In step S112, the query character sequence is input into a pre-trained first classifier to determine the dependence degree of the user query on external knowledge, so as to obtain the task type score. The task types include fact-based query, reasoning-based query and open-type query.

[0021] See Figure 3 , input the query character sequence into the first classifier. The first classifier is a fine-tuned lightweight pre-trained language model, which is used to identify the task type of the user query and output a corresponding task type score. The task type score is used to quantify the dependence degree of the query on external knowledge.

[0022] The task types include three categories: fact-based query, reasoning-based query and open-type query: the fact-based query refers to an inquiry involving objective facts, data or specific knowledge, the answer of which exists in an external knowledge base and has a high dependence on external knowledge; the reasoning-based query requires the model to perform logical inference, causal analysis or multi-step derivation, and partially depends on external knowledge; the open-type query focuses on opinion generation, creative divergence or subjective content, which can be completed mainly by relying on the internal knowledge of the large language model.

[0023] In an optional implementation, step S112 specifically includes step S1121 to step S1126: In step S1121, the query character sequence is input into the first classifier, and each query character in the query character sequence is mapped to a corresponding low-dimensional vector through a character embedding lookup table, so as to obtain a character vector sequence.

[0024] Input the query character sequence into the first classifier. The first classifier is internally configured with a character embedding lookup table, which is essentially a high-dimensional matrix, the number of rows of which is consistent with the size of the predefined vocabulary, and the number of columns corresponds to the dimension of the embedding vector. Each row in the character embedding lookup table represents the semantic representation of a specific character or lexical unit in the continuous vector space.

[0025] The process iterates through each character in the query sequence, retrieving its corresponding low-dimensional real vector from the character embedding lookup table based on its unique identifier in a predefined vocabulary. This transforms the originally discrete, symbolic character sequence into a continuous, semantically rich vector sequence—a character vector sequence. Each vector captures the semantic and syntactic attributes of the corresponding character. The parameters of the character embedding lookup table are learned during the pre-training phase and adaptively adjusted during fine-tuning according to the specific task to better reflect the semantic features required for task-type classification.

[0026] Step S1122: The character vector sequence is processed by a multi-layer self-attention encoder to obtain a latent vector sequence.

[0027] The character vector sequence is input into a multi-layer self-attention encoder structure. The self-attention encoder consists of multiple identical network layers stacked together, each layer containing a self-attention mechanism and a feedforward neural network.

[0028] A self-attention encoder, while processing the vector of each character, simultaneously considers the vectors of all other characters in the character vector sequence, thereby dynamically calculating the importance weight of each character relative to the other characters in the sequence. In this way, the character vector sequence can capture long-distance dependencies and complex contextual semantic relationships between characters, such as identifying core entities, actions, and their logical relationships in a query. A feedforward neural network then performs a non-linear transformation on the output of the self-attention mechanism, further enhancing the model's representational capabilities.

[0029] The character vector sequence passes through multiple such encoding layers. At each layer, the representation of the character vector sequence is further refined and deepened, ultimately outputting a completely new vector sequence, namely the latent vector sequence. Each latent vector in the latent vector sequence deeply integrates its own information with the contextual semantics of the entire query character sequence, providing rich feature representations for subsequent global semantic aggregation.

[0030] Step S1123: Perform a pooling operation on the latent vector sequence to obtain a query vector representation that characterizes the overall semantics of the user query.

[0031] The latent vector sequence is aggregated into a fixed-dimensional vector to represent the global semantics of the user query. Pooling is used to achieve this. Specifically, the latent vector corresponding to a special marker at the beginning of the latent vector sequence is selected as the input to the pooling operation. In the pre-trained language model, the special marker is used to aggregate the semantic information of the entire latent vector sequence. By directly extracting the latent vector corresponding to the special marker and using it as a semantic summary of the user query, the query vector representation is obtained. The advantage of this pooling strategy lies in its efficiency and effectiveness. The query vector representation compactly encodes the core intent and all semantic content of the user query.

[0032] Step S1124: Input the query vector representation into a multi-class classifier to obtain the probability distribution of the user query belonging to each task type; the multi-class classifier is obtained by fine-tuning a pre-trained language model based on a first labeled dataset containing multiple historical user queries and the corresponding task type labels for each historical user query.

[0033] The query vector representation is input into a multi-class classifier. The multi-class classifier is essentially a multilayer perceptron, whose input layer dimension is the same as the dimension of the query vector representation, and whose output layer dimension is the same as the number of categories of the task type (three categories in this embodiment).

[0034] The classifier performs nonlinear transformations and calculations on the input query vector, ultimately outputting a numerical vector. After normalization using the Softmax function, each element represents the probability that the user query belongs to one of three task types: factual, inference, or open-ended. The sum of all probability values ​​is one. This probability distribution quantifies the uncertainty of a query belonging to each task type, providing a probabilistic basis for the final type determination.

[0035] The multi-class classifier does not exist in isolation; the first classifier is obtained through end-to-end fine-tuning of a pre-trained language model based on a first labeled dataset. This first labeled dataset contains a large number of historical user query samples, each labeled with a true task type label (factual, inferential, or open-ended) by humans or through a powerful model (such as GPT-4). During fine-tuning, all parameters of the first classifier (including the character embedding lookup table, multi-layer self-attention encoder, and the top-level multi-class classifier) ​​are collaboratively optimized under the supervision of the labeled data to minimize the difference between the predicted task type and the task type label. Through this supervised learning, the first classifier learns to extract the features most relevant to task type discrimination from the original query text.

[0036] Step S1125: Determine the task type to which the user query belongs based on the probability distribution of the user query belonging to each task type.

[0037] After obtaining the probability distribution of user queries belonging to each task type, the decision is made using the maximum probability criterion. That is, the probability values ​​corresponding to factual, inferential, and open-ended questions in the probability distribution are compared, and the task type with the highest probability value is selected as the final type of the user query. For example, if the query has the highest probability of being classified as factual, then the query is determined to be a factual query.

[0038] Step S1126: Based on the preset task type rating mapping rule, the task type to which the user query belongs is mapped to a numerical rating to obtain the task type rating; wherein, the task type rating mapping rule maps fact-based queries, reasoning queries and open queries to different numerical values, and the numerical value corresponding to fact-based queries is the highest, and the numerical value corresponding to open queries is the lowest.

[0039] The conversion is performed according to the preset task type scoring mapping rules. The task type scoring mapping specifies the numerical scores corresponding to three task types. Its core design principle is that the score should directly reflect the degree of dependence of that type of query on the external knowledge base.

[0040] Based on this principle, factual queries, because their answers explicitly exist in an external knowledge base, have the highest dependence on external knowledge and are therefore mapped to the highest numerical score. Inference queries partially rely on external knowledge for logical verification, with a moderate dependence, and are therefore mapped to a medium numerical score. Open-ended queries primarily rely on knowledge generated within the model, with the lowest dependence on external knowledge, and are therefore mapped to the lowest numerical score. For example, in a specific implementation, factual queries could be mapped to 3 points, inference queries to 2 points, and open-ended queries to 1 point. Through this mapping process, a specific numerical score for each task type is ultimately obtained.

[0041] Step S113: Input the query character sequence into a pre-trained second classifier. Based on privacy, sensitivity, real-time performance, availability, and contextual relevance, evaluate the degree of supplementation of external knowledge required for the user query to obtain the information completeness score.

[0042] Please see Figure 3The query character sequence is input into the second classifier. The second classifier is also a fine-tuned model of a lightweight pre-trained language model, used to comprehensively evaluate the information completeness of the user query from multiple dimensions and output an information completeness score. The information completeness score measures whether the query needs to be supplemented with information from an external knowledge base to complete the answer. In this embodiment, the evaluation of the second classifier is based on five dimensions: privacy, sensitivity, real-time performance, availability, and contextual relevance. Each dimension is binarized and judged by an independent sub-classifier, outputting a judgment result on whether external knowledge supplementation is needed for that evaluation dimension. The five evaluation dimensions have been explained above and will not be repeated here.

[0043] In an optional implementation, step S113 specifically includes steps S1131 to S1132: Step S1131: The query character sequence is input in parallel to multiple sub-classifiers in the second classifier to obtain binary judgment results for multiple evaluation dimensions; wherein, the multiple evaluation dimensions include privacy, sensitivity, real-time performance, availability, and contextual relevance; each sub-classifier is obtained by fine-tuning a pre-trained language model based on a second labeled dataset containing multiple historical user queries and the binary labels of each historical user query on the corresponding evaluation dimension; each sub-classifier corresponds to one of the evaluation dimensions of privacy, sensitivity, real-time performance, availability, and contextual relevance.

[0044] The query character sequence is simultaneously input into multiple independent sub-classifiers within the second classifier in a parallel processing manner. Each sub-classifier is specifically responsible for binarizing and judging the user query across multiple evaluation dimensions. In this embodiment, the multiple evaluation dimensions are: privacy, sensitivity, real-time performance, availability, and contextual relevance.

[0045] Each sub-classifier is structurally a lightweight pre-trained language model (e.g., based on the RoBERTa architecture) and is specifically fine-tuned for its evaluation dimension. This fine-tuning relies on a specially constructed second-label dataset. This dataset contains a large number of historical user query samples, each precisely labeled with a binary tag for each of the five dimensions by human experts or a powerful annotation model (such as GPT-4). For example, for the privacy dimension, the tag is labeled as privacy-related or not; for the real-time dimension, the tag is labeled as requiring real-time information or not, and so on.

[0046] During inference, each sub-classifier works independently, performing deep semantic analysis on its assigned evaluation dimension. The query character sequence is processed through an embedding layer and multiple encoders for contextual semantic encoding, and a pooling operation is used to obtain a global semantic representation of the query. Finally, a binary classification output layer produces a binary decision result. The binary decision result is a binary output, for example, yes or no, 1 or 0, whose specific meaning corresponds to whether the query urgently needs supplementary information from an external knowledge base in the current evaluation dimension.

[0047] Specifically, the privacy subclassifier analyzes whether the query text implicitly or explicitly involves personally identifiable information, trade secrets, unpublished data, or other content subject to privacy regulations. If determined to be yes, it means that processing the user query requires access to a specific, access-controlled, confidential knowledge source, rather than a public knowledge base.

[0048] Sensitivity subclassifiers are used to identify whether a query is related to a specific area that requires expert guidance.

[0049] The real-time subclassifier is used to evaluate whether the information sought in a query has a strong time-sensitive characteristic. It analyzes time keywords and dynamic event descriptions in the query to determine whether the answer depends on recently occurring events, real-time changing data (such as stock prices or weather warnings), or recently updated policies and regulations. If the determination is yes, it indicates that a retrieval must be made from an external dynamic knowledge base containing the latest information, and the static knowledge within the large language model cannot meet the requirements.

[0050] The availability subclassifier is used to query the probability and availability of sought information in an authorized, established external knowledge base. It does not determine whether the information itself exists, but rather, based on the existing content distribution of the knowledge base, determines whether the information falls within the base's general coverage and is easily retrieved through retrieval. If the determination is negative, it indicates that even if a retrieval is triggered, effective supplementation may not be obtained, thus prompting consideration of other response strategies.

[0051] The contextual relevance subclassifier analyzes the semantic relevance of the current user query to previous conversation history or application scenario. It determines whether the current query is an isolated question or heavily relies on entities, events, or background information mentioned in previous conversations. If it is determined to be highly dependent and lacks sufficient contextual information (i.e., the output is yes), it indicates that relevant background information needs to be retrieved from an external knowledge base to supplement the context in order to generate a coherent and accurate answer.

[0052] Step S1132: Based on the preset importance weights, the binarized judgment results of the multiple evaluation dimensions are weighted and summed to obtain the information completeness score.

[0053] After obtaining the binarized judgment results from multiple sub-classifiers, these scattered, qualitative judgments are combined into a unified, quantitative information completeness score. The information completeness score ultimately reflects the degree to which the user's overall query requires supplementary external knowledge.

[0054] To achieve this goal, a weighted summation fusion strategy is adopted. Specifically, a pre-defined set of importance weights is obtained, containing five weight values ​​corresponding to the five evaluation dimensions: privacy, sensitivity, real-time performance, availability, and contextual relevance. Each importance weight is a real number greater than or equal to zero, and its magnitude represents the relative importance of that dimension in the overall information completeness assessment. For example, in a certain application scenario, real-time performance and privacy are assigned higher weights, while availability is assigned lower weights.

[0055] In a preferred implementation, the determination of weights integrates data-driven approaches with expert knowledge. On the one hand, mathematical tools such as Shapley analysis can be used to quantify the marginal contribution of each dimension to the correctness of the final retrieval decision based on historical data. On the other hand, the Analytic Hierarchy Process (AHP) can be combined to integrate the empirical judgments of domain experts, construct a judgment matrix, and thus calculate the importance weight of each dimension. Finally, the sum of the importance weights of the five dimensions is normalized to 1.

[0056] When calculating the information completeness score, the binary judgment result of each dimension (e.g., yes or 1 is mapped to the value 1, no or 0 is mapped to the value 0) is multiplied by its corresponding preset importance weight, and then the weighted results of all five dimensions are added together to obtain the information completeness score.

[0057] Therefore, the information completeness score is a continuous numerical value, theoretically ranging from 0 to 1. The information completeness score indicates the number of key dimensions on which a query is judged to require external knowledge support, and the higher the weight of these dimensions. In other words, the lower the overall information completeness of the query, the higher the dependence on external knowledge bases.

[0058] Step S114: Input the query character sequence into a pre-trained third classifier to determine the reasoning difficulty of the user query and obtain the complexity score. The complexity level includes basic, advanced and expert levels.

[0059] Please see Figure 3 The query character sequence is input into a third classifier. The third classifier is a lightweight classification model used to evaluate query complexity. Its function is to classify queries into different complexity levels and output corresponding complexity scores. The complexity score quantifies the reasoning difficulty of the query, including the required length of the reasoning chain, the breadth of information integration, and the depth of analysis.

[0060] The third classifier is built upon a pre-trained language model and fine-tuned using complexity-annotated data. It first performs deep semantic encoding on the query character sequence to extract the global semantic vector of the user query. Then, it outputs the probability distribution of the user query belonging to each complexity level through a text classification model. The complexity levels are categorized as basic, intermediate, and expert: basic queries are simple, direct questions requiring few reasoning steps and with clearly defined information needs; intermediate queries involve multi-step reasoning, partial information integration, or cross-domain knowledge association; and expert queries involve deep logical analysis, complex calculations, or highly specialized domain knowledge, requiring long reasoning chains and extensive knowledge support.

[0061] In an optional implementation, step S114 specifically includes steps S1141 to S1143: Step S1141: Input the query character sequence into the text classification model in the third classifier, classify the user query according to the complexity level, and obtain the probability distribution of the user query belonging to each complexity level; wherein, the text classification model is obtained by fine-tuning the pre-trained language model based on a third labeled dataset containing multiple historical user queries and the corresponding complexity level label of each historical user query.

[0062] The query character sequence is input into the third classifier. At its core is a classification model specifically designed for text complexity evaluation. Similarly, the third classifier is based on a lightweight pre-trained language model architecture and has been fine-tuned specifically for the complexity classification task.

[0063] The text classification model is trained using a pre-constructed third-labeled dataset. This dataset contains a large number of historical user query samples covering a wide range of domains. Each historical user query is assigned a true complexity level label by domain experts or through a reliable automated labeling process. These complexity levels are predefined in this invention as three levels: basic, advanced, and expert, corresponding to different levels of the reasoning chain length, information integration breadth, and logical analysis depth required to solve the query problem.

[0064] In one alternative implementation, the text classification model is constructed according to the following steps: Step S21: Use a pre-trained language model to label multiple historical user queries to obtain the complexity level label corresponding to each historical user query.

[0065] First, a large number of historical user query samples were collected, covering diverse domains, language styles, and query intents. Then, a combination of human expert annotation and automated annotation was used to assign complexity level labels to each query. The complexity levels were divided into basic, intermediate, and expert levels.

[0066] Step S22: Construct the third labeled dataset based on the multiple historical user queries and the complexity level label corresponding to each historical user query.

[0067] The labeled historical user queries underwent data cleaning and preprocessing, including removing duplicate queries, correcting spelling errors, and standardizing text formatting to improve data quality. The cleaned data was then divided into training, validation, and test sets. The training set was used for parameter learning of the text classification model, the validation set for hyperparameter tuning and training monitoring, and the test set for final model performance evaluation. During the partitioning, it was ensured that the complexity levels were evenly distributed across the dataset to avoid model bias due to class imbalance. The final constructed third labeled dataset is a structured, multi-class labeled corpus.

[0068] Step S23: In the first training phase, the pre-trained language model is trained based on the third labeled dataset using a category-weighted cross-entropy loss function and a first learning rate.

[0069] To address the potential class imbalance issue in historical user query sample data and accelerate the initial convergence of the text classification model, a pre-trained language model (such as RoBERTa) is first loaded as the infrastructure, and its output layer is adjusted to a multi-class classifier to adapt to the complexity level classification task. In the first stage (initial stage) of training, a class-weighted cross-entropy loss function is used as the optimization objective. This loss function mitigates training bias caused by uneven sample sizes across classes by assigning different weights to each complexity class. Specifically, classes with fewer samples are given higher weights, forcing the model to focus more on learning these classes during training, thus improving the text classification model's ability to identify minority classes. Simultaneously, a relatively high initial learning rate (i.e., the first learning rate) is set in the first stage to promote rapid updates of the model parameters in the early stages of training, quickly approaching the optimal solution region. This is achieved by combining class-weighted loss with a high learning rate.

[0070] Step S24: In the second training phase, the pre-trained language model is trained based on the third labeled dataset using the standard cross-entropy loss function and the second learning rate. The first training phase is earlier than the second training phase, and the first learning rate is higher than the second learning rate.

[0071] In the second training phase, the focus shifts to fine-tuning the model parameters and further improving the generalization ability of the text classification model. The second training phase uses the standard cross-entropy loss function, meaning that no additional weights are applied to each class; instead, the emphasis is placed on the discriminative ability of the text classification model under a balanced data distribution. This shift is based on the fact that the first training phase has already preliminarily balanced the text classification model's attention to each class. Using the standard loss function at this point helps the model further optimize the decision boundary and avoids the risk of overfitting due to continuous weighting. Simultaneously, the learning rate is adjusted to a lower value (i.e., the second learning rate) to achieve more stable and refined parameter updates. The lower learning rate allows the text classification model to search for the optimal solution with smaller step sizes, effectively preventing parameter oscillations or deviations from the optimal region, thereby improving the model's performance and robustness on the validation set. In other words, this embodiment first uses a higher learning rate and class-weighted loss for rapid convergence, and then uses a lower learning rate and standard loss for fine-tuning, enhancing the model's deep learning ability for complex features and improving its adaptability under different data distributions.

[0072] Step S25: Under the condition of meeting the preset iteration conditions, the text classification model is obtained.

[0073] Preset iteration conditions are typically based on the model's performance on the validation set or the number of training steps. For example, an early stopping mechanism can be set to automatically terminate training when the accuracy or F1 score on the validation set no longer improves over several consecutive training epochs, thus preventing overfitting. The resulting text classification model is a lightweight and efficient complexity classifier capable of quickly and accurately determining the complexity level of an input user query, outputting a probability distribution indicating whether it belongs to the basic, advanced, or expert level.

[0074] The reasoning process of the third classifier can be similar to or equivalent to that of the first classifier, and will not be detailed here. The third classifier ultimately obtains the probabilities that a user query belongs to the basic, advanced, and expert levels, that is, the probability distribution of the user query belonging to each complexity level. The sum of all probability values ​​is 1.

[0075] Step S1142: Determine the complexity level to which the user query belongs based on the probability distribution of the user query belonging to each complexity level.

[0076] This determination is made using the maximum probability decision criterion. Specifically, the probability values ​​corresponding to the basic, intermediate, and expert levels in the probability distribution obtained in step S1141 are compared. The complexity level corresponding to the probability value with the largest value is selected as the final level to which the user's query belongs.

[0077] For example, if a user query is judged by the model to have a 10% probability of being at the basic level, a 15% probability of being at the intermediate level, and a 75% probability of being at the expert level, then the complexity level of the user query will be determined to be expert level.

[0078] Step S1143: According to the preset complexity rating mapping rule, the complexity level to which the user query belongs is mapped to a numerical rating to obtain the complexity rating; wherein, the complexity rating mapping rule maps the basic level, intermediate level and expert level to incremental numerical values ​​respectively.

[0079] The complexity level of a user's query is converted according to a preset complexity rating mapping rule. The core principle is that the rating should be positively correlated with the actual complexity of the query, that is, the higher the complexity level, the higher the complexity rating.

[0080] For example, in a specific implementation, a basic query can be mapped to 1 point, an advanced query to 2 points, and an expert query to 3 points. Through this mapping process, a specific numerical complexity score is ultimately obtained.

[0081] Step S12: The scores of the multiple preset dimensions are merged into a search trigger probability.

[0082] Multiple feature scores from different semantic dimensions—task type score, information completeness score, and complexity score—are merged into a unified value with clear probabilistic meaning: the retrieval trigger probability. This probability quantifies the tendency to initiate a computationally expensive retrieval enhancement generation process for the current user query. This fusion process is not a simple numerical addition, but rather achieved through a parameterized decision function. It comprehensively considers the inherent meaning of each dimension's score and its relative importance in the final routing decision, thereby ensuring the accuracy of the decision.

[0083] In one optional implementation, step S12 specifically includes steps S121 to S123: Step S121: Normalize the task type score and the complexity score to eliminate the dimensional differences between the task type score and the complexity score.

[0084] Please see Figure 3First, the task type score and complexity score are preprocessed, i.e., normalized. This is because different feature scores may have different numerical ranges and dimensions in the original design. For example, in one specific implementation of this invention, the task type score and complexity score are discrete integers ranging from 1 to 3, corresponding to different types and levels; while the information completeness score may be in a continuous interval of 0 to 1 due to its calculation method (such as the weighted sum of multiple binarized results).

[0085] If these scores with different dimensions and ranges are directly merged, the dimensions with larger numerical ranges (such as 1 to 3) will naturally dominate in the subsequent weighted combination, potentially masking the information contained in the dimensions with smaller numerical ranges (such as 0 to 1), leading to decision bias. The purpose of normalization is precisely to eliminate this unfairness caused by different dimensions and value ranges, mapping all scores to the same dimensionless numerical range, between 0 and 1, so that the scores of each dimension can participate in the subsequent fusion calculation on a fair basis.

[0086] Specifically, for task type and complexity scores, a linear transformation method is used for normalization. By subtracting the minimum possible score and then dividing by the full range of scores (i.e., the difference between the maximum and minimum values), the original scores are linearly and proportionally scaled to the range of 0 to 1. After this processing, scores that originally ranged from 1 to 3 are transformed to the range of 0 to 1. For information completeness scores that were already in the 0 to 1 range, no additional normalization operation is required.

[0087] Step S122: Perform a weighted linear combination of the feature scores of the user query on multiple preset dimensions to obtain the decision value of the user query.

[0088] Weighted linear combination is a multi-feature fusion strategy. Specifically, it involves multiplying the score of each dimension by its corresponding preset weight coefficient, then summing the weighted results of all dimensions to obtain a comprehensive value, which serves as the decision value. Each weight coefficient is a real number greater than or equal to zero, and its magnitude clearly reflects the relative importance of that dimension in the final retrieval trigger decision. For example, if task type is considered the most direct key indicator reflecting knowledge needs, it can be assigned the highest weight; information completeness, as an auxiliary judgment, has a lower weight; and complexity level can be assigned a relatively lower weight.

[0089] The specific values ​​of the weight coefficients are optimized from labeled data through supervised learning. Using a training dataset containing a large number of historical queries and their corresponding true "should a retrieval be triggered" labels, the weight coefficients are adjusted using optimization algorithms (such as gradient descent) to minimize the difference between the model's predicted decisions and the actual decisions. During the optimization process, a non-negative constraint on the weights can be introduced to enhance the interpretability of the model's decision-making process. Ultimately, the decision value is a continuous real number that comprehensively and quantitatively represents the strength of the retrieval demand indicated by the overall characteristics of the current user query after multi-dimensional evaluation.

[0090] Step S123: The decision value of the user query is mapped to the retrieval trigger probability using the Sigmoid function.

[0091] The Sigmoid function is a special non-linear mathematical function that smoothly and monotonically compresses and maps any input value across the entire real number range to a fixed interval between zero and one. As the decision value approaches positive infinity, the output of the Sigmoid function approaches 1; as the decision value approaches negative infinity, its output approaches 0; and when the decision value is 0, the output is exactly 0.5. This makes the Sigmoid function suitable for interpreting the comprehensive decision value as a probability. Through the mapping of the Sigmoid function, the decision value is converted into a retrieval trigger probability. The retrieval trigger probability is a continuous value between 0 and 1, representing the confidence or likelihood of suggesting triggering a retrieval enhancement generation process. The closer the retrieval trigger probability is to 1, the greater the necessity of triggering a retrieval considering multi-dimensional features; the closer the probability value is to 0, the higher the probability of directly calling the internal knowledge of the large language model for generation.

[0092] Step S13: Obtain the dynamic threshold, and compare the retrieval trigger probability with the dynamic threshold to obtain a determination result on whether retrieval enhancement generation is triggered.

[0093] Based on the retrieval trigger probability, a final binary decision is made regarding whether to initiate the retrieval enhancement generation process. Unlike traditional methods that use fixed thresholds, this invention introduces a dynamic threshold mechanism. That is, the dynamic threshold is not a pre-set constant value, but rather an optimal value obtained through systematic optimization on the validation set based on actual task requirements, data distribution, and performance metrics. By directly comparing continuous retrieval trigger probabilities with the dynamic threshold, the retrieval trigger probability is transformed into an execution instruction, thereby completing the transformation from semantic understanding to action decision-making.

[0094] In an optional implementation, step S13 specifically includes steps S131 to S133: Step S131: Obtain the dynamic threshold and compare the retrieval trigger probability with the dynamic threshold.

[0095] Obtain the dynamic threshold used for decision-making. After obtaining the dynamic threshold, perform a comparison operation: directly compare the retrieval trigger probability for the current user query with the dynamic threshold to determine whether the probability value has reached or exceeded the preset decision threshold.

[0096] In one alternative implementation, the dynamic threshold is determined by the following steps: Step S31: Obtain a validation set, which contains multiple query samples, each of which is labeled with whether or not the retrieval enhancement generation should be triggered.

[0097] Figure 4 This is a schematic diagram of the dynamic threshold determination process in one embodiment of the present invention. Please refer to [link / reference]. Figure 4 A validation set is constructed or obtained, containing multiple representative query samples covering different types of user queries, including but not limited to factual queries, inferential queries, and open-ended queries, as well as queries with varying levels of complexity and information completeness. Each query sample is annotated manually or automatically, indicating whether the query should trigger the retrieval enhancement generation process. Specifically, the annotation result is a binary label indicating whether, ideally, the query should be retrieval from an external knowledge base to assist the large model in generating an answer.

[0098] Step S32: For each query sample in the validation set, calculate the decision value for that query sample.

[0099] For each query sample in the validation set, the aforementioned multidimensional feature extraction and scoring process is performed, specifically including task type scoring, information completeness scoring, and complexity scoring. Subsequently, these scores are processed according to the aforementioned steps, and a decision value corresponding to each query sample is calculated by mapping a weighted linear combination to the Sigmoid function. This decision value represents the probability that the query should trigger retrieval enhancement generation.

[0100] Step S33: Within a preset probability range, select multiple candidate thresholds through grid search.

[0101] Within a preset probability range, a series of candidate thresholds are systematically generated using a grid search method. The probability range is a closed interval from 0 to 1, covering all possible retrieval trigger probability values. Grid search is a parameter optimization method that constructs a candidate threshold set by uniformly dividing the probability interval into multiple fine-grained intervals and setting a candidate threshold at each interval. For example, the range from 0 to 1 can be divided with a step size of 0.01 to generate 100 candidate thresholds. The exhaustive search strategy ensures that the impact of different thresholds on model performance can be comprehensively examined, avoiding performance degradation or resource waste due to inappropriate threshold selection.

[0102] Step S34: For each candidate threshold, calculate an evaluation index value based on the validation set; wherein the evaluation index value includes at least one of response accuracy and F1 score.

[0103] For each candidate threshold, performance is evaluated using the aforementioned validation set. Specifically, the decision values ​​of all query samples in the validation set are binarized based on the target candidate threshold: if the decision value of any query sample is greater than or equal to the target candidate threshold, it is determined that retrieval enhancement generation is triggered; otherwise, it is determined that generation is performed directly. Based on this binarization result, it is further compared with the real labeled results in the validation set to calculate one or more evaluation metrics. Commonly used evaluation metrics include, but are not limited to, answer accuracy and F1 score. Answer accuracy reflects the ratio of the system making the correct path selection among all query samples; the F1 score comprehensively considers the precision and recall of retrieval trigger judgment and is suitable for evaluating the overall performance of the system under imbalanced positive and negative samples.

[0104] Step S35: Select the candidate threshold that optimizes the evaluation index value as the dynamic threshold.

[0105] Based on the evaluation metric values ​​corresponding to each candidate threshold, the candidate threshold that achieves the optimal evaluation metric value is selected as the final dynamic threshold. The term "optimal" can be defined according to the specific application scenario. For example, in a scenario where answer accuracy is paramount, the threshold that maximizes precision can be selected; in a scenario where precision and recall need to be balanced, the threshold that maximizes the F1 score can be selected. By comparing the performance of all candidate thresholds on the validation set, the dynamic threshold that achieves the best performance under the current task distribution and query characteristics is determined and used as the trigger boundary in dynamic routing decisions.

[0106] Step S132: If the retrieval trigger probability is greater than or equal to the dynamic threshold, determine that the determination result is to trigger retrieval enhancement generation.

[0107] Since the retrieval trigger probability reflects the comprehensive confidence in the necessity of the retrieval based on multi-dimensional feature evaluation of the query, a retrieval trigger probability greater than or equal to the dynamic threshold indicates sufficient reason to believe that introducing external knowledge bases to supplement information is necessary and beneficial for generating high-quality, highly accurate answers. Therefore, the determination result is set as triggering retrieval-enhanced generation.

[0108] Step S133: If the retrieval trigger probability is less than the dynamic threshold, determine that the determination result is not to trigger retrieval enhancement generation.

[0109] Conversely, when the retrieval trigger probability is below the dynamic threshold, it indicates that based on the feature evaluation of the current query, the information requirement is relatively simple or the information is relatively complete, and directly relying on the knowledge and capabilities within the large language model is sufficient to generate a satisfactory answer. In this case, still performing the retrieval would not only cause unnecessary computational resource overhead and response delay, but might also interfere with model generation by introducing irrelevant information, and even increase the risk of generating illusions. Therefore, the final decision is to not trigger retrieval-enhanced generation.

[0110] Step S14: If the determination result is to trigger retrieval enhancement generation, call the external knowledge base and the large language model to generate the final answer.

[0111] When the result triggers enhanced retrieval generation, to compensate for the limitations of the large language model's own knowledge, authoritative information from external knowledge bases is introduced to ensure the accuracy, timeliness, and depth of the generated answer. That is, it no longer relies solely on the large language model's built-in parameters for responses but instead invokes a retrieval engine. The retrieval engine receives the user query as input and then performs an efficient search in one or more pre-configured external knowledge bases. External knowledge bases can include structured databases, unstructured document collections, real-time updated online information sources, or domain-specific professional knowledge graphs. The retrieval engine's task is to quickly filter out the most relevant and reliable text fragments or data records from massive amounts of external information, forming a refined document set. Subsequently, the system concatenates this document set with the original user query to form an enhanced input prompt, which is then input into the large language model. Based on this enhanced input prompt, which includes internal knowledge and external evidence, the large language model performs deep semantic understanding and logical reasoning, ultimately generating a high-quality final answer that is information-rich, reliable, and highly targeted.

[0112] In an optional implementation, if the determination result triggers the generation of enhanced search, the method further includes: Step S41: Determine the path level of the retrieval enhancement generation based on the complexity score; wherein the path level includes single-hop retrieval enhancement generation path and multi-hop retrieval enhancement generation path.

[0113] Based on complexity scores, the levels of search-enhanced generation paths are divided into two categories: single-hop search-enhanced generation paths and multi-hop search-enhanced generation paths. Specifically, when the complexity score indicates that the user query is at the basic level or a relatively simple advanced level, it means that the problem can usually be solved through a one-time information retrieval and direct reasoning. In this case, choosing a single-hop search-enhanced generation path is suitable for scenarios where the answer is implicit in a single or a few closely related documents, characterized by fast processing speed and low computational cost. Conversely, when the complexity score indicates that the user query is at the expert level or a complex advanced level, it means that the problem itself has implicit multi-step reasoning requirements. The answer cannot be obtained by directly retrieving a single information source, but rather requires peeling an onion, gradually revealing intermediate answers through a series of continuous retrieval and reasoning steps, and finally synthesizing the final conclusion.

[0114] Step S42: In the case that the path level is a single-hop retrieval enhanced generation path, the document set retrieved from the external knowledge base according to the user query is input into the large language model along with the user query to generate the final answer.

[0115] In the case of a single-hop retrieval enhanced generation path, based on the user query, the retrieval tool is invoked to perform a one-time retrieval operation in an external knowledge base. The goal of this retrieval is to capture as comprehensive an information directly related to the query as possible. The retrieval tool returns a document set sorted by relevance, containing the core facts and information needed to answer the query. Subsequently, using a pre-configured prompt template, the user query and the retrieved document set are integrated and combined to form a rich and contextually clear enhanced input prompt. This enhanced input prompt is then fed into the large language model. Based on this, the large language model performs a one-time understanding and generation process, comprehensively utilizing its own language capabilities and externally provided evidence to directly produce the final answer, minimizing intermediate steps and aiming to respond to user queries with clear information needs and simple reasoning steps with minimal latency.

[0116] Step S43: When the path level is a multi-hop retrieval enhanced generation path, perform multiple rounds of retrieval from the external knowledge base based on the user query to obtain the retrieval results for each round.

[0117] For user queries deemed to require multi-hop retrieval to enhance the generated path, an iterative, multi-stage processing flow is initiated. Instead of retrieving all information at once, a controlled, cyclical process is launched. In the initial round, the retrieval engine uses the original user query as the retrieval instruction to obtain the first round of search results from an external knowledge base. These results may contain partial answers to the question or reveal key entities, concepts, or sub-questions needed to solve the problem.

[0118] Step S44: Generate intermediate inference results for each round based on the retrieval results of each round.

[0119] After obtaining the search results for each round, the document set retrieved in the current round, along with the sub-problem or reasoning state to be solved (which is the user query in the first round), is input into the large language model. However, the instruction given to the large language model this time is not to directly generate the final answer, but to act as a "reasoning engine," performing logical analysis, information extraction, hypothesis generation, or sub-problem solving based on the currently available information. The output of the large language model is the intermediate reasoning result. The intermediate reasoning result can be a preliminary judgment, an identified key entity, a newly decomposed sub-problem, or a comprehensive summary of the existing information. The intermediate result will become the new starting point or key input for the next round of the search cycle, guiding the search engine to look for information at a deeper level or in a more relevant direction.

[0120] Step S45: Input the intermediate inference results of each round together with the user query into the large language model to generate the final answer.

[0121] Upon completion of the preset iteration rounds, or when the system determines that the intermediate reasoning results are sufficient to support the generation of the final answer, all key information generated throughout the multi-round retrieval and reasoning process is summarized. Specifically, this includes: the original user query, the core document fragments retrieved in each round, and the intermediate reasoning results generated in each round, organized into a structured, context-rich, and historically significant comprehensive hint. This comprehensive hint demonstrates the entire problem-solving process and the chain of information evolution. Finally, the comprehensive hint is input into the large language model. Based on this, the large language model performs global understanding and deep reasoning, integrating all clues and intermediate conclusions to ultimately generate a comprehensive, in-depth, and evidence-complete final answer.

[0122] Step S15: If the determination result is that the retrieval enhancement generation is not triggered, the large language model is invoked to generate the final answer.

[0123] When the determination result is that retrieval enhancement generation will not be triggered, the user query is directly input into the large language model, relying on its powerful pre-trained semantic understanding and generation capabilities to generate an answer. It is suitable for handling simple user queries, common sense questions, or creative generation tasks, where the user query itself is complete or has low complexity and does not require external supplementation.

[0124] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. A task planning method for enhanced generation of large model retrieval, characterized in that, include: User queries are input in parallel to a first classifier, a second classifier, and a third classifier. The first, second, and third classifiers respectively perform intent recognition on the user query, determining feature scores for the user query across multiple preset dimensions. These feature scores include at least a task type score, an information completeness score, and a complexity score. The task type score indicates the degree of dependence of the user query on external knowledge, the information completeness score indicates the degree of supplementation of external knowledge required for the user query, and the complexity score indicates the reasoning difficulty of the user query. The first, second, and third classifiers are pre-trained using a labeled dataset to fine-tune a pre-trained language model. The labeled dataset is constructed based on task type, information completeness, and complexity. The scores from the multiple preset dimensions are merged into a search trigger probability; A dynamic threshold is obtained, and the retrieval trigger probability is compared with the dynamic threshold to obtain a determination result on whether retrieval enhancement generation is triggered; If the determination result indicates that the search enhancement generation has been triggered, an external knowledge base and a large language model are invoked to generate the final answer. If the determination result is that the retrieval enhancement generation is not triggered, the large language model is invoked to generate the final answer.

2. The method according to claim 1, characterized in that, The user query is input in parallel to a first classifier, a second classifier, and a third classifier. The first classifier, the second classifier, and the third classifier, respectively, perform intent recognition on the user query to determine the feature scores of the user query across multiple preset dimensions, including: The user query is split into multiple words to obtain a query character sequence; The query character sequence is input into a pre-trained first classifier to determine the degree of dependence of the user query on external knowledge, and the task type score is obtained. The task type includes factual query, inference query and open query. The query character sequence is input into a pre-trained second classifier. Based on privacy, sensitivity, real-time performance, availability, and contextual relevance, the degree of supplementation of external knowledge required by the user query is evaluated to obtain the information completeness score. The query character sequence is input into a pre-trained third classifier to determine the reasoning difficulty of the user query and obtain the complexity score. The complexity level includes basic, intermediate and expert levels.

3. The method according to claim 2, characterized in that, The query character sequence is input into a pre-trained first classifier to determine the degree of dependence of the user query on external knowledge, thereby obtaining a task type score, including: The query character sequence is input into the first classifier, and each query character in the query character sequence is mapped to a corresponding low-dimensional vector through a character embedding lookup table to obtain a character vector sequence. The character vector sequence is processed by a multi-layer self-attention encoder to obtain a latent vector sequence; The implicit vector sequence is pooled to obtain a query vector representation that characterizes the overall semantics of the user query; The query vector representation is input into a multi-class classifier to obtain the probability distribution of the user query belonging to each task type; the multi-class classifier is obtained by fine-tuning a pre-trained language model based on a first labeled dataset containing multiple historical user queries and the corresponding task type labels for each historical user query; Based on the probability distribution of the user query belonging to each task type, the task type to which the user query belongs is determined; Based on a preset task type rating mapping rule, the task type to which the user query belongs is mapped to a numerical rating to obtain the task type rating; wherein, the task type rating mapping rule maps factual queries, inferential queries and open queries to different numerical values, with factual queries having the highest numerical value and open queries having the lowest numerical value.

4. The method according to claim 2, characterized in that, The query character sequence is input into a pre-trained second classifier. Based on privacy, sensitivity, real-time performance, availability, and contextual relevance, the classifier evaluates the degree to which the external knowledge required for the user query is supplemented, resulting in an information completeness score, including: The query character sequence is input in parallel to multiple sub-classifiers in the second classifier to obtain binary judgment results for multiple evaluation dimensions. These multiple evaluation dimensions include privacy, sensitivity, real-time performance, availability, and contextual relevance. Each sub-classifier is obtained by fine-tuning a pre-trained language model based on a second labeled dataset containing multiple historical user queries and their binary labels on the corresponding evaluation dimension. Each sub-classifier corresponds to one of the evaluation dimensions: privacy, sensitivity, real-time performance, availability, and contextual relevance. Based on preset importance weights, the binarized judgment results of the multiple evaluation dimensions are weighted and summed to obtain the information completeness score.

5. The method according to claim 2, characterized in that, The query character sequence is input into a pre-trained third classifier to determine the reasoning difficulty of the user query, and the feature score of the user query on the complexity dimension is obtained, including: The query character sequence is input into the text classification model in the third classifier, and the user query is classified according to the complexity level to obtain the probability distribution of the user query belonging to each complexity level; wherein, the text classification model is obtained by fine-tuning the pre-trained language model based on a third labeled dataset containing multiple historical user queries and the corresponding complexity level label of each historical user query. The complexity level of the user query is determined based on the probability distribution of the user query belonging to each complexity level. According to the preset complexity rating mapping rule, the complexity level of the user query is mapped to a numerical rating to obtain the complexity rating; wherein, the complexity rating mapping rule maps the basic level, intermediate level and expert level to incremental numerical values ​​respectively.

6. The method according to claim 5, characterized in that, The text classification model is constructed according to the following steps: Multiple historical user queries were labeled using a pre-trained language model to obtain the complexity level label for each historical user query; Based on the multiple historical user queries and the complexity level label corresponding to each historical user query, the third labeled dataset is constructed; In the first training phase, the pre-trained language model is trained based on the third labeled dataset using a category-weighted cross-entropy loss function and a first learning rate. In the second training phase, the pre-trained language model is trained based on the third labeled dataset using the standard cross-entropy loss function and the second learning rate. The first training phase is earlier than the second training phase, and the first learning rate is higher than the second learning rate. The text classification model is obtained when the preset iteration conditions are met.

7. The method according to any one of claims 1-5, characterized in that, The scores from the multiple preset dimensions are merged into a retrieval trigger probability, including: The task type score and the complexity score are normalized to eliminate the difference in dimensions between them. The decision value of the user query is obtained by weighted linear combination of the feature scores of the user query on multiple preset dimensions. The sigmoid function is used to map the decision value of the user query to the retrieval trigger probability.

8. The method according to claim 7, characterized in that, The dynamic threshold is determined by the following steps: Obtain a validation set, which contains multiple query samples, each of which is labeled with whether or not the retrieval enhancement generation should be triggered; For each query sample in the validation set, calculate the decision value for that query sample; Within a preset probability range, multiple candidate thresholds are selected through grid search; For each candidate threshold, an evaluation metric value is calculated based on the validation set; wherein the evaluation metric value includes at least one of response accuracy and F1 score; The candidate threshold that optimizes the evaluation index value is selected as the dynamic threshold.

9. The method according to claim 1, characterized in that, In the case where the determination result is to trigger the generation of search enhancement, the following is also included: Based on the complexity score, the path level of the retrieval enhancement generation is determined; wherein, the path level includes single-hop retrieval enhancement generation path and multi-hop retrieval enhancement generation path; When the path level is a single-hop retrieval enhanced generation path, the document set retrieved from the external knowledge base according to the user query is input into the large language model along with the user query to generate the final answer; When the path level is a multi-hop retrieval enhanced generation path, multiple rounds of retrieval are performed from the external knowledge base based on the user query to obtain the retrieval results for each round. Based on the retrieval results of each round, generate intermediate inference results for each round; The intermediate inference results from each round, along with the user query, are input into the large language model to generate the final answer.

10. The method according to claim 1, characterized in that, Obtain a dynamic threshold, and compare the retrieval trigger probability with the dynamic threshold to obtain a determination result on whether retrieval enhancement generation is triggered, including: Obtain the dynamic threshold and compare the retrieval trigger probability with the dynamic threshold; If the retrieval trigger probability is greater than or equal to the dynamic threshold, the determination result is determined to trigger retrieval enhancement generation; If the probability of triggering the search is less than the dynamic threshold, the determination result is that the search enhancement generation is not triggered.

Images