Multi-round dialogue complexity assessment method and device, electronic equipment and computer readable storage medium

Abstract

The embodiment of the invention provides a multi-round dialogue complexity assessment method and device, electronic equipment and a computer readable storage medium, and relates to the technical field of artificial intelligence. According to the method, by fusing the trend of the session state, the semantic turning of the abstract and the correlation of the historical dialogues, the evaluation accuracy of the real complexity of the request in multiple rounds of dialogues is improved, and misjudgment caused by only depending on the original text length or keywords is avoided; and meanwhile, the context length transmitted to the large language model is effectively compressed by using the abstract containing intention transformation and the related dialogue, so that the Token cost is directly saved, the calculation overhead brought by redundant context transmission is reduced, and a more efficient and more intelligent model routing decision is supported. Accurate context compression and routing decision reduce the average duration of request processing, and the user experience can be improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to a method, apparatus, electronic device, and computer-readable storage medium for evaluating the complexity of multi-turn dialogues. Background Technology

[0002] In Artificial Intelligence (AI) application services, the AI ​​gateway acts as a middleware layer between client applications and one or more backend AI model services, responsible for request routing, load balancing, cost control, security management, and context orchestration. As a unified request entry point, the AI ​​gateway forwards user requests to different backend Large Language Models (LLMs) to achieve centralized management and scheduling of computing resources. To balance service cost and response performance, the system typically deploys multiple types of models, including high-cost, high-performance large models, medium-performance and cost-balanced intermediate models, and low-cost, fast-responding lightweight models. Intelligently routing requests to the most suitable model based on the actual needs of user requests is the core technical challenge for improving overall service efficiency.

[0003] With the widespread application of multi-turn dialogue scenarios, the complexity of a request is no longer solely determined by the text content input in the current turn. Existing technologies primarily employ two methods to handle multi-turn dialogue context: one is complete history concatenation, which preserves the complete interaction record but leads to a linear increase in token consumption with each dialogue turn, significantly increasing inference costs and prolonging the Time To First Token (TTFT) delay, making it difficult to sustain in long dialogue scenarios; the other is general summary compression, which uses a generic text summarization model to compress the historical dialogue into a short summary before each request turn and merge it with the current question before submitting it to the target model. However, such general models often prioritize high compression rates while neglecting key contextual details, easily losing important information affecting complexity assessment, thus leading to subsequent evaluation biases. Therefore, a technical solution capable of comprehensively evaluating the complexity of multi-turn dialogues is urgently needed. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a method, apparatus, electronic device and computer-readable storage medium for evaluating the complexity of multi-turn dialogues, which can improve the accuracy of complexity scoring and make routing decisions more efficient and intelligent.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows: In a first aspect, the present invention provides a method for evaluating the complexity of multi-turn dialogues, the method comprising: Receive the current round of user requests and update the previous round of session status information based on the current round of user requests; each round of session status information is used to describe the overall trend and dynamics of the completed dialogues in the session; The current round summary is obtained by updating the current round summary based on the current round user request and the previous round summary using a pre-trained dynamic summary model; the current round summary is used to record the intent change from the previous round user request to the current round user request. Relevance matching is performed on each completed dialogue and the current round of user requests to obtain relevant dialogues; A pre-trained judging model is used to evaluate the complexity of the current user request, the updated previous session state information, the current session summary, and the related dialogues to obtain a complexity score; the complexity score is used to match the large language model that processes the current user request.

[0006] In an optional implementation, the step of updating the current round of summarization using a pre-trained dynamic summarization model based on the current round of user requests and the previous round of summarization to obtain the current round of summarization includes: The current round of user requests and the previous round of summaries are fused using the dynamic summarization model to obtain summary information; If the current user request introduces a new topic, an intent shift identifier is generated, and the intent shift identifier and the summary information constitute the summary of the current round; If the current round of user requests does not introduce a new topic, the summary information will be determined as the summary for this round.

[0007] In an optional implementation, the step of performing relevance matching based on each completed dialogue and the current round of user requests to obtain relevant dialogues includes: Obtain the completed feature vector corresponding to each completed dialogue; An attention weighter is used to analyze the correlation between each completed feature vector and the current round of user requests to obtain the relevant weights corresponding to each completed feature vector. According to the relevant weights in descending order, a preset number of completed dialogues are obtained as the relevant dialogues.

[0008] In an optional implementation, the step of using an attention weighter to analyze the correlation between each completed feature vector and the current round of user requests to obtain the relevant weights corresponding to each completed feature vector includes: Generate the feature vector for this round based on the user requests in this round; Calculate the similarity between each completed feature vector and the current round feature vector to obtain a similarity score for each completed feature vector; The similarity scores are normalized to obtain the relevant weights for each completed feature vector.

[0009] In an optional implementation, after the method uses a pre-trained judging model to evaluate the complexity based on the current user request, the updated previous session state information, the current session summary, and the relevant dialogue to obtain a complexity score, the method further includes: The initial model level is determined based on the complexity score, low complexity threshold, and high complexity threshold. If the initial model level is the same as the model level used to process the previous round of user requests, the model used to process the previous round of user requests will be determined as the target model for this round. If the initial model level is different from the model level used to process the previous round of user requests, then the count of the initial model level is incremented, and the target model for the current round is determined based on the count of the initial model level. The target model for this round is used to process the user request and the summary for this round to obtain the model response corresponding to the user request for this round.

[0010] In an optional implementation, determining the target model for this round based on the count of the initial model level includes: If the count of the initial model level exceeds the count threshold, the model corresponding to the initial model level will be determined as the target model for this round. If the count of the initial model level does not exceed the count threshold, the model that processed the user request in the previous round will be determined as the target model for this round.

[0011] In an optional implementation, after processing the current round target model according to the current round user request and the current round summary to obtain the model response corresponding to the current round user request, the method further includes: The current session state information is obtained by updating the previous session state information based on the current user request, the complexity score, and the model response.

[0012] Secondly, the present invention provides a multi-turn dialogue complexity evaluation device, the device comprising: The processing module is used to receive the current round of user requests and update the previous round of session status information based on the current round of user requests; the session status information of each round is used to describe the overall trend and dynamics of the completed dialogues in the session; The processing module is also used to update the current round summary based on the current round user request and the previous round summary using a pre-trained dynamic summary model to obtain the current round summary; the current round summary is used to record the intention change from the previous round user request to the current round user request; The processing module is also used to perform relevance matching based on each completed dialogue and the current round of user requests to obtain relevant dialogues; The evaluation module is used to evaluate the complexity of the current user request, the updated previous session state information, the current session summary, and the related dialogue using a pre-trained judge model, and obtain a complexity score; the complexity score is used to match the large language model that processes the current user request.

[0013] Thirdly, the present invention provides an electronic device including a processor and a memory, the memory storing a computer program executable by the processor, the processor executing the computer program to implement the multi-turn dialogue complexity evaluation method described in any of the foregoing embodiments.

[0014] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multi-turn dialogue complexity evaluation method as described in any of the foregoing embodiments.

[0015] Compared to existing technologies, the multi-turn dialogue complexity evaluation method, apparatus, electronic device, and computer-readable storage medium provided in this invention improve the accuracy of evaluating the true complexity of requests in multi-turn dialogues by integrating the trends of session states, the semantic shifts in summaries, and the relevance of historical dialogues. This avoids misjudgments caused by relying solely on the original text length or keywords. Simultaneously, it effectively compresses the context length passed to large language models by utilizing summaries containing intent shifts and relevant dialogues, directly saving token costs and reducing the computational overhead caused by redundant context passing. This supports more efficient and intelligent model routing decisions. Accurate context compression and routing decisions reduce the average request processing time, improving user experience.

[0016] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a multi-turn dialogue complexity evaluation method provided in an embodiment of the present invention is shown.

[0019] Figure 2This diagram illustrates another flowchart of the multi-turn dialogue complexity evaluation method provided in this embodiment of the invention.

[0020] Figure 3 A block diagram of a multi-turn dialogue complexity evaluation device provided in an embodiment of the present invention is shown.

[0021] Figure 4 A block diagram of an electronic device provided in an embodiment of the present invention is shown.

[0022] Icons: 400 - Multi-turn dialogue complexity assessment device; 401 - Processing module; 402 - Assessment module; 403 - Scheduling module; 500 - Electronic device; 510 - Memory; 520 - Processor; 530 - Communication module. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0024] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0025] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0026] The inventors discovered that existing technologies, when processing multi-turn dialogue contexts by concatenating the entire dialogue history up to the current turn, chronologically concatenate the entire dialogue history into a long text (Prompt) and then send it to a large language model. To prevent exceeding the processing window of the large language model, the oldest history may be truncated. While this method preserves the complete interaction record, it significantly increases cost and first-word return latency.

[0027] Furthermore, existing technologies employ general summary compression to process multi-turn dialogue context. Before each round of requests, a general text summarization model is used to compress the historical dialogue into a short summary, which is then merged with the current question before being sent. However, in pursuit of high compression rates, general summary compression methods are prone to losing details, premises, or subtle shifts in context in the dialogue history that are crucial for determining the complexity of the current question.

[0028] Based on this, the multi-turn dialogue complexity evaluation method, apparatus, electronic device, and computer-readable storage medium provided in this invention improve the accuracy of evaluating the true complexity of requests in multi-turn dialogues by integrating the trend of session states, the semantic transition of summaries, and the relevance of historical dialogues. This avoids misjudgments caused by relying solely on the original text length or keywords. Simultaneously, it effectively compresses the context length passed to large language models by utilizing summaries containing intent shifts and relevant dialogues, directly saving token costs and reducing the computational overhead caused by redundant context passing. This supports more efficient and intelligent model routing decisions. Accurate context compression and routing decisions reduce the average request processing time, improving user experience.

[0029] Please refer to Figure 1 , Figure 1 A flowchart illustrating a multi-turn dialogue complexity evaluation method provided by an embodiment of the present invention is shown. The method includes the following steps: Step S10: Receive the current round of user requests and update the previous round of session state information based on the current round of user requests; the session state information of each round is used to describe the overall trend and dynamics of the completed dialogues in the session.

[0030] In this embodiment of the invention, a session is a series of consecutive multi-turn human-computer interactions marked by a unique session identifier (such as session_id). Each turn of human-computer interaction is a dialogue, including user requests and model responses. It should be understood that the context data of the session can be persistently stored and queried based on the session identifier. The context data includes, but is not limited to, session state information, dynamic summaries, and completed dialogues.

[0031] Upon receiving a user request belonging to a specific session, the system immediately updates the session state information associated with the previous session based on the content of the request and its relationship with historical interactions. A typical implementation of session state information is a Structured State Vector (SSV), a low-dimensional numerical vector used to quantify the overall trend and macro-state (i.e., dynamics) of completed dialogues within a session.

[0032] The core idea of ​​SSV is to abstract the dynamic and trend features in multi-turn dialogues that are difficult to capture through plain text into a low-dimensional, rapidly computed, and updatable numerical vector. It is not a replacement for the dialogue content, but rather a quantitative descriptor of the dialogue state, like an airplane's instrument panel, making the overall situation of the conversation readily apparent. This allows subsequent decisions (such as complexity assessment) to be based on richer dimensions that go beyond single-turn text.

[0033] Step S20: Update the current round summary based on the current round user request and the previous round summary using the pre-trained dynamic summary model to obtain the current round summary; the current round summary is used to record the intent change from the previous round user request to the current round user request.

[0034] In this embodiment of the invention, the Dynamic Summary Model (DSM) is a lightweight, independently deployed sequence-to-sequence model, such as the lightweight and small T5-small or DistilBERT. The DSM receives two inputs: the dynamic summary generated in the previous round and the complete user request for the current round. The DSM is fine-tuned using special instructions to enable it to recognize key semantic changes.

[0035] During the reasoning process, the dynamic summarization model not only generates and continuously updates the condensed text summary, but more importantly, it can detect whether a shift in intent has occurred between the previous and current rounds. Therefore, the current round summary is not only a compressed representation of the content, but also a record of important turning points in the dialogue process. This step can be understood as achieving precise capture of semantic leaps in multi-turn dialogues, compensating for the shortcomings of traditional general-purpose summarization methods in preserving key information.

[0036] Step S30: Perform relevance matching based on each completed dialogue and the current round of user requests to obtain relevant dialogues.

[0037] In this embodiment of the invention, relevant dialogues are obtained by comparing the relevance of each historical dialogue (i.e., completed dialogue) in the session with the current user request. Relevant dialogues refer to completed dialogues that are highly relevant to the current user request and are used for subsequent complexity evaluation. It should be noted that this process avoids the high computational overhead of concatenating all historical dialogues as input; instead, it focuses on the most semantically relevant historical segments, effectively achieving contextual information filtering and focusing, improving evaluation efficiency, and reducing noise interference.

[0038] Step S40: Using a pre-trained judging model, the complexity is evaluated based on the current user request, the updated previous session state information, the current summary, and related dialogues to obtain a complexity score; the complexity score is used to match the large language model that processes the current user request.

[0039] In this embodiment of the invention, the current user request, the updated previous session state information, the current summary, and related dialogues are integrated into a structured input package, which serves as the input data for the judging model. This provides the judging model with a compact input package that has extremely high information density and extremely low noise, thereby achieving high-precision, real-time judgment of the true complexity of the problem with minimal computational overhead. Specifically, the updated session state information provides a macro-level trend perspective, the current summary reveals key semantic changes, related dialogues supplement locally highly relevant contexts, and the current user request itself constitutes the direct object of evaluation.

[0040] The judging model, as a specially trained decision network (such as a BERT-like model), assesses the intrinsic complexity of a request by fusing and analyzing this heterogeneous information, outputting a continuous or hierarchical complexity score. This score reflects the true processing difficulty of the request in the current session context, and can then be used to match the appropriate large language model type for handling the request. This step enables collaborative reasoning from multi-source contextual signals, significantly improving the accuracy and robustness of complexity assessment, thereby supporting more intelligent model routing strategies.

[0041] In summary, the multi-turn dialogue complexity evaluation method provided by this invention improves the accuracy of evaluating the true complexity of requests in multi-turn dialogues by integrating the trends of conversation states, the semantic shifts in summaries, and the relevance of historical dialogues. This avoids misjudgments caused by relying solely on the original text length or keywords. Simultaneously, it effectively compresses the context length passed to a large language model by utilizing summaries containing intent shifts and relevant dialogues, directly saving token costs and reducing the computational overhead caused by redundant context passing. This supports more efficient and intelligent model routing decisions. Accurate context compression and routing decisions reduce the average request processing time, improving user experience.

[0042] Alternatively, one possible implementation for generating this round summary is provided below. Figure 1 The sub-steps of step S20 may include: Step S200: Use a dynamic summary model to merge the current round of user requests and the previous round of summaries to obtain summary information.

[0043] In this embodiment of the invention, the dynamic summarization model does not generate a comprehensive general summary, but rather acts as a specialized change detector. Its core task is to identify whether the user's intent has changed. The dynamic summarization model performs sequence-level fusion processing on the current user request and the previous summary. By continuating the historical summary (i.e., the previous summary) and integrating it with the current user request, it generates an intermediate text summary that reflects the latest session state, i.e., the summary information.

[0044] This process relies on the dynamic summarization model's ability to model contextual coherence, ensuring that the newly generated content retains key historical context while also covering new information added in this round. This achieves a rolling update mechanism for dialogue summaries, providing basic textual support for whether to introduce additional tags in the future.

[0045] It's important to note that when training the dynamic summarizing model, special instructions are used to fine-tune the model before training. The training process doesn't involve generating summaries from articles in the traditional way; instead, it generates updated summaries from the previous round of summaries and the latest round of requests. Specifically, the special instructions can be designed to request the generation of a new summary based on the old summary and the new dialogue. If the new dialogue introduces entirely new topics, instructions, or key technical terms, please mark the beginning of the new summary with '[Intent Shift]'.

[0046] Step S210: If the user requests a new topic in this round, generate an intent shift identifier, and the intent shift identifier and summary information constitute the summary of this round.

[0047] In this embodiment of the invention, while generating summary information, the dynamic summarization model determines, based on its internal logic, whether there is a significant change at the topic level in the current round, such as a user shifting from a daily question to a task or a professional field like mathematical reasoning. When a significant change in topic is detected (i.e., the introduction of a new topic), the dynamic summarization model generates an intent shift identifier. This intent shift identifier is added before the summary information in the form of display text (e.g., "[Intent Shift]"), together forming the final output summary for this round.

[0048] It should be understood that the existence of intent shift markers enables the judge model and the large language model to quickly identify significant semantic shifts occurring during the dialogue process without re-parsening the entire history. This can be understood as the mechanism enhancing the information density and readability of the summary, making it not only a content compression function but also a carrier of event-driven state signals.

[0049] As one possible implementation, for example, in the previous session, the user asked the large language model to briefly introduce the content of game A. Then, in the next session, the user asked the large language model to write a program for game A. The generated dynamic summary would then be as follows: [Intent Shift] From introducing game A to programming and implementing game A.

[0050] Step S220: If no new topic is introduced in this round of user requests, the summary information is determined as the summary for this round.

[0051] In this embodiment of the invention, when the dynamic summary model determines that the current user request has not caused a change in the topic level, that is, the dialogue is still in the original topic or task flow, the generation of intent change identifier is not triggered. Instead, the summary information is directly used as the summary for this round, which avoids introducing redundant markers in scenarios without significant changes and maintains the simplicity and consistency of summary expression.

[0052] As can be seen, by conditionally introducing intent shift identifiers, the embodiments of the present invention enable the generated summary to not only have content compression capabilities, but also retain semantic transition information that has a key impact on complexity assessment, thereby improving the ability of the subsequent judge model to perceive dynamic changes in the context and enhancing the information representation efficiency of the summary without significantly increasing the summary length.

[0053] Optionally, the following provides a possible implementation for obtaining the relevant dialogue. Figure 1 The sub-steps of step S30 may include: Step S300: Obtain the completed feature vector corresponding to each completed dialogue.

[0054] In this embodiment of the invention, when evaluating the complexity of each newly received user request, a pre-trained lightweight sentence embedding model (e.g., a miniature version of Sentence-BERT) is used to convert the completed dialogue of the previous round (including the user request of the previous round and the model response corresponding to the user request of the previous round) into a low-dimensional vector, thereby obtaining the completed feature vector corresponding to the completed dialogue of the previous round, and caching the completed feature vector of the previous round so that it can be directly obtained when evaluating the complexity in the next round.

[0055] For example, when evaluating the complexity of the second round of user requests, a sentence embedding model is used to convert the first round of user requests and model responses into low-dimensional vectors as completed dialogues, obtaining the first completed feature vector, which is then stored in the cache. This process continues, storing the completed feature vectors corresponding to each subsequent completed dialogue in the cache.

[0056] Step S310: Use an attention weighter to analyze the correlation between each completed feature vector and the user request in this round, and obtain the relevant weights corresponding to each completed feature vector.

[0057] In this embodiment of the invention, an attention weighter is used to analyze the correlation between each completed feature vector and the current user request, obtaining the correlation weight corresponding to each completed feature vector. The larger the correlation weight, the more relevant the completed dialogue is to the current user request. It should be noted that correlation analysis can also be calculated using lighter TF-IDF similarity or Jaccard similarity, which is not limited in this invention.

[0058] Step S320: Obtain a preset number of completed dialogues as relevant dialogues in descending order of their relevant weights.

[0059] In this embodiment of the invention, after obtaining the relevant weights corresponding to all completed dialogues, they are sorted in descending order according to their weight values, and the top-K completed dialogues are selected, where K is a preset number, such as 3 or 5. These selected dialogues constitute "relevant dialogues" and serve as one of the key inputs for the subsequent complexity evaluation by the judging model.

[0060] As can be seen, the embodiments of the present invention effectively identify and filter out the part of the historical dialogue most relevant to the current request through a vectorized semantic matching mechanism. While reducing redundant information input, it retains key context support, improves the accuracy and computational efficiency of subsequent complexity assessment, and avoids the high cost and inefficiency caused by splicing the entire history.

[0061] Optionally, regarding how to obtain the relevant dialogue, the following is a possible implementation method. The sub-steps of step S310 may include: Step S310-1: Generate the feature vector for this round based on the user requests in this round.

[0062] In this embodiment of the invention, the attention weighter includes a lightweight sentence embedding model and a similarity calculation unit. The sentence embedding model is used to convert the current user request into a low-dimensional vector, resulting in the current round feature vector. This current round feature vector represents the core semantic content of the current request in the semantic space and resides in the same vector space as the completed feature vectors corresponding to historical dialogues, thus providing a basis for subsequent cross-round semantic comparisons.

[0063] It should be noted that sentence embedding models, once trained, can map natural language text into low-dimensional numerical vectors of fixed dimensions. Sentence embedding models have low inference overhead and are suitable for high-concurrency, low-latency online service scenarios.

[0064] Step S310-2: Calculate the similarity between each completed feature vector and the feature vector of this round, and obtain the similarity score of each completed feature vector.

[0065] In this embodiment of the invention, the current feature vector is compared pairwise with each completed feature vector in turn. Cosine similarity is used as the metric to calculate the cosine similarity between the two in the vector space. This is a vector operation with extremely low computational cost. Cosine similarity reflects the semantic closeness between the completed dialogue and the current user request. A higher cosine similarity indicates greater semantic similarity and stronger correlation.

[0066] Step S310-3: Normalize all similarity scores to obtain the relevant weights for each completed feature vector.

[0067] In this embodiment of the invention, a normalization function (e.g., Softmax) is used to normalize all similarity scores, converting the similarity scores into a probability distribution form with a sum of 1, i.e., the relevance weights. The relevance weights explicitly express the importance of each completed dialogue in the overall context to the user's request in that round, allowing the most valuable historical dialogues to be selected based on their relevance weights.

[0068] It should be understood that normalization not only enhances the comparability of scores between different conversations, but also provides a unified ranking basis for To-related conversation filtering, thereby supporting the maximization of information utilization within a limited context window.

[0069] As can be seen, the embodiments of the present invention achieve efficient and quantifiable evaluation of the relevance of historical dialogues through similarity calculation and normalization mechanisms in vector space. This not only preserves the ability to identify key contexts but also avoids the inference overhead of high-complexity models, thereby improving the accuracy and computational efficiency of the relevant dialogue selection process.

[0070] Research has revealed that existing AI gateway technologies typically employ simple rules for routing. This means they apply static rules based on the surface characteristics of the current user request (such as token length, whether it contains specific keywords like "code" or "summary") to determine whether to route the request to a simple or complex model. Simple rule routing ignores the overall dynamics and cumulative effects of the session, leading to inaccurate judgments due to a single evaluation dimension. For example, a seemingly simple question (such as "Why?") may actually have extremely high complexity because it inherits context from thousands of words preceding it; existing technologies struggle to accurately identify this implicit complexity.

[0071] To address the issue of inaccurate model selection caused by using simple rule-based routing, this embodiment of the invention performs complexity evaluation on the current round of user requests, the updated previous round session state information, the current round summary, and the related dialogues, and obtains a large language model for processing the current round of user requests based on the complexity score.

[0072] Optionally, for large language models that utilize complexity scoring matching to handle this round of user requests, the following is a possible implementation. Please refer to... Figure 2 ,exist Figure 1 Following step S40, the following steps may also be included: Step S50: Determine the initial model level based on the complexity score, low complexity threshold, and high complexity threshold.

[0073] In this embodiment of the invention, after the judging model outputs the complexity score for the current round of requests, the complexity score is compared with preset low and high complexity thresholds to classify processing capability levels. This process maps continuous complexity scores to discrete model levels, forming an initial model level decision. This establishes a bridge between evaluation results and resource matching, providing a clear quantitative basis for the selection of large language models.

[0074] As one possible implementation, if the complexity score is lower than the low complexity threshold, it is determined to be a simple task, corresponding to a cheap and fast lightweight model; if the complexity score is higher than the high complexity threshold, it is determined to be a complex task, corresponding to a high-performance but high-cost large model; if the complexity score is greater than or equal to the low complexity threshold and less than or equal to the high complexity threshold, it can be assigned to a medium-performance model.

[0075] Step S60: If the initial model level is the same as the model level used to process the previous round of user requests, the model used to process the previous round of user requests will be determined as the target model for this round.

[0076] In this embodiment of the invention, the model level corresponding to the target model actually used to process user requests in the previous round is obtained and compared with the initial model level determined in the current round. When the two are consistent, it indicates that the complexity level of the current user request remains stable, and there is no need to change the service resources. Therefore, the original model is directly used as the target model for this round.

[0077] This design can be understood as avoiding frequent model switching caused by minor fluctuations in scores, thereby reducing latency spikes and KV Cache reconstruction overhead caused by context reload. This mechanism enhances the stability of the routing strategy and improves the consistency of the user experience. The KV Cache is an optimization mechanism within the large language model used to cache the key and value states of previous tokens to accelerate the generation of subsequent tokens.

[0078] Step S70: If the initial model level is different from the model level used to process the previous round of user requests, then the count of the initial model level is incremented, and the target model for this round is determined based on the count of the initial model level.

[0079] In this embodiment of the invention, when it is detected that the model level requested by the user in the previous round is different from the initial model level corresponding to the user request in the current round, model switching is not performed immediately. Instead, a counter for the new model level (i.e., the initial model level) is started, and the number of consecutive occurrences of the initial model level is accumulated. This count reflects the persistence of the current complexity trend, preventing misjudgments caused by a single abnormal score. By introducing a filtering mechanism in the time dimension, it is ensured that the model switching decision is based on a stable trend, rather than on instantaneous disturbances.

[0080] Step S80: Process the target model of this round according to the user request and the summary of this round to obtain the model answer corresponding to the user request of this round.

[0081] In this embodiment of the invention, once the target model for the current round is determined, the user request for the current round and the summary generated by the dynamic summary model for the current round are used as input and passed to the target model for reasoning and generation to obtain the model answer corresponding to the user request for the current round.

[0082] It should be noted that when switching between different models, existing technologies require all the necessary context to be re-provided to the new model for processing due to the incompatibility of KV Cache, resulting in a surge in cost and latency during the switching process.

[0083] In this embodiment of the invention, the current round summary is used as a context compression representation, which retains key historical information while significantly reducing the number of tokens required for input. Especially in cross-model switching scenarios, using the current round summary as a bridge context to pass to the new model can effectively reduce the surge in cost and latency caused by re-injecting context due to KV cache incompatibility, thereby minimizing switching costs. The optimized model switching strategy avoids sudden stuttering caused by KV cache reset.

[0084] As can be seen, by introducing a joint decision-making mechanism based on threshold judgment, state maintenance and count accumulation, the embodiments of the present invention effectively balance the accuracy of model resource matching and the stability of system operation, reduce the latency and computational overhead caused by frequent model switching, and improve the robustness and economy of service scheduling in multi-turn dialogue scenarios.

[0085] Alternatively, the following is a possible implementation method for determining the target model for this round based on counting. Figure 2 The sub-step in step S70, which determines the target model for this round based on the count of the initial model level, may include: Step S700: If the count of the initial model level exceeds the count threshold, the model corresponding to the initial model level is determined as the target model for this round.

[0086] In this embodiment of the invention, after detecting that the initial model level corresponding to the current complexity score is different from the model level used in the previous round, the number of consecutive occurrences of the initial model level is incremented. When the count exceeds a preset threshold, it indicates that the complexity trend of the current user request has consistently shifted towards a certain processing capability level, possessing sufficient stability and continuity. At this point, it is determined that a model switch should be performed, and the large language model corresponding to the initial model level is officially determined as the target model for this round. It should be noted that this mechanism avoids misjudgments and frequent switching caused by abnormal requests in a single round, ensuring that resource adjustments are based on sufficient trend confirmation.

[0087] Step S710: If the count of the initial model level does not exceed the count threshold, the model that processed the user request in the previous round is determined as the target model for this round.

[0088] In this embodiment of the invention, when it is detected that the initial model level has changed but its cumulative number of times has not reached the counting threshold, it is considered that the change may only be a temporary fluctuation or a short-term disturbance, which is insufficient to support the reallocation of service resources. Therefore, the actual model switching action is not triggered, but the original model used to process the previous round of user requests is continued to be used as the target model for this round.

[0089] This design can be understood as creating a "buffer window" that only responds when new demand patterns continue to emerge, thereby effectively suppressing decision-making oscillations, achieving a balance between dynamic evaluation and stable service, and ensuring the consistency of user experience and the economy of system operation.

[0090] It should be noted that complexity scores, dynamic summaries, and session state information can serve as valuable labeled data for continuous optimization of scoring models and routing strategies, forming a data-driven adaptive closed loop.

[0091] As can be seen, by introducing a hysteresis confirmation mechanism based on a counting threshold, this embodiment of the invention enhances the stability of routing decisions while ensuring the accuracy of model matching, effectively suppresses the oscillation problem caused by instantaneous changes in complexity scores, thereby reducing unnecessary model switching overhead and improving the consistency of overall operating efficiency and service quality.

[0092] Optionally, regarding how to generate the current round session state information, the following is one possible implementation. Please refer to... Figure 2 It may also include the following steps: Step S90: Update the previous session state information based on the current user request, complexity score, and model response to obtain the current session state information.

[0093] As one possible implementation, session state information includes, but is not limited to, session statistics, complexity dynamics, information injection metrics, and interaction patterns. Session statistics include, but are not limited to, the current round number, the total number of session tokens, and the session duration. Complexity dynamics include, but are not limited to, the complexity score of the previous user request, the average complexity score of the session, and the trend of complexity changes over the last N rounds (e.g., 3 rounds) (e.g., an increase, stabilization, or decrease derived from the slope). Information injection metrics include, but are not limited to, the average density of named entities in the dialogue, whether key entities not present in the current round are injected, and whether code blocks are included in the current round. The average density is calculated by dividing the number of times named entities appear by the total length (number of words) of the user's question. Interaction patterns include, but are not limited to, the number of times the user retried or restated the question, and the average length of the user's question.

[0094] Upon receiving the current user request, obtain the previous session state information corresponding to the session identifier (i.e., the session state information updated after the previous user request generated a model response). Update the current session number by accumulating the number of dialogue rounds. Update the number of times the user has retried or restated the question based on whether the current user request is a retry or restatement. Update the average question length of the user based on the question length of the current user request.

[0095] After obtaining the complexity score and model response for this round, the system enters the subsequent update phase. Based on the user request and model response for this round, the system updates the total number of session tokens, the average density of named entities in the dialogue, the key entity binary flag, and the code block flag. The session duration is also updated based on the time the model response was received. Specifically, the key entity binary flag indicates whether a key entity not present in this round was injected, and the code block flag indicates whether a code block was included in this round.

[0096] Specifically, the token count corresponding to the model's response is added to the total number of tokens in the session. Named entities (such as names of people, places, and technical terms) are counted based on the user's request and the model's response in this round, and the average density of named entities in the dialogue is updated. It is determined whether a key entity is injected in this round based on the user's request and the model's response; if a key entity is injected, the key entity binary flag is updated to 1; otherwise, it is updated to 0. The code block flag is updated based on whether the model's response contains a code block; if it does, the code block flag is updated to 1; otherwise, it is updated to 0.

[0097] In addition, the complexity score of the previous round of requests in the previous round of session state information is updated to the complexity score corresponding to the current round of user requests. Based on the complexity score corresponding to the current round of user requests and the average complexity score of the session in the previous round of session state information, the average complexity score is recalculated and updated. Based on the complexity score corresponding to the current round of user requests and the complexity of the most recent N-1 rounds, the trend of the complexity of the most recent N rounds is re-analyzed to finally obtain the current round of session state information.

[0098] It should be noted that the session state information and summary for each round are updated iteratively. The session state information, summary, and completed feature vectors corresponding to completed dialogues are all bound to the session identifier. The latest session state information (i.e., the session state information of this round), the summary of this round, and the completed feature vectors are all stored in a cache (such as Redis) with the session identifier as the index.

[0099] It should be understood that after this request is processed, a lightweight background process will asynchronously parse the interaction and update the session state information with minimal computational overhead to prepare for the next round of dialogue. In resource-constrained environments, the session state information can be simplified to include only the most critical dimensions, such as the current dialogue round number and the average complexity score of the session; this invention does not limit this.

[0100] As can be seen, by combining the evaluation results with the actual output content, the embodiments of the present invention realize the dynamic and progressive maintenance of the session state information, so that the state representation of each round is based on the complete context evolution, which improves the accuracy and continuity of state tracking in multi-round dialogues and provides a reliable and temporally consistent input basis for the complexity evaluation of subsequent rounds.

[0101] Based on the same inventive concept, the basic principle and technical effects of the multi-turn dialogue complexity evaluation device provided in this embodiment are the same as those in the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments.

[0102] Please refer to Figure 3 , Figure 3 This is a block diagram of a multi-turn dialogue complexity evaluation device 400 provided in an embodiment of the present invention. The multi-turn dialogue complexity evaluation device 400 includes a processing module 401, an evaluation module 402, and a scheduling module 403.

[0103] The processing module 401 is used to receive the current round of user requests and update the previous round of session status information based on the current round of user requests; the session status information of each round is used to describe the overall trend and dynamics of the completed dialogues in the session.

[0104] The processing module 401 is also used to update the current round summary based on the current round user request and the previous round summary using a pre-trained dynamic summary model; the current round summary is used to record the intention change from the previous round user request to the current round user request.

[0105] The processing module 401 is also used to perform relevance matching based on each completed dialogue and the current round of user requests to obtain relevant dialogues.

[0106] Evaluation module 402 is used to evaluate the complexity of the current user request, the updated previous session state information, the current summary, and related dialogues using a pre-trained judge model, and obtain a complexity score; the complexity score is used to match the large language model that processes the current user request.

[0107] In summary, the multi-turn dialogue complexity evaluation device provided in this invention improves the accuracy of evaluating the true complexity of requests in multi-turn dialogues by integrating the trends of conversation states, the semantic shifts in summaries, and the relevance of historical dialogues. This avoids misjudgments caused by relying solely on the original text length or keywords. Simultaneously, by utilizing summaries containing intent shifts and relevant dialogues, it effectively compresses the context length passed to the large language model, directly saving token costs and reducing the computational overhead caused by redundant context transmission. This supports more efficient and intelligent model routing decisions. Accurate context compression and routing decisions reduce the average request processing time, thereby improving the user experience.

[0108] Optionally, the processing module 401 is specifically used to merge the current round of user requests and the previous round of summaries using a dynamic summary model to obtain summary information; if the current round of user requests introduces a new topic, an intent change identifier is generated, and the intent change identifier and the summary information constitute the current round of summary; if the current round of user requests does not introduce a new topic, the summary information is determined as the current round of summary.

[0109] Optionally, the processing module 401 is specifically used to obtain the completed feature vector corresponding to each completed dialogue; to analyze the correlation between each completed feature vector and the current user request using an attention weighter, and to obtain the relevant weight corresponding to each completed feature vector; and to obtain a preset number of completed dialogues as relevant dialogues in descending order of their relevant weights.

[0110] Optionally, the processing module 401 is specifically used to generate the feature vector for this round based on the user request in this round; calculate the similarity between each completed feature vector and the feature vector in this round to obtain the similarity score of each completed feature vector; and perform normalization processing based on all similarity scores to obtain the relevant weight corresponding to each completed feature vector.

[0111] Optionally, the scheduling module 403 is used to determine the initial model level based on the complexity score, the low complexity threshold, and the high complexity threshold; if the initial model level is the same as the model level used to process the previous round of user requests, the model used to process the previous round of user requests is determined as the target model for this round; if the initial model level is different from the model level used to process the previous round of user requests, the count of the initial model level is incremented, and the target model for this round is determined based on the count of the initial model level; the target model for this round is used to process the user request and the summary for this round to obtain the model answer corresponding to the user request for this round.

[0112] Optionally, the scheduling module 403 is specifically used to determine the model corresponding to the initial model level as the target model for this round if the count of the initial model level exceeds the counting threshold; and to determine the model that processed the user request in the previous round as the target model for this round if the count of the initial model level does not exceed the counting threshold.

[0113] Optionally, the processing module 401 is also used to update the updated previous session state information based on the current user request, complexity score and model answer to obtain the current session state information.

[0114] Please refer to Figure 4This is a block diagram illustrating an electronic device 500 provided in an embodiment of the present invention. The electronic device 500 includes, but is not limited to, a personal computer (PC), a personal digital assistant (PDA), a laptop computer, a tablet computer, and a server. The electronic device 500 includes a memory 510, a processor 520, and a communication module 530. The memory 510, processor 520, and communication module 530 are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0115] The memory 510 is used to store programs or data. The memory 510 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0116] The processor 520 is used to read / write data or programs stored in the memory 510 and perform corresponding functions. For example, when a computer program stored in the memory 510 is executed by the processor 520, the multi-turn dialogue complexity evaluation method disclosed in the above embodiments can be implemented.

[0117] The communication module 530 is used to establish a communication connection between the electronic device 500 and other communication terminals via a network, and to send and receive data via the network.

[0118] It should be understood that, Figure 4 The structure shown is only a schematic diagram of the electronic device 500. The electronic device 500 may also include components that are larger than those shown. Figure 4 The more or fewer components shown, or having the same Figure 4 The different configurations shown. Figure 4 The components shown can be implemented using hardware, software, or a combination thereof.

[0119] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor 520, implements the multi-turn dialogue complexity evaluation method disclosed in the above embodiments.

[0120] This invention also provides a program product that, when executed by processor 520, implements the multi-turn dialogue complexity evaluation method disclosed in the above embodiments.

[0121] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0122] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0123] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0124] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for evaluating the complexity of multi-turn dialogues, characterized in that, The method comprises: receiving a current round of user requests and updating last round of session state information based on the current round of user requests; each round of session state information is used to describe the overall trend and dynamics of completed dialogues in the session; updating the current round of summary based on the current round of user requests and the last round of summary by using a pre-trained dynamic summary model; the current round of summary is used to record the intention change from the last round of user requests to the current round of user requests; performing relevance matching on each completed dialogue and the current round of user requests to obtain relevant dialogues; performing complexity evaluation on the current round of user requests, the updated last round of session state information, the current round of summary and the relevant dialogues by using a pre-trained judge model to obtain a complexity score; the complexity score is used to match a large language model for processing the current round of user requests.

2. The method of claim 1, wherein, The method of updating the current round of summary based on the current round of user requests and the last round of summary by using a pre-trained dynamic summary model comprises: fusing the current round of user requests and the last round of summary by using the dynamic summary model to obtain summary information; if the current round of user requests introduces a new topic, generating an intention change identifier, and constructing the current round of summary from the intention change identifier and the summary information; if the current round of user requests does not introduce a new topic, determining the summary information as the current round of summary.

3. The method of claim 1, wherein, The method of performing relevance matching on each completed dialogue and the current round of user requests to obtain relevant dialogues comprises: obtaining a completed feature vector corresponding to each completed dialogue; analyzing the relevance of each completed feature vector and the current round of user requests by using an attention weighter to obtain a relevance weight corresponding to each completed feature vector; obtaining a preset number of completed dialogues as the relevant dialogues in the order from high to low of the relevance weights.

4. The method of claim 3, wherein, The method of analyzing the relevance of each completed feature vector and the current round of user requests by using an attention weighter to obtain a relevance weight corresponding to each completed feature vector comprises: generating a current round of feature vector according to the current round of user requests; calculating the similarity of each completed feature vector and the current round of feature vector respectively to obtain a similarity score of each completed feature vector; normalizing all the similarity scores to obtain a relevance weight corresponding to each completed feature vector.

5. The method of claim 1, wherein, After the method of performing complexity evaluation on the current round of user requests, the updated last round of session state information, the current round of summary and the relevant dialogues by using a pre-trained judge model to obtain a complexity score, the method further comprises: determining an initial model level according to the complexity score, a low complexity threshold and a high complexity threshold; if the initial model level is the same as the model level for processing the last round of user requests, determining a model for processing the last round of user requests as a current round of target model; if the initial model level is different from the model level for processing the last round of user requests, accumulating the count of the initial model level, and determining a current round of target model according to the count of the initial model level; The current target model is determined according to the count of the initial model level.

6. The method of claim 5, wherein, The current target model is determined according to the count of the initial model level. If the count of the initial model level exceeds a count threshold, the model corresponding to the initial model level is determined as the current target model. If the count of the initial model level does not exceed the count threshold, the model processing the previous round of user requests is determined as the current target model.

7. The method of claim 5, wherein, After the model answer corresponding to the current round of user requests is obtained by processing the current round of user requests and the current round of summary based on the current target model, the method further comprises: The updated previous round of session state information is updated based on the current round of user requests, the complexity score and the model answer to obtain current round of session state information.

8. A multi-turn dialogue complexity evaluation apparatus, characterized by, The device comprises: The processing module is configured to receive a current round of user requests and update a previous round of session state information based on the current round of user requests; each round of session state information is used to describe the overall trend and dynamics of completed dialogues in the session. The processing module is further configured to obtain a current round of summary based on the current round of user requests and the updated previous round of summary by using a pre-trained dynamic summary model; the current round of summary is used to record the intention change from the previous round of user requests to the current round of user requests. The processing module is further configured to perform relevance matching on each completed dialogue and the current round of user requests to obtain relevant dialogues. The evaluation module is configured to perform complexity evaluation on the current round of user requests, the updated previous round of session state information, the current round of summary and the relevant dialogues by using a pre-trained evaluator model to obtain a complexity score; the complexity score is used to match a large language model processing the current round of user requests.

9. An electronic device, comprising: The computer program is executed by the processor to implement the multi-round dialogue complexity evaluation method of any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the multi-round dialogue complexity evaluation method of any one of claims 1-7.

Images