Session processing method, device and computer program product based on large model switching

Abstract

The application discloses a conversation processing method, device and computer program product based on large model switching, which comprises the following steps: first, obtaining the conversation history information about the target user according to the target conversation request initiated by the target user. Then, when it is detected that the trigger condition of large model switching is met, the conversation history information about the target user is converted into conversation information meeting the input format requirements of all candidate large models. Not only the conversion quality of the converted conversation context is evaluated through multiple dimensions, realizing accurate quantification of the conversion quality and improving the information retention rate. Moreover, the capabilities of all candidate large models are detected from multiple dimensions, improving the detection accuracy and realizing dynamic optimization of the detection strategy, so as to help selecting the optimal LLM from all candidate large models more quickly and accurately for conversation with the user, thereby improving the conversation efficiency and the conversation experience of the target user.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a conversation processing method, device and computer program product based on large model switching. Background Technology

[0002] With the continuous development of artificial intelligence and information technology, the application scenarios of human-computer interaction are becoming more and more widespread. Various large language models (LLMs) are appearing in people's lives and work, providing intelligent interaction functions for various application scenarios such as knowledge question answering and multi-turn dialogue.

[0003] Currently, to handle massive, high-concurrency user requests, a multi-instance distributed deployment architecture based on LLM (Limited Management Model) is typically adopted, combined with Server-Sent Events (SSE) technology to achieve real-time streaming data push from the server to the client. However, in this multi-instance environment, each service instance runs as an independent computing unit on different physical or virtual nodes, with completely isolated memory spaces, making it impossible to directly share state data. This not only leads to difficulties in cross-instance synchronization of session state but also causes challenges in session context migration when switching between LLMs. Existing methods for evaluating session context transition quality typically rely solely on simple cosine similarity comparisons of overall semantics, neglecting considerations of local loss of key information and the degree of preservation of functional information such as parameters, thus reducing the accuracy of transition quality evaluation. Furthermore, when selecting an LLM to switch to, fixed test cases are used to probe the capabilities of each candidate LLM. This not only requires test case design to rely on expert experience and is susceptible to temporary factors but also fails to adaptively adjust the testing strategy based on LLM feedback, reducing the accuracy of probing the capabilities of each candidate LLM. Consequently, it may be impossible to switch to the optimal LLM for user interaction, thereby reducing session efficiency and user experience. Summary of the Invention

[0004] In view of the above problems, this application provides a session processing method, device, and computer program product based on large model switching. It not only evaluates the conversion quality of the transformed session context through multiple dimensions (such as global semantics, local information, entity preservation, functional constraints, and time decay), achieving precise quantification of conversion quality and strategy optimization, but also probes the ability of all candidate LLMs from multiple dimensions (such as test coverage, result stability, cross-validation, historical consistency, and adaptive weighting), realizing dynamic optimization of the probe strategy. This helps to select the optimal LLM to switch to for user conversations, thereby improving session efficiency and user experience. The specific solution is as follows: This application provides a session processing method based on large model switching, including: Obtain the target session request initiated by the target user; and based on the target session request, obtain the session history information about the target user; When the triggering condition for switching to a large model is detected, the session history information is converted into session information that meets the input format requirements of the K candidate large models after the switch, resulting in K sets of converted session history information; where K is a positive integer greater than 0. The conversion quality of the K converted session history information is evaluated from M dimensions to obtain evaluation results; and the capabilities of the K candidate large models are probed from N dimensions to obtain probe results; where M and N are both positive integers greater than 1. Based on the evaluation results, the detection results, and the current load of the K candidate large models, the optimal large model is selected from the K candidate large models; and the response result of the target session request generated using the optimal large model is fed back to the target user.

[0005] In one possible implementation, the evaluation of the conversion quality of the K converted session history information from M dimensions to obtain the evaluation result includes: The conversion quality of the K converted session history information is evaluated from five dimensions: global semantics, local information, entity preservation, functional constraints, and time decay, and the evaluation results are obtained.

[0006] In one possible implementation, the ability to probe the K candidate large models from N dimensions to obtain the probe results includes: The capabilities of the K candidate large models are detected from five dimensions: test coverage, result stability, cross-validation, historical consistency, and adaptive weights, and the detection results are obtained.

[0007] In one possible implementation, obtaining session history information about the target user based on the target session request includes: The session identifier of the target user is identified from the target session request; and the complete session context information of the target user is obtained from one of the three-layer caches—local cache, distributed cache, or persistent storage database cache—based on the session identifier, as session history information.

[0008] In one possible implementation, converting the session history information into session information that conforms to the input format requirements of the K candidate large models after the switch, to obtain K sets of converted session history information, includes: The session history information is converted into session information that conforms to the input information format requirements of the K candidate large models after the switch, resulting in K session history messages with adapted message formats. Extract the generation control parameters and prompt words of the currently used large model from the session history information; map the generation control parameters of the currently used large model to the parameter space of K candidate large models to obtain K mapped control parameters; and reorganize the prompt words of the currently used large model into prompt words that conform to the prompt word format of K candidate large models to obtain K reorganized prompt words. The K sets of session history messages adapted to the message format, the K sets of mapped control parameters, and the K sets of recombined prompts are used to construct the K sets of converted session history information.

[0009] In one possible implementation, the evaluation of the conversion quality of the K converted session history information from five dimensions—global semantics, local information, entity preservation, functional constraints, and time decay—to obtain the evaluation results includes: The i-th converted session history information in K converted session history information is subjected to multi-scale semantic similarity fusion processing with the original session history information to obtain the global semantic fidelity of the i-th converted session history information; where i is a positive integer greater than 0 and not greater than K; Key information fragments are extracted from the conversation history information before conversion. After scoring the importance of each key information fragment and calculating the similarity between each key information fragment and the fragment most similar to it in the i-th converted conversation history information, the importance score and similarity are weighted and summed to obtain the local key information fidelity of the i-th converted conversation history information. Entity information is extracted from the session history information before conversion and the i-th session history information after conversion, and the entity retention integrity of each type of entity is calculated by comparing the extracted entity information. Calculate the retention degree of the candidate large model generation control parameters, the retention degree of functional constraints, and the retention degree of the output format corresponding to the i-th transformed session history information, and use the retention degree of the parameters, the retention degree of functional constraints, and the retention degree of the output format to calculate the retention degree of functional constraints of the i-th transformed session history information. Based on the time dimension corresponding to the session history information before conversion, calculate the time decay weighted similarity of the i-th converted session history information; The global semantic fidelity, local key information fidelity, entity retention, functional constraint preservation, and time decay weighted similarity of the i-th converted session history information are weighted and summed. The conversion quality of the i-th converted session history information is evaluated based on the calculation results, and the evaluation result for the i-th converted session history information is obtained. Similarly, the evaluation results for all K converted session history information can be obtained.

[0010] In one possible implementation, the step of evaluating the conversion quality of the i-th converted session history information based on the obtained calculation results, and obtaining the evaluation result for the i-th converted session history information, includes: When the calculated result is lower than the first evaluation threshold and not lower than the second evaluation threshold, the lowest value among the global semantic fidelity, local key information fidelity, entity retention integrity, functional constraint retention, and time decay weighted similarity is identified, and the value of the evaluation dimension corresponding to the lowest value is optimized in a targeted manner, so as to redetermine the evaluation result of the i-th converted session history information using the optimized evaluation dimension value. Alternatively, when the calculated result is lower than the second evaluation threshold, invalid values ​​among the global semantic fidelity, local key information fidelity, entity retention integrity, functional constraint retention, and time decay weighted similarity are identified, and the values ​​of the evaluation dimensions corresponding to the invalid values ​​are recalculated so as to determine the evaluation result of the i-th converted session history information again using the recalculated evaluation dimension values.

[0011] In one possible implementation, the ability of the K candidate large models is detected from five dimensions: test coverage, result stability, cross-validation, historical consistency, and adaptive weights, to obtain the detection results, including: The capability of the j-th candidate large model among K candidate large models is tested using P test cases, and the test coverage of the P test cases is calculated; where P is a positive integer greater than 0; and j is a positive integer greater than 0 and not greater than K. The j-th candidate large model is tested Q times using each test case, and the stability of the model output is determined; where Q is a positive integer greater than 1. Using S different methods, the common ability of the j-th candidate large model is verified, and the cross-validation consistency of the model's output results for different methods is calculated; where S is a positive integer greater than 1. By comparing the current detection results for the capability of the j-th candidate large model with the historical detection results, it is determined whether the current detection results meet expectations, thus determining historical consistency. Calculate the business importance and detection difficulty of the j-th candidate large model; and adaptively adjust the weights using the business importance and detection difficulty to obtain the adjusted weights; The test coverage, output stability, cross-validation consistency, historical consistency, and adjusted weights of the j-th candidate large model are weighted and summed. The capability of the j-th candidate large model is then probed based on the calculated results, and the probe results for the capability of the j-th candidate large model are obtained. Similarly, the probe results for the capability of all K candidate large models can be obtained.

[0012] In one possible implementation, the step of probing the capability of the j-th candidate large model based on the obtained calculation results, and obtaining the probing result for the capability of the j-th candidate large model, includes: When the obtained calculation result is lower than the detection threshold, an enhanced detection strategy is used to enhance the detection of the capability of the j-th candidate large model, and an enhanced detection result for the capability of the j-th candidate large model is obtained; the enhanced detection strategy includes at least one of the following strategies: increasing the number of test cases, increasing the number of repeated tests, expanding the cross-validation method, and cross-validation of different candidate large models.

[0013] In one possible implementation, selecting the optimal large model from the K candidate large models based on the evaluation results, the detection results, and the current load status of the K candidate large models includes: The evaluation results, the detection results, and the scores corresponding to the current load status of the K candidate large models are weighted and summed to obtain the comprehensive score corresponding to each of the K candidate large models. The candidate large model with the highest comprehensive score is selected as the optimal large model.

[0014] In one possible implementation, the triggering conditions for the large model switching include one or more of the following: The current large model's service instance load is higher than a preset threshold; the current large model's capabilities in the domain of the target session request are insufficient; the cost of the current large model is higher than the cost of the large model required to respond to the target session request; the current large model needs to be updated to a new version; or the current large model is experiencing a service failure.

[0015] In one possible implementation, the method for generating the response result for the target session request using the optimal large model is as follows: Using the converted session history information that meets the input format requirements of the optimal large model, a session request instruction is generated, which is then input into the optimal large model for response, resulting in a response to the target session request.

[0016] This application also provides a session processing device based on large model switching, including: a processor, a memory, and a system bus; The processor and the memory are connected via the system bus; The memory is used to store one or more programs, the one or more programs including instructions, which, when executed by the processor, cause the processor to perform any of the above-described implementations of the session processing method based on large model switching.

[0017] This application also provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform any of the above-described implementations of the session processing method based on large model switching.

[0018] This application also provides a computer program product, which, when run on a terminal device, causes the terminal device to execute any of the above-described implementations of the session processing method based on large model switching.

[0019] This application provides a session processing method, device, and computer program product based on large model switching. First, it acquires a target session request initiated by a target user. Then, based on the target session request, it acquires session history information about the target user. Next, when a trigger condition for large model switching is detected, the session history information is converted into session information conforming to the input format requirements of K candidate large models after the switch, resulting in K converted session history information sets; where K is a positive integer greater than 0. Then, the conversion quality of the K converted session history information sets is evaluated from M dimensions to obtain evaluation results; and the capabilities of the K candidate large models are probed from N dimensions to obtain probe results; where M and N are both positive integers greater than 1. Finally, based on the evaluation results, probe results, and the current load of the K candidate large models, the optimal large model is selected from the K candidate large models; and the response result for the target session request generated using the optimal large model is fed back to the target user.

[0020] As can be seen, when this application detects the need to switch large models to engage in a conversation with the target user, it converts the conversation history information about the target user into conversation information that conforms to the input format requirements of all candidate large models. It not only evaluates the conversion quality of the converted conversation context through multiple dimensions (such as global semantics, local information, entity preservation, functional constraints, and time decay), achieving precise quantification of conversion quality and optimal strategy selection, thus improving information retention, but also probes the capabilities of all candidate large models from multiple dimensions (such as test coverage, result stability, cross-validation, historical consistency, and adaptive weights), improving probe accuracy and dynamically optimizing the probe strategy. This helps to more quickly and accurately select the optimal LLM from all candidate large models for engaging in conversation with the user, thereby improving conversation efficiency and the target user's conversation experience. Attached Figure Description

[0021] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart illustrating a session processing method based on large model switching, provided for an embodiment of this application; Figure 2 This application provides an example diagram illustrating the process of obtaining session history information about a target user based on a target session request. Figure 3 Example diagram of the process for obtaining converted session history information provided in the embodiments of this application; Figure 4 An example diagram illustrating the process of evaluating the conversion quality of the converted session history information from five dimensions—global semantics, local information, entity preservation, functional constraints, and time decay—provided for embodiments of this application. Figure 5 An example diagram illustrating the process of probing the ability of candidate large models from five dimensions—test coverage, result stability, cross-validation, historical consistency, and adaptive weights—provided for embodiments of this application. Figure 6 An example diagram illustrating the process of selecting the optimal large model and then feeding back the response result for the target session request generated by the optimal large model to the target user, as provided in this embodiment of the application. Figure 7 This is a schematic diagram of the structure of a session processing device based on large model switching, provided in an embodiment of this application. Detailed Implementation

[0022] With the rapid development of large model (LLM), question answering systems based on large model have been widely used in various fields such as intelligent customer service, online education, medical diagnosis, content creation, and law, to provide users with knowledge-based question answering and multi-turn dialogue services in various scenarios.

[0023] Currently, when dealing with massive, high-concurrency user requests (such as the AICC-ASO project), a multi-instance distributed deployment architecture based on LLM is typically adopted, combined with SSE technology to achieve real-time streaming data push from the server to the client. This architecture distributes user connections to different backend service instances through load balancing, effectively improving the horizontal scalability, throughput, and fault tolerance of the question-and-answer system, enabling the system to support tens of thousands of concurrent users interacting online simultaneously.

[0024] However, while this multi-instance distributed deployment architecture brings significant performance improvements, it also introduces new technical challenges. This is because in traditional single-instance deployments, user session state and dialogue context are typically stored in the server's local memory, allowing for fast access through in-process caching. But in this multi-instance environment, each service instance runs as an independent computing unit on different physical or virtual nodes, with completely isolated memory spaces, making it impossible to directly share state data. This presents at least two challenges: (1) It is difficult to synchronize session state across instances.

[0025] Specifically, current mainstream session solutions typically use SSE to establish persistent long-lived connections between the client and server, enabling real-time streaming of large language model inference results (i.e., implementing a long-connection-based session communication mechanism). SSE is a one-way push technology based on HTTP, making it simpler to implement and naturally supporting reconnection after disconnection and event ID tracking. Therefore, in scenarios like the AICC-ASO project that handle massive, high-concurrency user requests, the SSE session solution is commonly used. Each user session is distinguished by a unique session ID. The session ID can be understood as a globally unique identifier automatically generated by the question-answering system for each user session. Its specific structure is not limited; for example, it could be "sess_20240315_user001_abc123," containing the date, user ID, and a random string, used to accurately locate and track the conversation state of a specific user in a distributed environment. The server continuously pushes the token stream generated by the model through the SSE channel.

[0026] Therefore, in a multi-instance deployment environment, since SSE connections are bound to specific service instances, when the load balancer routes subsequent user requests to other service instances, the new service instances cannot access the session history and context information stored in the original service instance, resulting in the loss of session memory (i.e., session context). Although a "sticky session" strategy can be used to consistently route requests from the same user to the same service instance, this reduces the effectiveness of load balancing and prevents smooth session migration in the event of instance failure.

[0027] (2) The challenge of session context migration during cross-LLM switching.

[0028] Specifically, in practical applications, question-answering systems often need to support dynamic switching between multiple large language models (such as iFlytek Spark, GPT-4, etc.) to meet different performance, cost, and functional requirements. For example, users may switch from a general dialogue model to a domain-specific model during a conversation, or perform intelligent routing between different models based on load. Thus, because user sessions may need to switch between different LLM service instances (e.g., from iFlytek Spark to GPT-4) during model upgrades, A / B testing, or intelligent routing scenarios, they face at least the following three challenges: Firstly, there is the issue of heterogeneity in the representations within the model.

[0029] Each LLM is optimized based on different pre-trained data, architectural design, vocabulary, and training objectives, resulting in significant differences in their internal semantic representation spaces. This heterogeneity makes it impossible to directly pass the hidden states or attention weights of the LLM to achieve context transfer; context transfer must be "downgraded" to the textual form of the dialogue history. However, different LLMs also differ in their ability to understand the same text and their reasoning paths. Even when provided with the exact same historical dialogue text, a new LLM may not be able to fully capture the deep semantics, implicit intentions, and character settings in previous dialogues, leading to abrupt changes in dialogue style or logical inconsistencies.

[0030] Secondly, the heterogeneity of API formats and the high complexity of conversion are significant issues.

[0031] It is understandable that the API interface designs of different LLM providers often differ significantly, mainly in the following aspects: differences in request format, inconsistent parameter naming, differences in support for special functions, and differences in response format.

[0032] The differences in request formats can be illustrated by examples such as: the OpenAI API uses a standardized messages array structure, while other LLMs may use a query / history key-value pair structure. Inconsistent parameter naming can be understood as: parameters such as temperature (temperature coefficient / generation randomness), top_p (kernel sampling threshold), and max_tokens (maximum generation length / maximum number of tokens) may be named sampling_temperature, topP, max_length, etc., in some LLMs. Differences in special function support can be understood as: some LLMs support advanced features such as system prompts, function calls, and multimodal input, while others may not. Differences in response formats can be understood as: the data format, completion identifier, and error code system of streaming responses may all differ.

[0033] Thirdly, the consistency of the user experience is compromised.

[0034] Understandably, due to the technical hurdles of context transfer, users often encounter the following problems after switching to an LLM: First, there is a break in conversational memory, meaning the new LLM "forgets" the previously discussed topics and user preferences, leading to repetitive questioning. Second, there is inconsistency in roles and styles, meaning the new LLM cannot maintain the same role-playing and conversational style after switching. Third, there is a break in reasoning logic, meaning that for complex tasks requiring multiple rounds of reasoning, switching to an LLM can cause the reasoning chain to break.

[0035] These problems force users to re-enter background information and repeat their needs, severely reducing dialogue efficiency and user satisfaction.

[0036] In response to this, during LLM-based session processing, when a user switches from one LLM to another, the session context needs to undergo format conversion and semantic adaptation. This requires evaluating the conversion quality of the converted session context information and probing the capabilities of each candidate LLM that can be used as the converted LLM. The candidate LLM with the best capabilities is then selected as the converted LLM. This allows for conversations with the user based on a session context with high conversion quality (i.e., high information retention), thereby improving the efficiency of dialogue with the user and the user's session experience.

[0037] However, current methods for evaluating the quality of conversation context switching typically rely solely on simple cosine similarity comparisons of overall semantics, which presents at least four problems: First, they only consider overall semantics, neglecting the local loss of key information (such as names, order numbers, amounts, etc.); second, they fail to consider the differences in importance of different information; third, they cannot quantify the degree of retention of functional information constraints (such as role settings and format requirements); and fourth, they lack temporal context decay modeling, thus reducing the accuracy of conversion quality assessment.

[0038] Furthermore, in multi-LLM switching scenarios, the question-answering system also needs to accurately evaluate the actual capabilities of each candidate LLM in order to select the optimal LLM as the switched-off LLM. However, when selecting the optimal LLM after switching, existing technologies typically probe the capabilities of each candidate LLM based on fixed test cases. This has problems such as insufficient test coverage, the influence of random factors on the results of a single probe, and the inability to quantify the reliability of the probe results. Specifically, these problems manifest in the following five aspects: First, test case design relies on expert experience, which may miss key capabilities; second, the results of a single probe may be affected by temporary factors (network fluctuations, model instability); third, the reliability of the probe results cannot be quantified; fourth, the probe difficulty varies for different capabilities, but treating them uniformly will cause deviations in the probe results; fifth, it is impossible to adaptively adjust the test strategy based on LLM feedback. As a result, the accuracy of probing the capabilities of each candidate LLM is low, which leads to the inability to switch to the optimal LLM to conduct a conversation with the user, thereby reducing conversation efficiency and the user's conversation experience.

[0039] To address the aforementioned shortcomings, this application provides a session processing method based on large model switching. First, a target session request initiated by the target user is obtained. Then, based on the target session request, session history information about the target user is acquired. Next, when a trigger condition for large model switching is detected, the session history information is converted into session information conforming to the input format requirements of K candidate large models after the switch, resulting in K converted session history information sets; where K is a positive integer greater than 0. Then, the conversion quality of the K converted session history information sets is evaluated from M dimensions to obtain evaluation results; and the capabilities of the K candidate large models are probed from N dimensions to obtain probe results; where M and N are both positive integers greater than 1. Finally, based on the evaluation results, probe results, and the current load of the K candidate large models, the optimal large model is selected from the K candidate large models. Finally, the response result for the target session request generated using the optimal large model is fed back to the target user.

[0040] As can be seen, when this application detects the need to switch large models to engage in a conversation with the target user, it converts the conversation history information about the target user into conversation information that conforms to the input format requirements of all candidate large models. It not only evaluates the conversion quality of the converted conversation context through multiple dimensions (such as global semantics, local information, entity preservation, functional constraints, and time decay), achieving precise quantification of conversion quality and optimal strategy selection, thus improving information retention, but also probes the capabilities of all candidate large models from multiple dimensions (such as test coverage, result stability, cross-validation, historical consistency, and adaptive weights), improving probe accuracy and dynamically optimizing the probe strategy. This helps to more quickly and accurately select the optimal LLM from all candidate large models for engaging in conversation with the user, thereby improving conversation efficiency and the target user's conversation experience.

[0041] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0042] See Figure 1 This is a flowchart illustrating a session processing method based on large model switching provided in this embodiment. The method includes the following steps: S101: Obtain the target session request initiated by the target user; and obtain the session history information about the target user based on the target session request.

[0043] In this embodiment, any conversation request requiring a response input by the user in intelligent interactive conversation software or device such as LLM is defined as a target conversation request to be responded to, and the user initiating the target conversation request is defined as the target user. It should be noted that this embodiment does not limit the way the target user initiates the target conversation request; it can be initiated via voice or text input. Furthermore, when the target user initiates the target conversation request via voice, this embodiment does not limit the language type of the voice input, such as Chinese or English, and it does not limit the length of the voice input, such as a single sentence or multiple sentences. Similarly, when the target user initiates the target conversation request via text input, this embodiment does not limit the language type of the input text, such as Chinese or English, and it does not limit the length of the input text, such as sentence text (i.e., a collection of words) or paragraph text (i.e., a collection of sentences).

[0044] For example, a target user's target session request could be something like, "Can you teach me how to book a flight from Beijing to Shanghai tomorrow?"

[0045] Specifically, after obtaining the target session request initiated by the target user, such as Figure 2 As shown, the load balancer can further identify the target user's session ID from the target session requests to determine if it is a new conversation. If so (i.e., no session ID for the target user is identified), then a session ID for the target user needs to be generated. If the target user's session identifier (e.g., "sess_20240315_user001_abc123") is not identified, the target session request is routed to the corresponding service instance based on the session identifier (e.g., "sess_20240315_user001_abc123"). The service instance then retrieves (loads) the target user's complete session context information from one of the three cache layers: local cache (which can be understood as L1 memory), distributed cache (e.g., Redis, which can be understood as L2 memory), or persistent storage database (e.g., MongoDB, which can be understood as L3 memory). This information is used as the target user's session history information for subsequent step S102.

[0046] For the L1 local memory cache (latency <1ms), each service instance maintains a copy of the context of the most recently active sessions in its local memory, typically using an LRU (Least Recently Used) eviction policy, with a capacity of approximately 1000 sessions. When a session request initiated by a target user is routed to the same service instance, the session context can be retrieved directly from local memory with sub-millisecond latency, without any network communication.

[0047] For the L2 distributed cache Redis (latency <10ms), the context of all active sessions is stored in the Redis cluster, serving as a shared intermediate cache layer across instances. When an L1 cache miss occurs (e.g., a session request initiated by the target user is routed to a new service instance), the question-answering system can load the session context from the L2 distributed cache Redis, with latency controlled within 10 milliseconds. The L2 distributed cache Redis can also serve as a message bus for cross-instance state synchronization, broadcasting session updates to all service instances via a publish / subscribe (Pub / Sub) mechanism.

[0048] Furthermore, for L3 persistent storage MongoDB (latency <100ms), the complete history of all sessions is persistently stored in MongoDB as the final data buffer. When L2 also fails (e.g., a session is evicted by Redis after prolonged inactivity), the question-answering system can recover the complete session context information from MongoDB. Additionally, MongoDB can also be used for long-term archiving and analysis of session data.

[0049] Next, this application will provide examples of the target user's session history information (i.e., session context information) stored in local cache, distributed cache, or persistent storage database: Suppose the target user previously initiated a dialogue request with the LLM (such as Spark Assistant) of the question-and-answer system, asking "Can you teach me how to book a flight from City A to City B tomorrow?". After three rounds of interaction, the dialogue history between the LLM (such as Spark Assistant) and the target user can be shown in Table 1 below: Table 1

[0050] Based on this, the target session request is routed to the corresponding service instance according to the session identifier (such as "sess_20240315_user001_abc123"). The complete session context information (i.e., the target user's session history information) of the target user can be obtained (loaded) from the L1 local memory cache, L2 distributed cache Redis, or L3 persistent storage MongoDB using the service instance. This information can include the following: { "session_id": "sess_20240315_user001_abc123", "current_model": "SparkDesk-v3.5", "system_prompt": "You are a professional flight booking assistant. Please use concise and friendly language to help users complete flight bookings." "conversation_history": [ {"role": "user", "content": "Can you teach me how to book a flight from City A to City B for tomorrow?"}, {"role": "assistant", "content": "Okay, I'll help you check flight tickets from City A to City B for tomorrow..."}, {"role": "user", "content": "Departing this morning, economy class, preferably XX Airlines"}, {"role": "assistant", "content": "The following flights were found for you: ①Flight number 1..."}, {"role": "user", "content": "Third shift, please help me book using Zhang's ID card 132XXXX0001"}, {"role": "assistant", "content": "Okay, reserving flight number 3 for Mr. Zhang (ID card: 132XXXX0001)..."} ], "extracted_entities": { "PERSON": ["Zhang Mou"], "ID_NUMBER": ["132XXXX0001"], "LOCATION": ["City A", "City B"], "DATE": ["Tomorrow"], "PRODUCT": ["Flight Number 3"], "MONEY": ["¥xx"], "ORG": ["XX Airlines"] }, "user_preferences": { "cabin_class": "Economy Class", "time_preference": "morning", "airline_preference": "XX Airlines" }, "functional_constraints": { "role": "Flight Booking Assistant", "tone": "simple and friendly", "output_format": "Structured list" }, "parameters": { "temperature": 0.7, "max_tokens": 2048, "top_p": 0.9} } Here, "current_model" represents the currently used main model; "system_prompt" represents the prompt words used by the currently used main model; "conversation_history" represents the multi-turn dialogue history with the target user; "extracted_entities" represents the entity extraction results, which can also be understood as the structured data after dialogue semantic parsing; "PERSON" represents a person's name (e.g., Zhang); "ID_NUMBER" represents an ID number (e.g., 132XXXX0001); "LOCATION" represents a location (e.g., City A, City B); "DATE" represents a date (e.g., tomorrow); "PRODUCT" represents a product (in this example, a specific flight number, such as flight number 3); "MONEY" represents the amount / price; and "ORG" represents an organization (in this example, an airline). "user_preferences" represents user preferences and constraints, such as "economy class", "departing in the morning", "XX airline", etc.; "functional_constraints" represents functional constraints; "role" represents role restrictions; "tone" represents tone constraints; "output_format" represents a structured list; "parameters" represents model generation parameters used to control the output features of the large model, such as "temperature" being 0.7, indicating a certain degree of randomness and creativity, and not being too rigid; "max_tokens" for maximum output length is 2048, indicating that the model's response length is limited to approximately 1000-1500 Chinese characters; "top_p" for kernel sampling threshold is 0.9, indicating that words are selected from words with a cumulative probability of 90%, balancing the accuracy and richness of the response.

[0051] S102: When the triggering condition for large model switching is detected, the session history information is converted into session information that meets the input format requirements of the K candidate large models after the switch, and K converted session history information is obtained; where K is a positive integer greater than 0.

[0052] In this embodiment, in step S101, the load balancer routes the target session request to the corresponding service instance based on the session identifier (e.g., "sess_20240315_user001_abc123"). After the service instance obtains (loads) the complete session context information (i.e., the target user's session history information) of the target user from the L1 local memory cache, L2 distributed cache Redis, or L3 persistent storage MongoDB, during the dialogue with the target user, when the trigger condition for switching to a large model is detected, it is necessary to switch from the currently used large model to another large model to continue the session with the target user. However, due to the differences in the API interface format of different large models, the complete session context information (i.e., the target user's session history information) of the target user obtained through step S101 is stored based on the format required by the previously used large model (e.g., the format based on iFlytek Spark), and it cannot be directly used for the large model after the switch. Therefore, the target user's session history information needs to be converted from the currently used large model format (such as the format of iFlytek Spark Assistant) to the session information that meets the requirements of all K (the specific value is not limited and can be any positive integer greater than 0) candidate large model input formats after the switch, so as to obtain K converted session history information for the subsequent step S103.

[0053] This application does not limit the specific content of the triggering conditions for large model switching, and can set them according to the actual situation and experience. For example, the triggering conditions for large model switching can include one or more of the following: (1) The load of the service instance of the large model currently in use is higher than the preset threshold.

[0054] This can be understood as the service instance of the currently used large model (such as iFlytek Spark) being overloaded (e.g., CPU utilization > 85%, request queue length > 100, etc.). The question-and-answer system can automatically route new target session requests initiated by the target user to other large model service instances with lower load.

[0055] (2) The current large model is not capable enough in the domain to which the target session request belongs.

[0056] This can be understood as the target session request indicating that the next session with the target user needs to enter a specific domain (such as transitioning from casual conversation to coding, legal consultation, etc.). If the current large model is not capable enough in this specific domain, the question answering system needs to automatically switch to a large model that is more proficient in this specific domain based on the capability profile.

[0057] (3) The cost of the large model currently in use is higher than the cost of the large model required to respond to the target session request.

[0058] For example, for simple confirmation dialogues (such as "Please confirm whether to submit the order"), the question-answering system can switch from the currently used high-cost answer model to a low-cost answer model to reduce inference costs.

[0059] (4) The current large model needs to be updated to the new version.

[0060] In practical applications, if the provider of the current large model releases a new version of the large model, the question answering system needs to smoothly migrate active sessions to the new version of the large model.

[0061] (5) The large model currently in use is experiencing a service failure.

[0062] In practical applications, if the currently used large model service encounters an anomaly (such as timeout, increased error rate, etc.), the question-answering system can automatically switch to a backup large model to ensure service continuity.

[0063] This embodiment does not limit the method for determining the K candidate large models. One possible implementation is as follows: Figure 3 As shown, when the triggering conditions for large model switching are detected (such as detecting that the service instance of the currently used large model is overloaded), a large model switch is required. At this time, a list of currently available candidate large models can be obtained from the model registry (which can be understood as the core management component in a multi-model, multi-instance system, responsible for uniformly managing the metadata, version, endpoint, health status, and routing policies of all available large models (including general models and professional domain models). Then, based on basic capability matching (such as whether it supports Chinese dialogue, whether it supports multi-turn conversations, whether it supports streaming output, etc.), K candidate large models are selected. Taking K=3 as an example, the three candidate large models obtained can be shown in Table 2 below: Table 2

[0064] Based on this, by executing steps S102-S104, the session history information is transformed, the transformation quality is evaluated, and the capability detection is performed on these K candidate large models respectively. The optimal large model can then be selected as the switched large model for subsequent sessions with the target user.

[0065] Specifically, the conversion process for the target user's session history information can be as follows: First, the session history information can be converted into session information that meets the input information format requirements of the K candidate large models after the switch, resulting in K session history messages with adapted message formats.

[0066] For example, conversation history information can be converted from the format of the currently used large model (such as the format of iFlytek Spark Assistant) to the input format requirements of the first candidate large model after the switch (the specific structure is not limited and can be any existing or future large model). Based on the above example, the conversation history information of the target user can be converted into conversation history information that meets the input information format requirements of the first candidate large model, resulting in conversation history messages with adapted message format, as shown below: { "model": "First candidate large model", "messages": [ {"role": "system", "content": "You are a professional flight booking assistant. Please use concise and friendly language to help users complete flight bookings."}, {"role": "user", "content": "Can you teach me how to book a flight from City A to City B for tomorrow?"}, {"role": "assistant", "content": "Okay, I'll check flight tickets for you from City A to City B tomorrow. Do you have a preferred departure time? Economy or business class?"}, {"role": "user", "content": "Departing this morning, economy class, preferably XX Airlines"}, {"role": "assistant", "content": "The following flights were found for you: ① Flight 1 08:00-10:20 ¥xx ② Flight 2 10:30-12:50 ¥xx ③ Flight 3 11:00-13:20 ¥xx. Which flight would you like to book?"}, {"role": "user", "content": "Third shift, please help me book using Zhang's ID card 132XXXX0001"}, {"role": "assistant", "content": "Okay, booking flight number 3 for Mr. Zhang (ID card: 132XXXX0001), departing from City A to City B tomorrow at 11:00, economy class ¥xx. Please confirm to submit order?"} ], "temperature": 0.7, "max_tokens": 2048, "top_p": 0.9, "stream": true } Here, "stream": true indicates that the first candidate large model adopts a text-to-text streaming output mode.

[0067] Secondly, the generation control parameters and prompts of the currently used large model can be extracted from the session history information; the generation control parameters of the currently used large model can be mapped to the parameter space of K candidate large models to achieve parameter normalization and obtain K mapped control parameters; and the prompts of the currently used large model can be reorganized into prompts that conform to the prompt format of K candidate large models to obtain K reorganized prompts.

[0068] For example: Based on the above example, the temperature (randomness) of the currently used large model (such as the format of iFlytek Spark Assistant) remains unchanged at 0.7 in the first candidate large model. However, in some other candidate large models, it may be necessary to scale it proportionally. For example, if the temperature range of the second subsequent large model is 0-2, then the temperature (randomness) value needs to be mapped to 1.4, etc.

[0069] Furthermore, when reorganizing the original role settings, functional constraints, and other information in the currently used large model into the optimal system prompt word format for the candidate large model, some candidate large models may not support independent system role messages. In such cases, the prompt words need to be embedded into the first target user's session message to complete the fusion, ensuring that the candidate large model can accurately understand the task requirements.

[0070] Next, K sets of session history messages adapted to the message format, K sets of mapped control parameters, and K sets of recombined prompts can be used to construct K sets of converted session history information.

[0071] For example: Continuing with the example above, the conversation history information is converted into conversation information that conforms to the input format requirements of the first candidate large model. The converted conversation history information can be shown below: "[System prompt] You are a professional flight booking assistant. Please use a concise and friendly tone to help users complete flight bookings."

[0072] [Round 1 - User] Can you teach me how to book a flight from City A to City B for tomorrow?

[0073] [Round 1 - Assistant] Okay, let me check flight tickets for you from City A to City B tomorrow. Do you have a preferred departure time? Economy or business class?

[0074] [Round 2 - User] Departing in the morning, economy class, preferably with XX Airlines.

[0075] [Round 2 - Assistant] We found the following flights for you: ① Flight 1 08:00-10:20 ¥xx ② Flight 2 10:30-12:50 ¥xx ③ Flight 3 11:00-13:20 ¥xx. Which flight would you like to book?

[0076] [Round 3 - User] Third shift, please help me book using Zhang's ID card number 132XXXX0001.

[0077] [Round 3 - Assistant] Okay, we are booking flight number 3 for Mr. Zhang (ID card: 132XXXX0001), departing from City A to City B tomorrow at 11:00, economy class ¥xx. Please confirm whether to submit the order?

[0078] Next, by executing the subsequent step S103, it can be evaluated from multiple dimensions whether the K converted session history information faithfully retains the key points of the session history information before conversion, and the ability to detect all K candidate large models from multiple dimensions. Based on the selection criteria, the optimal large model is selected from all K candidate large models by executing the subsequent step S104, which is used as the switched large model for subsequent sessions with the target user, so as to improve the session efficiency and session experience of the target user.

[0079] S103: Evaluate the conversion quality of K converted session history information from M dimensions to obtain the evaluation results; and probe the capabilities of K candidate large models from N dimensions to obtain the probe results; where M and N are both positive integers greater than 1.

[0080] In this embodiment, after obtaining K sets of converted session history information through step S102, the conversion quality of the K sets of converted session history information can be further evaluated from M dimensions (the value is not limited and can be any positive integer greater than 1). If it is unqualified, it can be automatically retried for optimization. Compared with the existing technology that only uses simple cosine similarity to compare the overall semantics, it can achieve accurate quantification of conversion quality and strategy optimization, thereby improving the information retention rate. Furthermore, this application also probes the capabilities of K candidate large models from N dimensions (the value is not limited and can be any positive integer greater than 1). Compared with the existing technology that only probes the capabilities of each candidate LLM based on fixed test cases, it can further improve the detection accuracy and realize dynamic optimization of the detection strategy. This helps to select the optimal LLM from all K candidate large models more quickly and accurately as the converted large model to continue the conversation with the target user, thereby improving the conversation efficiency and the target user's conversation experience.

[0081] Specifically, one possible implementation is, as follows: Figure 4 As shown, after obtaining K sets of transformed session history information, to improve the evaluation accuracy, the Context Transformation Fidelity Evaluation (CTFE) formula can be used to evaluate the transformation quality of these K sets of transformed session history information from five dimensions: global semantics, local information, entity retention, functional constraints, and time decay. This yields more accurate evaluation results, enabling precise quantification of transformation quality and strategy optimization. Experiments have shown that using CTFE evaluation can increase the key information retention rate from 52% to 91%. The specific evaluation process is shown in steps A1-A6 below: Step A1: Perform multi-scale semantic similarity fusion processing on the i-th converted session history information and the original session history information from the K converted session history information to obtain the global semantic fidelity of the i-th converted session history information, denoted as Sg, which is used to measure the overall semantic consistency of the session history information before and after conversion. Here, i is a positive integer greater than 0 and not greater than K.

[0082] In this implementation, the specific formula for calculating the global semantic fidelity Sg is as follows: Sg = (w1×Scos + w2×Sbert + w3×Swmd) / (w1 + w2 + w3) Here, Scos represents cosine similarity, which is obtained by first encoding the i-th transformed conversation history information and the original conversation history information (i.e., the complete dialogue context) into 768-dimensional semantic vectors using a sentence-level bidirectional encoder representation model (Sentence-BERT), and then calculating the cosine similarity between the two vectors. Here, "vector" refers to the semantic representation vector obtained after encoding the i-th transformed conversation history information and the original conversation history information using Sentence-BERT.

[0083] Sbert represents a bidirectional encoder representation model (BERT). At the token semantic level, it compares the i-th transformed session history information with the untransformed session history information and calculates the semantic overlap between the two. It outputs precision, recall, and F1 composite score, which can be defined as BERTScore-F1, to measure the fine-grained semantic similarity between two pieces of session history information.

[0084] Swmd represents the normalized word move distance semantic matching index. It relies on word move distance (WMD) to measure the semantic difference between the i-th transformed conversation history and the original conversation history. WMD represents the minimum cumulative distance required for word migration between these two conversation texts within the semantic space; the smaller the original value, the closer the semantics. After normalizing WMD, a complement transformation is performed, i.e., "Swmd = 1 - WMD normalization", so that the index score increases with semantic similarity, and also ensures that the index value range is consistent with other evaluation dimensions.

[0085] w1, w2, and w3 all represent weights, and their specific values ​​are not limited. They can be set according to the actual situation and experience. For example, they can be set as: w1=0.4, w2=0.4, w3=0.2.

[0086] For example: Based on the above example, the process of calculating the global semantic fidelity Sg of the i-th transformed session history information (such as the input format of the first candidate large model) and the session history information before transformation (such as the input format of iFlytek Spark Assistant) is as follows: Scos = 0.94 (which can be understood as the cosine similarity calculated after Sentence-BERT encoding).

[0087] Sbert = 0.91 (i.e., the value of BERTScore-F1).

[0088] Swmd = 1 - 0.12 = 0.88 (that is, the normalized distance of WMD is 0.12).

[0089] Then Sg = (w1×Scos + w2×Sbert + w3×Swmd) / (w1 + w2 + w3) = (0.4×0.94 + 0.4×0.91 + 0.2×0.88) / (0.4+0.4+0.2) = 0.916.

[0090] As can be seen, since Sg takes the value of 0.916, it indicates that the global semantic fidelity is high, and the overall semantics of the i-th converted session history information is basically consistent with that of the unconverted session history information.

[0091] Step A2: Extract key information fragments from the pre-conversion session history information, score the importance of each key information fragment, and calculate the similarity between each key information fragment and the most similar fragment in the i-th converted session history information. Then, perform a weighted summation of the importance score and similarity to obtain the local key information fidelity of the i-th converted session history information, denoted as Sl, which is used to measure the degree of retention of key information fragments.

[0092] In this implementation, the local key information fidelity Sl can be used to accurately track whether each key information fragment is retained based on information entropy and attention mechanisms. The specific calculation formula for Sl is as follows: Sl = (1 / K) × Σ [i=1 to K] (IMP(ki) × Match(ki)) Where K represents the number of key information segments extracted from the pre-conversion session history information; ki represents the i-th key information segment; IMP(ki) represents the importance score calculated for the i-th key information segment; Match(ki) represents the skewness similarity between the two segments calculated using methods such as Sentence-BERT after matching the segment most similar to the i-th key information segment in the i-th converted session history information (such as the input format of the first candidate large model).

[0093] Specifically, we can first use methods such as Term Frequency-Inverse Document Frequency (TF-IDF) to evaluate the importance of a word to a document in a document set, and then combine this with Named Entity Recognition (NER) techniques to extract key information fragments (such as automatically identifying entities like person names, place names, and organization names) from the conversation history information (context) before conversion.

[0094] For example: Based on the example above, the six key information fragments (i.e., K=6) extracted from the conversation history information (context) before the conversion can be as follows: k1 = "Zhang Mou" (passenger's name) k2 = "132XXXX0001" (ID number) k3 = "Flight No. 3" (flight number) k4 = "From City A to City B" (Itinerary) k5 = "¥xx" (ticket price) k6 = "Economy class, departing in the morning" (preference constraint) Then, the importance of each extracted key information fragment can be scored, and the calculation formula is as follows: IMP(ki)=log(1 + TF-IDF(ki))×EntityWeight(ki)×PositionBoost(ki) Wherein, TF-IDF(ki) represents the term frequency-inverse document frequency value of the i-th key information fragment ki, reflecting the importance of the fragment in the conversation; EntityWeight(ki) represents the entity weight, which can be 1.5 if the fragment contains named entities, otherwise it can be 1.0; PositionBoost(ki) represents the position boosting coefficient, which can be "1.0 + 0.3×(1 - i / K)", with higher weights for fragments that appear earlier, because the information that appears first is often the core topic of the conversation.

[0095] For example: Based on the example above, substituting the six key information fragments from the example into the importance score calculation formula yields: IMP(k1) = log(1 + 0.82) × 1.5 × 1.25 = 0.60 × 1.5 × 1.25 = 1.125, which shows that “Zhang Mou” contains the PERSON (person's name) entity and is located at the beginning.

[0096] IMP(k2) = log(1+0.91) × 1.5 × 1.20 = 0.65 × 1.5 × 1.20 = 1.170, which shows that “132XXXX0001” contains an ID entity.

[0097] IMP(k3) = log(1+0.88) × 1.5 × 1.15 = 0.63 × 1.5 × 1.15 = 1.087, which shows that "Flight No. 3" contains the PRODUCT entity.

[0098] IMP(k4) = log(1+0.75) × 1.5 × 1.10 = 0.56 × 1.5 × 1.10 = 0.924, which reflects the entity of "from City A to City B" including LOCATION.

[0099] IMP(k5) = log(1+0.70) × 1.5 × 1.05 = 0.53 × 1.5 × 1.05 = 0.835, which shows that “¥xx” contains the entity MONEY (amount / price).

[0100] IMP(k6) = log(1+0.65) × 1.0 × 1.00 = 0.50 × 1.0 × 1.00 = 0.500, which shows that "Economy Class, departing in the morning" is not a physical entity.

[0101] Then perform normalization: Let ΣIMP(ki) = K = 6, then the normalization coefficient = 6 / (1.125 + 1.170 + 1.087 + 0.924 + 0.835 + 0.500) = 6 / 5.641 = 1.064 Thus, after normalization, we get: IMP'(k1)=1.197, IMP''(k2)=1.245, IMP'(k3)=1.157, IMP'(k4)=0.983, IMP'(k5)=0.888, IMP'(k6)=0.53.

[0102] Next, in the i-th transformed session history information, the segment most similar to the i-th key information segment can be matched, and the similarity between the two segments can be calculated using methods such as Sentence-BERT. The specific calculation formula is as follows: Match(ki) = max [j=1 to M] (Similarity(ki, tj)) For example, based on the above example, the similarity calculation formula above can be used to obtain: Match(k1) = 0.98 (This reflects that "Zhang Mou" is fully preserved in the i-th converted conversation history text). Match(k2) = 0.99 (This reflects that "132XXXX0001" was completely preserved) Match(k3) = 0.97 (This reflects that "Flight No. 3" was completely preserved). Match(k4) = 0.96 (This reflects the complete preservation of the data from City A to City B). Match(k5) = 0.98 (This reflects that "¥xx" was fully preserved) Match(k6) = 0.93 (This reflects that the "economy class, morning departure" condition has been largely maintained). Therefore, based on this, Sl = (1 / K) × Σ [i=1 to K] (IMP(ki) × Match(ki)) = (1 / 6) × (1.197 × 0.98 + 1.245 × 0.99 + 1.157 × 0.97 + 0.983 × 0.96 + 0.888 × 0.98 + 0.532 × 0.93) = 0.973. It can be seen that since Sl takes a value of 0.973, it indicates the fidelity of local key information.

[0103] Step A3: Extract entity information from the session history information before conversion and the i-th session history information after conversion, and calculate the entity preservation score (Se) of each type of entity after conversion by comparing the extracted entity information. This score measures the degree of preservation of named entities.

[0104] In this implementation, the entity retention integrity score Se can be used to perform a hierarchical retention assessment based on entity type, assigning differentiated weights to different types of named entities. The specific calculation formula for Se is as follows: Se=Σ [c∈C] (TypeWeight(c)×R(c)) Where C represents a set of entity types, including PERSON (person's name), ORG (organization's name), CASE_NO (case number / order number, etc.), DATE (date and time), MONEY (amount), PRODUCT (product name), LOCATION (location), etc.

[0105] R(c) represents the entity retention rate of type c, specifically calculated as: R(c) = |number of entities of this type retained after conversion| / |total number of entities of this type before conversion|.

[0106] TypeWeight(c) represents the entity type weight matrix of type c. Different weights can be assigned according to the importance of the entity in the business scenario. Examples are shown in Table 3 below: Table 3

[0107] It should be noted that this application employs the following two special processing rules when performing entity comparison: First, fuzzy matching of entities is performed, which allows approximate matching. For example, “Court of District C, City A” ≈ “Court of District C” (similarity > 0.85 is considered a match).

[0108] Second, an entity substitution penalty is applied. If an entity is incorrectly replaced with another entity (e.g., "Zhang" becomes "Li"), the value of R(c) is reduced by 0.3 (0.3 is just an example and can be replaced with other values ​​depending on the actual situation).

[0109] For example, after comparing the entity information of the conversation history before and after the conversion, the entity retention status and the corresponding values ​​of R(c) can be shown in Table 4 below: Table 4

[0110] Then Se=Σ [c∈C](TypeWeight(c)×R(c))=0.2×1.00+0.18×1.00+0.15×1.00+0.12×1.00+0.12×1.00+ 0.10×1.00+0.08×1.00+0.05×1.00=1.00.

[0111] As can be seen, since Se is 1.00, it indicates that all entities are completely preserved. This is because the format conversion in the above example is relatively standardized, and no entities are lost. If "Zhang Mou" is omitted in a certain conversion, then PERSON's R(c) = 0 / 1 = 0.00, and Se will drop to 0.80. At this point, the specific loss point of "PERSON entity loss" can be accurately located.

[0112] Step A4: Calculate the retention of the control parameters, functional constraints, and output format of the candidate large model generation control parameters corresponding to the i-th transformed session history information. Using the retention of the parameters, functional constraints, and output format, calculate the functional constraint retention of the i-th transformed session history information, denoted as Sf, which measures the degree of retention of constraints such as parameters, format, and roles.

[0113] In this implementation, the degree of preservation of functional constraints Sf can be used to quantify the extent to which non-semantic information (parameters, formats, constraints, etc.) is preserved. The specific calculation formula is as follows: Sf=(Fparam+Fformat+Fconstraint) / 3 Here, Fparam represents the parameter preservation degree, used to measure the extent to which the control parameters generated by the corresponding candidate model are retained. The specific calculation formula is: Fparam = 1 - (1 / N) × Σ|ParamDiff(pi)|, where ParamDiff(pi) = |parameter value after transformation - parameter value before transformation| / parameter value range. pi represents the i-th parameter (such as temperature, max_tokens, etc.), and N represents the total number of parameters. If a parameter is deleted after transformation, the corresponding ParamDiff = 1.0 (i.e., the maximum penalty).

[0114] For example: Continuing with the previous example, if `temperature` is 0.7 before conversion and also 0.7 after conversion, with a range of 0-2, then the corresponding `ParamDiff` is |0.7-0.7| / 2.0 = 0.00. Similarly, if `max_tokens` is 2048 before conversion and also 2048 after conversion, then the corresponding `ParamDiff` is |2048-2048| / 4096 = 0.00. Furthermore, if `top_p` is 0.9 before conversion and also 0.9 after conversion, then the corresponding `ParamDiff` is |0.9-0.9| / 1.0 = 0.00. Based on this, we can further derive `Fparam` = 1 - (1 / 3) × (0.00 + 0.00 + 0.00) = 1.00.

[0115] Fformat represents the format preservation degree, which measures the consistency of the output format tags before and after conversion. In practical applications, the Jaccard Similarity Coefficient can be used to achieve this, that is, the specific value is: Fformat = Jaccard(set of format tags before conversion, set of format tags after conversion), where Jaccard similarity coefficient = |A∩B| / |A∪B, and the format tags can include, but are not limited to: JSON, Markdown, tables, lists, code blocks, etc.

[0116] For example: Suppose the format tag before conversion is: {structured list, plain text}. And the format tag after conversion is: {structured list, plain text}. Then, Fformat = |{structured list, plain text}| / |{structured list, plain text}| = 2 / 2 = 1.00.

[0117] Furthermore, Fconstraint represents the constraint retention degree, used to verify one by one whether the functional constraints in the session history information before the transformation are retained after the transformation. The specific calculation formula is: Fconstraint = number of constraints satisfied / total number of original constraints.

[0118] For example: Based on the above example, the constraint type and verification result may specifically include the following: If the role is set as "Flight Booking Assistant" and is retained in the System Prompt after conversion, then the value is 1 (i.e., it satisfies the condition).

[0119] The tone requirement is "concise and friendly," and this is also preserved in the System Prompt after conversion, so the value is 1 (that is, it is satisfied).

[0120] If the output format is "structured list", and this is retained after conversion, then the value will also be 1 (i.e., it satisfies the condition).

[0121] Therefore, Fconstraint = 3 / 3 = 1.00.

[0122] Based on this, we can obtain Sf = (Fparam + Fformat + Fconstraint) / 3 = (1.00 + 1.00 + 1.00) / 3 = 1.00. It is evident that since Sf takes the value 1.00, the functional constraints are completely preserved.

[0123] Step A5: Based on the time dimension corresponding to the conversation history information before conversion, calculate the time decay weighted similarity of the i-th converted conversation history information, denoted as St, which is used to measure the conversion quality of each round of dialogue by weighting according to the time proximity.

[0124] In this implementation, the time-decay weighted similarity St fully considers the time dimension of the target user's dialogue history. That is, recent dialogues have a greater impact on the target user's current experience, so they can be given higher weight. The specific calculation formula is as follows: St=Σ [i=1 to N] (DecayWeight(i)×LocalSim(ti_before,ti_after)) / ΣDecayWeight(i) Where N represents the number of dialogue rounds with the target user; ti represents the i-th round of dialogue with the target user; DecayWeight(i) represents the exponentially decaying weight function, and its specific value is calculated as: DecayWeight(i) = e^(-λ×(Ni)), where λ represents the decay coefficient, and its specific value is not limited. It can be set according to the actual situation and experience, such as 0.15. It can be understood that the larger λ is, the more attention is paid to recent dialogues. When λ=0, it degenerates into equal weight; LocalSim(ti_before,ti_after) represents the local similarity of the conversation information before and after the i-th round of dialogue transformation. Its specific value can be calculated using Sentence-BERT.

[0125] For example, taking a 3-round dialogue with λ=0.15 as an example, the calculation process of the attenuation weight in each of these 3 rounds of dialogue can be as follows: Round 3 (Latest): DecayWeight(3) = e^(-0.15×(3-3)) = e^0 = 1.000 Round 2: DecayWeight(2) = e^(-0.15×(3-2)) = e^(-0.15) = 0.861 Round 1 (earliest): DecayWeight(1) = e^(-0.15×(3-1)) = e^(-0.30) = 0.741.

[0126] Furthermore, the calculation results of the local similarity of each round in these three rounds of dialogue (such as the semantic similarity of each round of dialogue before and after transformation using Sentence-BERT) can be shown below: Round 1: LocalSim = 0.95 (simple Q&A, high conversion fidelity) Round 2: LocalSim = 0.93 (includes flight list, format slightly different) Round 3: LocalSim = 0.97 (Contains key booking information, fully retained) Based on this, we can further obtain: St = (1.000×0.97+0.861×0.93+0.741×0.95) / (1.000+0.861+0.741) = 0.951. It is evident that since St takes the value of 0.951, it fully reflects the design concept of this application: "the latest dialogue has the highest weight." That is, the weight of the 3rd round (latest) is 1.000, while the weight of the 1st round (earliest) is only 0.741, which conforms to the natural decay characteristics of human dialogue memory.

[0127] Step A6: Perform a weighted summation of the global semantic fidelity, local key information fidelity, entity retention, functional constraint preservation, and time decay weighted similarity of the i-th converted session history information, and evaluate the conversion quality of the i-th converted session history information based on the calculation results to obtain the evaluation result for the i-th converted session history information. Similarly, the evaluation results for all K converted session history information can be obtained.

[0128] In this implementation, after obtaining the values ​​of global semantic fidelity Sg, local key information fidelity Sl, entity integrity retention Se, functional constraint preservation Sf, and time decay weighted similarity St through the above steps A1-A5 (the specific execution order is not limited), the values ​​of the five evaluation dimensions can be further weighted and summed according to the weight coefficients to obtain the final conversion fidelity comprehensive score (denoted as CTFE) corresponding to the i-th converted session history information. The specific calculation formula is as follows: CTFE=α×Sg+β×Sl+γ×Se+δ×Sf+ε×St Where α, β, γ, δ, and ε all represent weights, and their specific values ​​are not limited. They can be set according to actual conditions and experience. For example, they can be set as: α=0.25, β=0.3, γ=0.2, δ=0.15, ε=0.1.

[0129] For example: Based on the above example, when Sg=0.916, Sl=0.973, Se=1.00, Sf=1.00, St=0.951, taking α=0.25, β=0.3, γ=0.2, δ=0.15, ε=0.1 as an example, we can further calculate CTFE=0.25×0.916+0.30×0.973+0.20×1.000+0.15×1.000 +0.10×0.951=0.966, with a full score of 1.0. It can be seen that the conversion quality of the i-th converted session history information is excellent.

[0130] By doing so, the evaluation of all K converted session history information can be achieved.

[0131] Furthermore, in practical applications, two evaluation thresholds for the Comprehensive Transformation Fidelity Evaluator (CTFE) can be preset and defined as the first evaluation threshold and the second evaluation threshold, respectively. The values ​​of these two thresholds are not limited; for example, they can be set to 0.85 and 0.75, respectively. This allows for the implementation of different challenge strategies based on different CTFE values.

[0132] Specifically, such as Figure 4 As shown, when the value of CTFE is not lower than the first evaluation threshold (e.g., 0.85), it indicates that the conversion quality of the converted session history information is very high, and the converted session history information can be directly used for subsequent dialogues. For example, CTFE=0.966 in the example above belongs to this case.

[0133] Alternatively, if the CTFE value is lower than the first evaluation threshold (e.g., 0.85) and not lower than the second evaluation threshold (e.g., 0.75), it indicates that the conversion quality of the converted session history information is acceptable but still needs optimization. In this case, the lowest value among global semantic fidelity Sg, local key information fidelity Sl, entity retention integrity Se, functional constraint retention Sf, and time decay weighted similarity St can be identified, and the values ​​of the evaluation dimensions corresponding to the lowest value can be optimized in a targeted manner. The evaluation results of the (i-th) converted session history information can be re-determined using the optimized evaluation dimension values.

[0134] If Sg is identified as low, semantic alignment processing can be added to semantically refine the converted conversation history text, making it closer to the semantics of the original conversation history. If Sl is identified as low, the Match values ​​of key information fragments can be checked first, and then the missing key information can be explicitly added to the converted conversation history (context). If Se is identified as low, missing entities can be located first, and then forcibly injected into the converted conversation history (context) (e.g., adding "Please note the following key information: Passenger Zhang, ID card number 132XXXX0001" to the System Prompt). If Sf is identified as low, lost functional constraints can be repaired, such as restoring deleted parameters and supplementing role settings. If St is identified as low, the conversion quality of recent dialogue turns can be optimized.

[0135] Alternatively, if the CTFE value is lower than the second evaluation threshold (e.g., 0.75), it indicates that the conversion quality of the converted session history information is unqualified. In this case, invalid values ​​can be identified in the global semantic fidelity Sg, local key information fidelity Sl, entity retention Se, functional constraint retention Sf, and time decay weighted similarity St. The values ​​of the evaluation dimensions corresponding to the invalid values ​​are recalculated, and the evaluation results of the i-th converted session history information are determined again using the recalculated evaluation dimensions.

[0136] It should be noted that for evaluation dimensions corresponding to identified invalid values ​​(such as missing entities / information gaps / constraint violations), although targeted optimization and re-execution of the transformation are possible, to conserve computational resources, a maximum of 3 retries are typically set. If it still fails after 3 attempts, the corresponding candidate large model is marked as having substandard session history information transformation quality, and its priority can be reduced when selecting the optimal large model (e.g., ...). Figure 4 (The "label deweighting" shown in the image).

[0137] For example, suppose a transformation identifies Se=0.27 (due to the missing entity "Zhang Mou"). Then, the specific loss point can be located: the "PERSON" type entity "Zhang Mou" does not find a match in the transformed session history (context) text. Next, repair can be performed, that is, the missing entity information is added to the transformed session history (context). Then, the CTFE value is recalculated to verify the repair effect.

[0138] Thus, this application employs a five-dimensional fusion of conversion fidelity evaluation (CTFE) – global semantic fidelity (Sg), local key information fidelity (Sl), entity integrity retention (Se), functional constraint retention (Sf), and time-decayed weighted similarity (St) – based on a local matching mechanism of information entropy, a hierarchical weight matrix of entity types, a functional constraint quantification model, and exponential decay time modeling. This achieves a refined, differentiated, and multi-dimensional comprehensive evaluation of conversion quality. Compared to existing methods that only calculate the overall similarity score (e.g., 0.85) using cosine similarity, this application can precisely pinpoint specific loss points (e.g., "loss of personal name entity leading to Se=0.27"), improving the evaluation effect of conversion quality from "evaluable" to "interpretable and optimizable." Furthermore, experiments demonstrate that the evaluation method provided by this application can increase the conversion quality pass rate from 72% to over 91%.

[0139] In addition, another alternative implementation is, as Figure 5 As shown, after conducting CTFE evaluation and adjusting the conversation history information (context) from five dimensions (global semantics, local information, entity preservation, functional constraints, and time decay) as disclosed above, and obtaining K sets of transformed conversation history information suitable for the corresponding candidate large models, in order to improve the accuracy of the detection estimation of the capabilities of each candidate large model and select the optimal large model as the transformed large model to conduct conversations with the target user, this application not only inputs these K sets of transformed conversation history information into K candidate large models (such as the first candidate large model, the second candidate large model, the third candidate large model, etc.) to generate response results through the K candidate large models, but also proposes to adopt the Adaptive Confidence Capability Evaluation Model (ACCEM) to detect the actual capabilities of all K candidate large models from five dimensions: test coverage, result stability, cross-validation, historical consistency, and adaptive weights, to obtain more accurate detection results and realize dynamic optimization of the detection strategy. Experiments have also shown that the detection accuracy can be improved to 35% by using the ACCEM detection method.

[0140] One approach to generating response results using K candidate large models is as follows: K sets of high-quality, suitable-for-the-corresponding-candidate-large-model-used converted conversation history information are integrated into a pre-built prompt template to generate a conversation request instruction. The generated conversation request instruction is then input into the corresponding candidate large model (LLM) for response, resulting in a response to the target conversation request.

[0141] For example: Suppose the converted session history information is: the user's latest request is "Third shift, please help me book using Zhang's ID card 132XXXX0001". This is integrated into a pre-built prompt template to generate a session request instruction. After inputting this instruction into the first, second, and third candidate models respectively, the following are examples of the responses from each candidate model: (1) The response content generated by the first candidate large model: Okay, your booking information has been confirmed. Passenger: Zhang (ID card number: 132XXXX0001) Flight: Flight No. 3, XX Airlines Date: Tomorrow (date in [year], month], and day) 11:00-13:20 Route: City A → City B Cabin class: Economy Ticket price: ¥xx Please confirm that the above information is correct, and I will submit your order. Please let me know if you need any changes.

[0142] (2) The response content generated by the second candidate large model: "Received, I'll place the order for you. Order details are as follows:" Passenger Name: Zhang ID Number: 132XXXX0001 Flight Information: Flight No. 3 (XX Airlines), departing from City A at 11:00 tomorrow and arriving in City B at 13:20. Seating class: Economy Ticket price: xx yuan If everything is correct, please reply "Confirm" and I will submit it for you immediately.

[0143] (3) The response content generated by the third candidate large model: Okay! Let me organize your booking information for you~ Passenger: Zhang Flight: Flight number 3 tomorrow morning Flying from City A to City B, economy class xx yuan. Do I need to confirm my order?

[0144] Next, the ACCEM probing method used in this application will be described in detail. The specific probing process is shown in steps B1-B6 below: Step B1: Test the capabilities of the j-th candidate large model among the K candidate large models using P test cases, and calculate the test coverage of the P test cases, denoted as TestCoverage(j), which measures the sufficiency of the test cases in covering the capabilities of the probed candidate LLM. Here, P is a positive integer greater than 0; j is a positive integer greater than 0 and not greater than K.

[0145] Test cases refer to standardized test inputs and expected outputs designed in advance to probe a specific capability of the candidate large model. For example, in the above example scenario of flight booking, test case examples designed for the candidate large model's "multi-turn dialogue context understanding" capability can be shown in Table 5 below: Table 5

[0146] For the j-th candidate large model (such as the first candidate large model), the specific formula for calculating test coverage (TestCoverage(j)) is as follows: TestCoverage(j)=(N_actual / N_required)×DiversityScore(j) Where N_actual represents the number of test cases actually executed; N_required represents the number of test cases theoretically required to fully cover the capabilities of the candidate large model (such as the "multi-turn dialogue context understanding" capability); DiversityScore(j) represents the diversity score of the test cases, and the specific value can be calculated using Shannon Entropy.

[0147] For example, suppose theoretically 10 test cases are needed to fully cover the "multi-turn dialogue context understanding" capability test, but 7 test cases were actually executed (as mentioned in Table 5 above, T1-T7). Then, N_actual / N_required = 7 / 10 = 0.70. Furthermore, as shown in Table 5 above, of these 7 test cases, 2 are easy (28.6%), 3 are medium (42.8%), and 2 are difficult (28.6%). The process of calculating the diversity score (DiversityScore(j)) and test coverage (TestCoverage(j)) of these 7 test cases according to difficulty is as follows: Shannon Entropy=-(0.286×log2(0.286)+0.428×log2(0.428)+0.286×log2(0.286))=1.558 MaxEntropy = log2(3) = 1.585 DiversityScore(j)=1.558 / 1.585=0.983 TestCoverage(j)=0.70×0.983=0.688 Step B2: Repeat the test Q times for the j-th candidate large model using each test case, and determine the stability of the model output results, denoted as ResultStability(j), which measures the consistency of the results across multiple probes. Here, Q is a positive integer greater than 1.

[0148] The stability of the results, ResultStability(j), can be measured by repeatedly executing the j-th candidate large model Q times (e.g., 3 times) for each test case to measure the consistency of the model's output results. The specific formula for calculating ResultStability(j) is as follows: ResultStability(j)=1-CV(results)×(1+OutlierPenalty) Here, CV stands for Coefficient of Variation, calculated as: CV = σ / μ, which is the standard deviation σ divided by the mean μ. It measures the dispersion of the data. A smaller CV value indicates a more stable result. OutlierPenalty represents the penalty for outliers, calculated as: OutlierPenalty = number of results that deviate from the median by more than 2 standard deviations / total number of tests. If a result deviates significantly from the normal range, it indicates that the detection result is unreliable and a penalty needs to be increased.

[0149] For example: Suppose that the accuracy of the "multi-turn dialogue context understanding" ability of the j-th candidate large model (such as the first candidate large model) is obtained in 3 trials as follows: 0.89 in the first trial, 0.91 in the second trial, and 0.9 in the third trial. Then the mean μ = (0.89 + 0.91 + 0.9) / 3 = 0.900, the standard deviation σ = 0.008, and the CV = σ / μ = 0.008 / 0.900 = 0.009.

[0150] When performing outlier detection, considering that the deviations of all the above results from the median (0.90) do not exceed 2σ (i.e., 2 × 0.008 = 0.016), there are no outliers. Therefore, OutlierPenalty = 0 / 3 = 0.00.

[0151] Based on this, we can further obtain: ResultStability(j) = 1 - 0.009 × (1 + 0.00) = 0.991. It is evident that since ResultStability(j) is 0.991, the detection results are highly stable and have high reliability.

[0152] Step B3: Utilize S different methods to verify the same capability of the j-th candidate large model, and calculate the cross-validation consistency of the model's output results for different methods, denoted as CrossValidation(j), which is used to verify the consistency of the same capability using different methods. Here, S is a positive integer greater than 1.

[0153] CrossValidation(j) demonstrates the ability to validate a model using multiple different methods and calculates the consistency of the results from each method. The specific formula for calculating CrossValidation(j) is as follows: CrossValidation(j) = (1 / S) × Σ [z=1 to S] Agreement(method_z, ground_truth) Where S represents the number of verification methods used; the Agreement() function is used to measure the degree of agreement between the results of each verification method and the ground truth (i.e., ground_truth).

[0154] Furthermore, it should be noted that this application does not limit the specific content of the verification method, and it can be set according to the actual situation and experience. For example, the S different methods can be set as: direct testing method, indirect inference method, verification method based on user feedback, and verification method based on the declaration of the j-th candidate large model document.

[0155] The "direct testing method" refers to sending test cases to the candidate large model to check whether the model's response is correct. For example, if the j-th candidate large model (such as the first candidate large model) is sent with the 7 test cases mentioned in Table 5 above, and 6 of them are answered correctly, then Agreement = 6 / 7 = 0.857.

[0156] The "indirect inference method" refers to inferring the target capability from the performance of candidate large models in related capabilities. For example, if the j-th candidate large model (such as the first candidate large model) supports long text processing (such as a 32K context window) and multi-turn dialogue API, it can be inferred that the large model has a strong "multi-turn dialogue context understanding" capability. In this case, the test results are consistent with those of the "direct testing method" mentioned above, and the agreement value is 0.95.

[0157] Furthermore, the "user feedback-based verification method" refers to verification based on statistical data from real users in a production environment. For example, if statistical data from a production environment shows that the user satisfaction rate of the j-th candidate large model (such as the first candidate large model) in a multi-turn dialogue scenario is 88%, then the test result is consistent with the "direct testing method" mentioned above, and the agreement value is 0.90.

[0158] Furthermore, the "verification method based on the documentation statement of the j-th candidate large model" refers to verifying the capabilities by referring to the official capability statement of the provider of the j-th candidate large model (such as the first candidate large model). For example, if the official documentation of the j-th candidate large model (such as the first candidate large model) states that it supports multi-turn dialogue and the context window is 32K, then the result is consistent with the test results, and Agreement=1.00 can be obtained.

[0159] Based on this, we can obtain: CrossValidation(j) = (0.857 + 0.95 + 0.90 + 1.00) / 4 = 0.927. It is evident that since CrossValidation(j) takes a value of 0.927, the detection result is stable and has high reliability.

[0160] Step B4: By comparing the current detection result of the ability of the j-th candidate large model with the historical detection results, it is determined whether the current detection result meets the expectations, so as to determine historical consistency, which is denoted as HistoricalConsistency(j), and is used to determine the degree of consistency between the new detection result and the historical trend.

[0161] The calculation of Historical Consistency(j) reflects the comparison between the current detection results and historical trends to determine whether the results meet expectations. The specific formula for calculating Historical Consistency(j) is as follows: HistoricalConsistency(j)=1-|CurrentValue-PredictedValue| / Range Where CurrentValue represents the actual value of this detection result; PredictedValue represents the prediction result obtained by predicting based on historical data, such as the prediction obtained by using a linear regression or autoregressive integrated moving average (ARIMA) model; Range represents the range of values, which can usually be set to 0-1.

[0162] For example: Suppose that the "multi-turn dialogue context understanding" capability of the j-th candidate large model (such as the first candidate large model) is tested three times, and the historical detection records are as follows: the first detection time is January 2024, with a result of 0.85; the second detection time is February 2024, with a result of 0.87; and the third detection time is March 2024, with a result of 0.89. Then, the formula for predicting the current April 2024 using linear regression is: y = 0.83 + 0.02 × t, and the corresponding predicted value is 0.85 + 0.02 × 4 = 0.91.

[0163] If the actual value of the detection result, CurrentValue, is 0.90, then HistoricalConsistency(j) can be further calculated as 1 - |0.90 - 0.91| / 1.0 = 0.990.

[0164] As can be seen, since HistoricalConsistency(j) is 0.990, it indicates that the results of this detection are highly consistent with historical trends and there are no abnormal fluctuations. If this value is lower than 0.8, it may mean that the j-th candidate large model (such as the first candidate large model) has undergone significant changes (such as version upgrades) or that there are errors in the detection process, requiring enhanced detection.

[0165] Step B5: Calculate the business importance and detection difficulty of the capability of the j-th candidate large model; and adaptively adjust the weights using these business importance and detection difficulty to obtain the adjusted weights, denoted as AdaptiveWeight(j), which is used to dynamically adjust the weights based on the criticality of the capability (i.e., business importance) and the detection difficulty. The specific calculation formula for AdaptiveWeight(j) is as follows: AdaptiveWeight(j)=ImportanceScore(j) / DifficultyScore(j) Here, ImportanceScore(j) represents the operational importance score of the capability; DifficultyScore(j) represents the detection difficulty score of the capability.

[0166] Specifically, the formula for calculating ImportanceScore(j) is as follows: ImportanceScore(j)=w1×UsageFrequency(j)+ w2×BusinessImpact(j)+w3×UserDemand(j) Here, UsageFrequency(j) represents the frequency of capability usage, referring to the proportion of the capability used in the transition task. For example, the "multi-turn dialogue context understanding" capability is needed in almost all session transitions, so its corresponding UsageFrequency(j) value can be 0.90 (for example only).

[0167] BusinessImpact(j) represents the business impact of a capability, referring to the severity of the impact on the business when the capability fails. For example, taking a 0-1 rating as an example, a failure of contextual understanding will lead to a complete break in the conversation, so the corresponding BusinessImpact(j) value could be 0.95 (for example only).

[0168] UserDemand(j) represents the user demand for a capability, referring to the level of attention based on user surveys and feedback. For example, if users are most concerned about whether the conversation is smooth after switching, the corresponding UserDemand(j) value could be 0.88 (for example only).

[0169] w1, w2, and w3 all represent weights, and their specific values ​​are not limited. They can be set according to actual conditions and experience. For example, they can be set as: w1=0.5, w2=0.3, w3=0.2.

[0170] Therefore, we can obtain: ImportanceScore(j) = 0.5 × 0.90 + 0.3 × 0.95 + 0.2 × 0.88 = 0.911.

[0171] And, the formula for calculating DifficultyScore(j) is as follows: DifficultyScore(j)=w1×TestComplexity(j)+ w2×AmbiguityLevel(j)+w3×TemporalVariability(j) Here, TestComplexity(j) represents the testing complexity of a capability, referring to the difficulty of designing and executing the test. For example, taking a 0-1 rating as an example, a multi-turn dialogue test requires constructing a complete dialogue history, so the corresponding TestComplexity(j) value could be 0.70 (for example only).

[0172] AmbiguityLevel(j) represents the ambiguity of the ability test result, which can be understood as the clarity of judging "whether it is supported". For example, if we take a 0-1 rating as an example, the correctness of the context understanding is relatively easy to judge, so the corresponding AmbiguityLevel(j) value can be 0.35 (for example only).

[0173] TemporalVariability(j) represents the volatility of an ability over testing time, which can be understood as the coefficient of variation (CV) of the ability's historical detection results. For example, if we take a 0-1 rating as an example, multi-turn dialogue abilities are relatively stable, so the corresponding TemporalVariability(j) value can be 0.05 (for example only).

[0174] w1, w2, and w3 all represent weights, and their specific values ​​are not limited. They can be set according to the actual situation and experience. For example, they can be set as: w1=0.4, w2=0.3, w3=0.3.

[0175] Therefore, we can obtain: DifficultyScore(j) = 0.4 × 0.70 + 0.3 × 0.35 + 0.3 × 0.05 = 0.400.

[0176] Therefore, AdaptiveWeight(j) = ImportanceScore(j) / DifficultyScore(j) = 0.911 / 0.400 = 2.278. Further, by normalizing it (i.e., dividing by the maximum possible value of 3.0), we get: AdaptiveWeight_normalized = 2.278 / 0.30 = 0.759.

[0177] Step B6: Perform a weighted summation of the test coverage, output stability, cross-validation consistency, historical consistency, and adjusted weights for the j-th candidate large model, and probe the capability of the j-th candidate large model based on the calculated results. This yields the probe results for the capability of the j-th candidate large model. Similarly, the probe results for the capability of all K candidate large models can be obtained.

[0178] In this process, after obtaining the values ​​of TestCoverage(j), ResultStability(j), CrossValidation(j), HistoricalConsistency(j), and AdjustedWeight(j) of the j-th candidate large model through steps B1-B5 (the specific execution order is not limited), the values ​​of the five probe dimensions can be further weighted and summed according to the weight coefficients to obtain the probe result of the j-th candidate large model's capability (specifically reflected in the confidence score), denoted as ConfidenceScore(j). The specific calculation formula is as follows: ConfidenceScore(j)=α×TestCoverage(j)+β×ResultStability(j)+ γ×CrossValidation(j)+δ×HistoricalConsistency(j)+ε×AdaptiveWeight(j) In this context, α, β, γ, δ, and ε all represent weights, and their specific values ​​are not limited. They can be set according to actual conditions and experience. For example, they can be set as: α=0.25, β=0.25, γ=0.20, δ=0.15, ε=0.15.

[0179] For example: Continuing with the example above, when TestCoverage(j) = 0.688, ResultStability(j) = 0.991, CrossValidation(j) = 0.927, HistoricalConsistency(j) = 0.990, and AdaptiveWeight(j) = 0.759, taking α = 0.25, β = 0.25, γ = 0.20, δ = 0.15, and ε = 0.15 as an example, then ConfidenceScore(j) = 0.25 × 0.688 + 0.25 × 0.991 + 0.20 × 0.927 + 0.15 × 0.990 + 0.15 × 0.759 = 0.868.

[0180] By doing so, the capabilities of all K candidate large models can be explored.

[0181] Furthermore, in practical applications, an evaluation threshold for the detection result (specifically, the confidence score) can be preset and defined as the detection threshold. The value of this threshold is not limited; for example, it can be set to 0.6. Thus, based on different values ​​of the detection result (specifically, the confidence score), a comparison can be made with the detection threshold to determine whether to adopt the preset enhanced detection strategy and execute enhanced detection.

[0182] Specifically, such as Figure 5 As shown, if the confidence score of the ACCEM detection result of the candidate large model is not lower than the detection threshold (e.g., 0.6), it indicates that the confidence of the detection result of the capability of the j-th candidate large model is reliable.

[0183] Alternatively, if the confidence score of the ACCEM probe result of the candidate large model is lower than the probe threshold (e.g., 0.6), it indicates that an enhanced probe strategy is needed to enhance the probe of the capability of the j-th candidate large model, thereby obtaining an enhanced probe result for the capability of the j-th candidate large model. This application does not limit the specific content of the enhanced probe strategy, and it may include, but is not limited to, increasing the number of test cases, increasing the number of repeated tests, expanding the cross-validation method, and cross-validating different candidate large models.

[0184] One example of an enhanced probing strategy that "increases the number of test cases" is to increase the number of test cases from the basic 6-7 to 12-15, covering more boundary cases and extreme scenarios. For example, adding test cases for very long dialogues (more than 20 rounds), multilingual dialogues, and dialogues containing special characters.

[0185] An example of an enhanced probing strategy that “increases the number of repeated tests” could be to increase the number of times each test case is executed from 3 to 5, in order to more accurately assess the stability of the results and eliminate the interference of random fluctuations.

[0186] An example of an enhanced probing strategy for the "extended cross-validation method" is to add "adversarial testing" (verifying whether the large model can handle it correctly by deliberately providing misleading context) and "progressive testing" (locating the capability boundaries of the large model by gradually increasing the complexity of the dialogue) to the original S verification methods (such as direct testing, indirect inference, user feedback-based verification, and verification based on the declaration of the j-th candidate large model document).

[0187] An example of an enhanced probing strategy for "cross-validation of different candidate large models" is to compare the probing results of multiple candidate large models horizontally. If the score of a candidate large model in a certain capability differs significantly from that of other candidate large models (e.g., more than twice the standard deviation), then that capability of the candidate large model can be retested in a focused manner to distinguish between "the model is indeed inadequate" and "random errors exist in the probing process".

[0188] Experiments show that, based on at least one of the aforementioned enhanced probing strategies, implementing enhanced probing can improve the confidence score in low-confidence scenarios from 0.5-0.6 to 0.75-0.85. This is because more test data and more diverse validation methods can more accurately reflect the true capabilities of the large model, thereby reducing evaluation bias caused by insufficient samples or random factors. The increased confidence also means that the question-answering system has a higher degree of credibility with the probing results, thus enabling it to more accurately select the optimal large model for the target user.

[0189] Thus, this application employs a closed-loop capability probing system combining "multi-dimensional confidence assessment + adaptive probing strategy." It utilizes multi-dimensional fusion probing (such as fusion probing of five dimensions: test coverage, result stability, cross-validation consistency, historical consistency, and adaptive weighting) to precisely quantify the credibility of each probing result and dynamically adjust the probing intensity based on the confidence level. For example, it automatically increases the number of test cases and repetitions in low-confidence scenarios (<0.6), while simplifying the testing process and saving costs in high-confidence scenarios (>0.8). This reduces the false positive rate from 18% to 3%, improves probing accuracy by 35%, and increases efficiency by 40%.

[0190] S104: Based on the evaluation results, probe results, and the current load of the K candidate large models, select the optimal large model from the K candidate large models; and feed back the response results about the target session request generated using the optimal large model to the target user.

[0191] In this embodiment, after obtaining the evaluation result (such as the CTFE evaluation result, i.e., the conversion fidelity comprehensive score CTFE) and the detection result (such as the confidence score corresponding to the ACCEM detection result) through step S103, further, as follows: Figure 6 As shown, the evaluation results, detection results, and scores corresponding to the current load status of the K candidate large models can be weighted and summed to obtain the comprehensive score corresponding to each of the K candidate large models. Then, the candidate large model with the highest comprehensive score is selected as the optimal large model, and the response result of the target session request generated by the optimal large model is fed back to the target user.

[0192] The specific formulas for calculating the comprehensive score of each of the K candidate large models are as follows: FinalScore(model)=w_ctfe×CTFE(model)+w_accem×ACCEM(model)+w_load×LoadScore(model) Wherein, CTFE(model) represents the CTFE evaluation result of the corresponding candidate large model capability (i.e., the transformation fidelity comprehensive score CTFE); ACCEM(model) represents the probe confidence score of the corresponding candidate large model capability; LoadScore(model) represents the current load health score of the corresponding candidate large model (the lower the load, the higher the score); w_ctfe, w_accem, and w_load all represent weights, and their specific values ​​are not limited. They can be set according to the actual situation and experience, such as: w_ctfe=0.40, w_accem =0.40, w_load=0.20.

[0193] For example, for the first, second, and third candidate large models shown in Table 2 above, when w_ctfe = 0.40, w_accem = 0.40, and w_load = 0.20, the comprehensive scores of these three candidate large models calculated using the above formula are shown in Table 6 below: Table 6

[0194] As can be seen from Table 6 above, if K=3, and the K candidate large models are respectively the first, second, and third candidate large models shown in Table 6, then the candidate large model corresponding to the highest comprehensive score can be selected, that is, the first candidate large model corresponding to the highest comprehensive score of 0.904 can be selected as the optimal large model. Then, the response results of the first candidate large model regarding the target session request can be fed back to the target user in real time through SSE streaming push.

[0195] It should be noted that after obtaining K sets of high-quality, suitable converted session history information for the corresponding candidate large model, these K sets of converted session history information can be input into K candidate large models (such as the first candidate large model, the second candidate large model, the third candidate large model, etc.) as mentioned above. This allows the K candidate large models to generate response results. Once the optimal large model is determined, the response result generated by the optimal large model regarding the target session request is fed back to the target user. Alternatively, after determining the optimal large model, the high-quality converted session history information can be input into that optimal large model to generate a response result regarding the target session request and then fed back to the target user. The specific implementation method is not limited.

[0196] For example: Based on the above example, the first candidate large model, as the optimal large model, can generate the following response: Okay, your booking information has been confirmed. Passenger: Zhang (ID card number: 132XXXX0001) Flight: Flight No. 3, XX Airlines Date: Tomorrow (date in [year], month], and day) 11:00-13:20 Route: City A → City B Cabin class: Economy Ticket price: ¥xx Please confirm that the above information is correct, and I will submit your order. Please let me know if you need any changes.

[0197] Experiments show that the entire large model switching process takes about 800ms (of which CTFE evaluation takes about 200ms, ACCEM detection takes about 400ms, format conversion takes about 100ms, and network communication takes about 100ms). The target user will only perceive the normal model response delay and will not be aware that the large model used in their conversation has been switched from the previously used large model (such as iFlytek Spark) to the optimal large model (i.e., the first candidate large model). In other words, they will not be aware that a large model switch has occurred in the backend, and the conversation experience remains consistent, thereby improving the conversation experience of the target user.

[0198] Furthermore, after completing a session with the target user using the optimal large model, such as Figure 6 As shown, you can first update the `current_model` in the target user's session history information (e.g., update it to the first candidate large model) and append the session history content. Then, perform incremental snapshot synchronization, specifically generating incremental snapshots of the session state every 500 milliseconds. These snapshots are broadcast to all service instances via Redis's publish / subscribe (Pub / Sub) mechanism, and each instance updates its local L1 cache, achieving cross-instance state synchronization (latency <50ms). Simultaneously, log recording is invoked, specifically recording a complete log of this large model switch, including but not limited to the switch reason, candidate large model list, CTFE and ACCEM scores of each candidate large model, final selection result, first token response time, total time consumption, and other metrics. Next, the evaluation data of this large model switch can be fed back to the rule engine to optimize subsequent large model routing scheduling weights, achieving a feedback optimization closed loop. Specifically, the rule engine can dynamically adjust the priority weights of each large model based on the CTFE and ACCEM performance of each model in the historical switch records. That is, large models with consistently excellent performance can obtain higher routing priority, while large models with poor performance have their priority reduced or trigger alarms. This allows for an adaptive closed-loop processing flow of "routing → conversion → evaluation → detection → selection → feedback → route optimization".

[0199] Furthermore, embodiments of this application also provide a session processing device based on large model switching, including: a processor, a memory, and a system bus; The processor and the memory are connected via the system bus; The memory is used to store one or more programs, the one or more programs including instructions that, when executed by the processor, cause the processor to perform any of the above-described implementations of the session processing method based on large model switching.

[0200] refer to Figure 7The diagram illustrates a suitable structural schematic for implementing the large-model-based session processing device in the embodiments of this application. The large-model-based session processing device in the embodiments of this application may include, but is not limited to, terminals equipped with large models, cloud platforms, servers, and server clusters. Figure 7 The session processing device based on large model switching shown is merely an example of an electronic device and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0201] like Figure 7 As shown, the large-mode switching-based session processing device may include a processor (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 702 or a program loaded from storage device 708 into random access memory (RAM) 703. When the large-mode switching-based session processing device is powered on, RAM 703 also stores various programs and data required for device operation. The processor 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0202] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 708 including, for example, memory cards, hard drives, etc.; and communication devices 709. Communication device 709 allows the electronic device to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 A session processing apparatus based on large model switching with various devices is shown; however, it should be understood that implementation or having all of the devices shown is not required. More or fewer devices may be implemented alternatively.

[0203] Furthermore, embodiments of this application also provide a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform any of the above-described implementations of the session processing method based on large model switching.

[0204] Furthermore, this application also provides a computer program product that, when run on a terminal device, causes the terminal device to execute any of the above-described implementations of the session processing method based on large model switching.

[0205] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0206] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0207] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0208] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A session processing method based on large model switching, characterized in that, include: Obtain the target session request initiated by the target user; And based on the target session request, obtain the session history information about the target user; When the triggering condition for switching to a large model is detected, the session history information is converted into session information that meets the input format requirements of the K candidate large models after the switch, resulting in K sets of converted session history information; where K is a positive integer greater than 0. The conversion quality of the K converted session history information is evaluated from M dimensions to obtain evaluation results; and the capabilities of the K candidate large models are probed from N dimensions to obtain probe results; where M and N are both positive integers greater than 1. Based on the evaluation results, the detection results, and the current load of the K candidate large models, the optimal large model is selected from the K candidate large models; and the response result of the target session request generated using the optimal large model is fed back to the target user.

2. The method according to claim 1, characterized in that, The evaluation of the conversion quality of the K converted session history information from M dimensions yields the evaluation results, including: The conversion quality of the K converted session history information is evaluated from five dimensions: global semantics, local information, entity preservation, functional constraints, and time decay, and the evaluation results are obtained.

3. The method according to claim 1, characterized in that, The ability to probe the K candidate large models from N dimensions, and to obtain the probe results, includes: The capabilities of the K candidate large models are detected from five dimensions: test coverage, result stability, cross-validation, historical consistency, and adaptive weights, and the detection results are obtained.

4. The method according to claim 1, characterized in that, The step of obtaining session history information about the target user based on the target session request includes: The session identifier of the target user is identified from the target session request; and the complete session context information of the target user is obtained from one of the three-layer caches—local cache, distributed cache, or persistent storage database cache—based on the session identifier, as session history information.

5. The method according to claim 1, characterized in that, The step of converting the session history information into session information that conforms to the input format requirements of the K candidate large models after the switch, to obtain K sets of converted session history information, includes: The session history information is converted into session information that conforms to the input information format requirements of the K candidate large models after the switch, resulting in K session history messages with adapted message formats. Extract the generation control parameters and prompt words of the currently used large model from the session history information; map the generation control parameters of the currently used large model to the parameter space of K candidate large models to obtain K mapped control parameters; and reorganize the prompt words of the currently used large model into prompt words that conform to the prompt word format of K candidate large models to obtain K reorganized prompt words. The K sets of session history messages adapted to the message format, the K sets of mapped control parameters, and the K sets of recombined prompts are used to construct the K sets of converted session history information.

6. The method according to claim 2, characterized in that, The conversion quality of the K converted session history information is evaluated from five dimensions: global semantics, local information, entity preservation, functional constraints, and time decay. The evaluation results include: The i-th converted session history information in K converted session history information is subjected to multi-scale semantic similarity fusion processing with the original session history information to obtain the global semantic fidelity of the i-th converted session history information; where i is a positive integer greater than 0 and not greater than K; Key information fragments are extracted from the conversation history information before conversion. After scoring the importance of each key information fragment and calculating the similarity between each key information fragment and the fragment most similar to it in the i-th converted conversation history information, the importance score and similarity are weighted and summed to obtain the local key information fidelity of the i-th converted conversation history information. Entity information is extracted from the session history information before conversion and the i-th session history information after conversion, and the entity retention integrity of each type of entity is calculated by comparing the extracted entity information. Calculate the retention degree of the candidate large model generation control parameters, the retention degree of functional constraints, and the retention degree of the output format corresponding to the i-th transformed session history information, and use the retention degree of the parameters, the retention degree of functional constraints, and the retention degree of the output format to calculate the retention degree of functional constraints of the i-th transformed session history information. Based on the time dimension corresponding to the session history information before conversion, calculate the time decay weighted similarity of the i-th converted session history information; The global semantic fidelity, local key information fidelity, entity retention, functional constraint preservation, and time decay weighted similarity of the i-th converted session history information are weighted and summed. The conversion quality of the i-th converted session history information is evaluated based on the calculation results, and the evaluation result for the i-th converted session history information is obtained. Similarly, the evaluation results for all K converted session history information can be obtained.

7. The method according to claim 6, characterized in that, The step of evaluating the conversion quality of the i-th converted session history information based on the obtained calculation results, and obtaining the evaluation result for the i-th converted session history information, includes: When the calculated result is lower than the first evaluation threshold and not lower than the second evaluation threshold, the lowest value among the global semantic fidelity, local key information fidelity, entity retention integrity, functional constraint retention, and time decay weighted similarity is identified, and the value of the evaluation dimension corresponding to the lowest value is optimized in a targeted manner, so as to redetermine the evaluation result of the i-th converted session history information using the optimized evaluation dimension value. Alternatively, when the calculated result is lower than the second evaluation threshold, invalid values ​​among the global semantic fidelity, local key information fidelity, entity retention integrity, functional constraint retention, and time decay weighted similarity are identified, and the values ​​of the evaluation dimensions corresponding to the invalid values ​​are recalculated so as to determine the evaluation result of the i-th converted session history information again using the recalculated evaluation dimension values.

8. The method according to claim 3, characterized in that, The ability of the K candidate large models is detected from five dimensions: test coverage, result stability, cross-validation, historical consistency, and adaptive weights. The detection results include: The capability of the j-th candidate large model among K candidate large models is tested using P test cases, and the test coverage of the P test cases is calculated; where P is a positive integer greater than 0; and j is a positive integer greater than 0 and not greater than K. The j-th candidate large model is tested Q times using each test case, and the stability of the model output is determined; where Q is a positive integer greater than 1. Using S different methods, the common ability of the j-th candidate large model is verified, and the cross-validation consistency of the model's output results for different methods is calculated; where S is a positive integer greater than 1. By comparing the current detection results for the capability of the j-th candidate large model with the historical detection results, it is determined whether the current detection results meet expectations, thus determining historical consistency. Calculate the business importance and detection difficulty of the j-th candidate large model; and adaptively adjust the weights using the business importance and detection difficulty to obtain the adjusted weights; The test coverage, output stability, cross-validation consistency, historical consistency, and adjusted weights of the j-th candidate large model are weighted and summed. The capability of the j-th candidate large model is then probed based on the calculated results, and the probe results for the capability of the j-th candidate large model are obtained. Similarly, the probe results for the capability of all K candidate large models can be obtained.

9. The method according to claim 8, characterized in that, The S different methods include direct testing, indirect inference, verification based on user feedback, and verification based on the declaration of the j-th candidate large model document.

10. The method according to claim 8, characterized in that, The step of probing the capability of the j-th candidate large model based on the obtained calculation results, and obtaining the probing results for the capability of the j-th candidate large model, includes: When the obtained calculation result is lower than the detection threshold, an enhanced detection strategy is used to enhance the detection of the capability of the j-th candidate large model, and an enhanced detection result for the capability of the j-th candidate large model is obtained; the enhanced detection strategy includes at least one of the following strategies: increasing the number of test cases, increasing the number of repeated tests, expanding the cross-validation method, and cross-validation of different candidate large models.

11. The method according to any one of claims 1-10, characterized in that, The step of selecting the optimal large model from the K candidate large models based on the evaluation results, the detection results, and the current load status of the K candidate large models includes: The evaluation results, the detection results, and the scores corresponding to the current load status of the K candidate large models are weighted and summed to obtain the comprehensive score corresponding to each of the K candidate large models. The candidate large model with the highest comprehensive score is selected as the optimal large model.

12. The method according to any one of claims 1-10, characterized in that, The triggering conditions for the large model switching include one or more of the following: The current large model's service instance load is higher than a preset threshold; the current large model's capabilities in the domain of the target session request are insufficient; the cost of the current large model is higher than the cost of the large model required to respond to the target session request; the current large model needs to be updated to a new version; or the current large model is experiencing a service failure.

13. The method according to any one of claims 1-10, characterized in that, The method for generating the response result for the target session request using the optimal large model is as follows: Using the converted session history information that meets the input format requirements of the optimal large model, a session request instruction is generated, which is then input into the optimal large model for response, resulting in a response to the target session request.

14. A session processing device based on large model switching, characterized in that, include: Processor, memory, system bus; The processor and the memory are connected via the system bus; The memory is used to store one or more programs, the one or more programs including instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1-13.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a terminal device, cause the terminal device to perform the method described in any one of claims 1-13.

16. A computer program product, comprising a computer program, characterized in that, The program product includes a computer program that, when executed by an electronic product, can implement the method described in any one of claims 1-13.

Images