Intelligent question answering system based on large-scale pre-training model and optimization method thereof

Abstract

The application discloses an intelligent question and answer system based on a large-scale pre-training model and an optimization method thereof, and particularly relates to the technical field of artificial intelligence question and answer, and comprises the following steps: constructing a model resource pool by presetting a plurality of large-scale pre-training models, receiving a user question and answer request, and generating an answer by a current model; according to answer confidence information, extracting statistical features to determine whether to trigger double adaptability evaluation; evaluating the adaptability degree and interactive adaptability degree of the model answer, and dividing the current model into a high or low adaptability type; if it is a low adaptability type, then an optimized model is selected from the model resource pool to realize dynamic switching according to the evaluation result; through double evaluation of the answer adaptability index and the interactive adaptability index, the application can find problems of a decrease in the quality of the model answer or interaction abnormalities in real time, timely switch the model, guarantee the stability and reliability of the question and answer system, and effectively reduce the risk of model hallucination, irrelevant answers or non-compliant answers.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence question answering technology, and more specifically, to an intelligent question answering system based on a large-scale pre-trained model and its optimization method. Background Technology

[0002] With the continuous advancement of artificial intelligence technology, natural language processing systems based on large-scale pre-trained models have been widely applied in various scenarios such as intelligent question answering, virtual assistants, and intelligent customer service. Large-scale pre-trained models rely on massive training corpora and complex deep learning architectures, achieving significant results in language understanding and text generation. However, most existing intelligent question answering systems use a single pre-trained model to provide question answering services. While they can achieve a certain level of language understanding and information retrieval, they are clearly insufficient in handling complex, diverse, and dynamically changing user needs.

[0003] Single models often have specific training biases or domain limitations, making it difficult to comprehensively cover cross-domain, multi-task, and multi-type question-answering requests. When user needs exceed the model's expertise, issues such as answer bias, insufficient accuracy, and response latency can easily arise, even leading to a degraded user experience. Secondly, existing systems lack real-time evaluation and intelligent scheduling capabilities for model application status. They cannot dynamically adjust model selection based on the model's current adaptability, interaction performance, and user feedback, potentially impacting service quality during prolonged interactions due to model incompatibility.

[0004] Most existing intelligent question-answering systems still rely on manual adjustments or static configurations for model optimization, making it difficult to achieve adaptive optimization based on user behavior feedback and interaction data. For question-answering needs in different scenarios, the systems lack flexible and efficient model switching and dynamic optimization mechanisms, making it difficult to ensure continuous and efficient operation in changing environments. Therefore, this invention presents an intelligent question-answering system based on a large-scale pre-trained model and its optimization method to address the aforementioned problems. Summary of the Invention

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] The intelligent question answering optimization method based on large-scale pre-trained models includes the following steps:

[0007] Multiple large-scale pre-trained models are preset to form a model resource pool. Each pre-trained model is used for different application scenarios. It receives the user's first input of the question and answer request information and generates the answer by the large-scale pre-trained model of the current application.

[0008] Based on the answer confidence information of the large-scale pre-trained model output by the current application, within a specified question-and-answer round window, the statistical features of the confidence are extracted and it is determined whether the dual adaptability evaluation mechanism is triggered.

[0009] When the dual fitness assessment mechanism is triggered, the current large-scale pre-trained model is assessed for both the model response fitness and the model interaction fitness. Based on the dual assessment results, the current large-scale pre-trained model is classified into a highly fit type or a low-fit type.

[0010] When the partitioning result is a low-fit type, an optimized model is selected from the model resource pool based on the current large-scale pre-trained model and the dual evaluation results for switching.

[0011] In a preferred embodiment, when receiving the user's initial question-and-answer request information, it is converted into a request vector. At the same time, each large-scale pre-trained model in the model resource pool is converted into an answer vector. The answer vector with the highest similarity to the request vector is selected, and the corresponding large-scale pre-trained model is used to provide the initial answer.

[0012] In a preferred embodiment, the statistical characteristics of confidence level refer to:

[0013] Within the specified question-and-answer round window, obtain the confidence information of the answers for each round, and then calculate the confidence entropy value and the confidence skewness value respectively.

[0014] In a preferred embodiment, the logic for triggering the dual adaptability evaluation mechanism is as follows:

[0015] The confidence entropy value is compared with a preset standard disorder threshold, and the confidence skewness value is compared with a standard threshold range consisting of preset negative and positive thresholds. If the confidence entropy value is less than or equal to the preset standard disorder threshold and the confidence skewness value falls within the standard threshold range, the dual adaptability assessment mechanism is not triggered. If the confidence entropy value is not less than or equal to the preset standard disorder threshold and the confidence skewness value does not fall within the standard threshold range, the dual adaptability assessment mechanism is triggered.

[0016] In a preferred embodiment, evaluating the model response fit and the model interaction fit refers to:

[0017] The model response fit of the current large-scale pre-trained model is evaluated, and a model response fit index is generated; the model interaction fit of the current large-scale pre-trained model is evaluated, and a model interaction fit index is generated.

[0018] In a preferred embodiment, the logic for obtaining the model's response fit index is as follows:

[0019] Get the average confidence score of the answers within a specified question-and-answer round window and record it as . Answer the confidence entropy value and record it as . The average cosine similarity between the response vector and the historical dialogue summary vector in each round is used as the consistency score and denoted as . Implicit feedback based on user behavior is mapped to the average score of user feedback in the range [0,1] and denoted as . ;

[0020] To emphasize the interaction and coupling between indicators, an interaction term between the confidence factor and the entropy factor is designed. :

[0021] ; This is an adjustment coefficient used to strengthen the influence of confidence level, with a value range of [1,3]. The preset entropy sensitivity coefficient, The preset entropy inflection point threshold controls the occurrence of entropy reversal inhibition. This is the preset maximum entropy value used for normalization;

[0022] Design consistency and feedback factor interaction terms :

[0023] ; This is a preset amplification factor used to control the amplification factor of the consistency score. Preset user feedback influencing factors;

[0024] Design dynamic penalty items :

[0025] ; and Both are preset fluctuation sensitivity coefficients, and their sum is one. The standard deviation of the confidence level fluctuation. This represents the average confidence decline rate of responses from adjacent rounds.

[0026] The formula for calculating the model response fit index is:

[0027] ; The model is given a fit index.

[0028] In a preferred embodiment, the logic for obtaining the model interaction adaptation index is as follows:

[0029] Within a specified question-and-answer round window, the total number of model inputs and outputs is divided by the window duration to obtain the interaction activity level. The number of times historical information was cited in each round of responses was obtained and the average value was calculated to obtain the context preservation degree. The change fitness is calculated by obtaining the ratio of the number of topic changes in all rounds to the total number of inputs. The interaction responsiveness is obtained by calculating the average time taken for each round of responses. The formula for calculating the model interaction adaptation index is:

[0030] ;

[0031] This is a preset interaction activity adjustment factor. The preset gating steepness adjustment factor, The preset adaptive gating threshold, This is the preset power-law adjustment factor. The preset response time adjustment factor, This is the overall gain adjustment factor. The interaction complexity term is calculated using the following formula: ; This is the model interaction adaptation index.

[0032] In a preferred embodiment, the overall fit of the current large-scale pre-trained model is analyzed based on the model response fit index and the model interaction fit index using a pre-trained convolutional neural network model, and it is classified into a highly fit type or a low-fit type.

[0033] In a preferred embodiment, when the classification result is a low-fit type, the ID of the current large-scale pre-trained model, the model response fit index, and the model interaction fit index are obtained and used together as input variables of fuzzy logic. The IDs of the other models in the model resource pool are used as output variables of fuzzy logic. The input variables are fuzzified and their values ​​are converted into fuzzy sets. The output variables are also fuzzified and their values ​​are converted into fuzzy sets. Fuzzy rules are formulated to describe the fit degree of the other models in the model resource pool under different data type combinations. The fuzzified input variables are inferred through fuzzy rules to obtain the ID of the optimized model in the model resource pool.

[0034] In a preferred embodiment, an intelligent question-answering system based on a large-scale pre-trained model includes:

[0035] The model resource pool stores a variety of large-scale pre-trained models;

[0036] The confidence monitoring module monitors the confidence information of the responses output by the large-scale pre-trained models of the current application.

[0037] The interaction anomaly detection module extracts the statistical features of confidence scores within a specified question-and-answer round window and determines whether the dual adaptability assessment mechanism is triggered.

[0038] The adaptability assessment module is used to assess the adaptability of the model's response and the adaptability of the model's interaction when the dual adaptability assessment mechanism is triggered.

[0039] The type classification module classifies the current large-scale pre-trained models into highly adapted or poorly adapted types based on the dual evaluation results.

[0040] The optimization switching module, when the classification result is a low-fit type, searches for an optimized model from the model resource pool for switching based on the current large-scale pre-trained model and the dual evaluation results;

[0041] The context migration module is used to pass the user's dialogue context when switching models, enabling seamless migration;

[0042] The question-answering execution module is used to perform intelligent question-answering interaction based on the current model.

[0043] The technical effects and advantages of this invention are as follows:

[0044] This invention pre-trains multiple large-scale models in a model resource pool, with each model trained specifically for different domains and task scenarios, enabling dynamic switching between specialized and general-purpose models. When a user's request involves specific industry knowledge, the system can intelligently identify it and have a specialized model provide the initial response, improving the accuracy and professionalism of the answer and avoiding the lack of professionalism caused by the breadth of knowledge in general-purpose models. If the suitability of the specialized model decreases due to user topic changes or other reasons, the system can be optimized to provide a broader knowledge coverage for the general-purpose model to provide the response.

[0045] This invention employs a dual adaptability evaluation mechanism to dynamically assess the quality of model responses and interactive performance, enabling real-time monitoring of the model's application status. Based on the evaluation results, the system automatically identifies the current model's adaptability and, combined with a fuzzy logic inference system, switches models, enhancing the system's adaptability to complex and ever-changing user needs and ensuring efficient responses in different user scenarios.

[0046] This invention utilizes a dynamic evaluation and intelligent scheduling mechanism to call upon different large-scale pre-trained models on demand, avoiding resource waste caused by prolonged high-load operation of a single model. Specialized models are used in specialized scenarios, while general-purpose models are used for general needs, achieving reasonable allocation and scheduling of model resources, reducing computational costs and energy consumption, and improving the overall system operating efficiency.

[0047] This invention uses a dual evaluation of the answer fit index and the interaction fit index to detect problems such as declining model answer quality or abnormal interaction in real time, and promptly switches to a better model, thus ensuring the stability and reliability of the question-answering system.

[0048] This invention can automatically identify model output risks and switch to an optimized model through dynamic adaptation evaluation, effectively reducing the risks of model illusion, irrelevant answers, or non-compliant responses, and improving the overall reliability and business compliance of the question-and-answer system. Through a fuzzy logic reasoning system, complex model adaptability evaluation data is intelligently fused with resource pool model status information, achieving automated and intelligent model switching decisions. The system requires no manual intervention or configuration, improving its intelligence level and reducing maintenance complexity. Attached Figure Description

[0049] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings;

[0050] Figure 1 This is a schematic diagram of the intelligent question answering optimization method based on a large-scale pre-trained model in this invention.

[0051] Figure 2 This is a schematic diagram of the intelligent question-answering system based on a large-scale pre-trained model in this invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] Reference Figure 1 - Figure 2 The following examples were obtained:

[0054] Example 1:

[0055] With the rapid development of artificial intelligence technology, large-scale pre-trained models have achieved significant breakthroughs in the field of natural language processing and are widely used in intelligent question-answering systems. Existing large-scale pre-trained models, trained through deep learning and big data, possess strong language understanding and generation capabilities. However, current intelligent question-answering systems generally rely on a single model to perform question-answering tasks, and the problems of insufficient model adaptability and poor flexibility are becoming increasingly prominent.

[0056] In practical applications, the question-and-answer needs of different users are complex and varied, involving knowledge backgrounds from multiple domains and across different scenarios. A single large-scale pre-trained model, due to limitations in training data and model structure constraints, struggles to comprehensively cover the knowledge systems of different application scenarios. When a model handles tasks beyond its domain of expertise, the accuracy, reliability, and interactive experience of its answers often cannot be effectively guaranteed, and situations such as model illusion or irrelevant answers may even occur. Furthermore, most existing systems lack real-time dynamic evaluation and optimization mechanisms for model adaptability, resulting in low model resource utilization and an inability to adaptively optimize overall system performance.

[0057] Current technologies also suffer from insufficient evaluation of model adaptability and interactivity, lack effective evaluation metrics and switching mechanisms, and are unable to dynamically determine and adjust the usage status of large-scale pre-trained models, further limiting the stability and accuracy of intelligent question answering systems.

[0058] This invention aims to solve the technical problems of existing intelligent question answering systems, such as the single large-scale pre-trained model, insufficient adaptability, and lack of dynamic optimization scheduling mechanism, and to improve the flexibility and intelligence level of intelligent question answering systems in multiple scenarios.

[0059] The intelligent question-answering optimization method based on large-scale pre-trained models proposed in this invention involves pre-setting multiple large-scale pre-trained models to form a model resource pool, and selecting the optimal model to perform question-answering tasks for different application scenarios. When receiving the user's initial question-answering request information, the system uses a similarity matching mechanism between the request vector and the answer vectors of each large-scale pre-trained model in the model resource pool to achieve the optimal selection of the initial answering model, thereby improving the accuracy and efficiency of the initial question-answering.

[0060] This invention further introduces statistical feature analysis based on response confidence information, employing a joint determination of confidence entropy and confidence skewness values ​​to dynamically determine whether a dual fitness assessment mechanism needs to be triggered. Under the dual fitness assessment mechanism, the current model is evaluated for both model response fitness and model interaction fitness, generating model response fitness index and model interaction fitness index respectively, providing refined and comprehensive data support for model fitness analysis.

[0061] By comprehensively analyzing the model's answer fit index and interaction fit index using pre-trained convolutional neural network models, the overall fit of the current large-scale pre-trained models is determined, dynamically classifying them into highly fit or poorly fit types. For poorly fit models, the system uses a fuzzy logic reasoning mechanism, combining the current model number, dual evaluation results, and model resource pool information, to accurately reason and switch to the optimal model, achieving seamless and dynamic optimization switching, thus improving the overall system response efficiency and interactive experience. This invention, through a dynamic evaluation mechanism and intelligent switching method, significantly improves the adaptability, interactivity, and self-optimization capabilities of large-scale pre-trained models in intelligent question-answering systems, enhancing the system's comprehensive performance in handling multi-scenario and multi-domain question-answering needs, and possesses significant application value and promising industrial application prospects.

[0062] Specifically, this invention proposes an intelligent question-answering optimization method based on a large-scale pre-trained model, comprising the following steps:

[0063] Multiple large-scale pre-trained models are pre-configured to form a model resource pool. Each pre-trained model is used for different application scenarios. It receives the user's initial question-and-answer request information and generates the answer using the large-scale pre-trained model of the current application. This step builds a model resource pool by pre-loading and deploying multiple large-scale pre-trained models, configuring dedicated models for different application domains or scenarios (such as general question answering). Each model, based on its training data source and algorithm optimization strategy, forms a highly targeted and domain-adaptive intelligent question-answering capability.

[0064] Upon receiving the user's initial question-and-answer request, the system first preprocesses the request information, using Natural Language Understanding (NLU) technology to convert the user's input text into a request vector. Simultaneously, each pre-trained model in the model resource pool is transformed into an answer vector. The request vector and answer vector are compared using semantic similarity algorithms (such as cosine similarity and vector distance measure), and the optimal matching model is selected to achieve the initial question-and-answer response. The selected large-scale pre-trained model immediately generates and outputs the answer without relying on additional evaluation, ensuring the real-time and efficient user question-and-answer experience.

[0065] By pre-setting multiple models, this system breaks away from the limitations of traditional intelligent question-answering systems that rely on a single model, significantly improving the overall coverage and domain accuracy of the system's question-answering capabilities. Employing a request vector and answer vector matching mechanism enables real-time and intelligent model selection, ensuring high-quality answers during the initial interaction and enhancing user experience. This lays the foundation for subsequent dynamic model optimization and switching, improving system flexibility and scalability.

[0066] Based on the answer confidence information output by the large-scale pre-trained model of the current application, within a specified question-and-answer round window, the system extracts statistical features of the confidence level and determines whether to trigger a dual fit assessment mechanism. While the large-scale pre-trained model generates answers, the system acquires the answer confidence information output by the model. This confidence information can be obtained based on data such as the model output probability distribution, classification decision boundary, or token probability weights of the generative model. Within a specified question-and-answer round window (e.g., the most recent N rounds), the system continuously collects answer confidence information and extracts key statistical features, including but not limited to confidence entropy and confidence skewness. The system compares the confidence entropy value of the current window with a preset standard disorder threshold, and simultaneously compares the confidence skewness value with a preset positive and negative threshold range, forming a dual judgment mechanism. If the confidence entropy value is less than or equal to the standard disorder threshold, and the confidence skewness value falls within the standard threshold range, the current model is considered to have normal fit, and the assessment mechanism is not triggered; otherwise, the subsequent dual fit assessment process is triggered.

[0067] By monitoring changes in model confidence through a dynamic window, potential problems with model stability and adaptability can be captured in real time. A joint judgment method based on confidence entropy and skewness is adopted to improve the accuracy and robustness of the judgment, avoid misjudgment based on a single indicator, and construct a dynamic adaptability evaluation trigger mechanism to enable the system to have self-monitoring and adaptive optimization capabilities, thereby reducing the risk of the system misusing unsuitable models.

[0068] When the dual fitness assessment mechanism is triggered, the current large-scale pre-trained model is evaluated for both model response fitness and model interaction fitness. Based on the dual assessment results, the current large-scale pre-trained model is classified into a highly fit type or a low fit type. When the dual fitness assessment mechanism is triggered, the system evaluates the current application model for both model response fitness and model interaction fitness.

[0069] The model response fit assessment is mainly based on the model output data within a specified question-and-answer window. It comprehensively analyzes information such as the mean confidence score of the model response, the confidence entropy value, the consistency score (such as the average cosine similarity between the response vector and the historical dialogue summary vector), and the implicit feedback score of user behavior, and uses nonlinear interaction to calculate the model response fit index.

[0070] The model interaction adaptability assessment analyzes interaction activity (total number of model inputs and outputs / window duration), context preservation (average number of times answers reference historical information), change adaptability (ratio of topic changes to total inputs), and interaction responsiveness (average response time) to calculate the model interaction adaptability index. The system uses a convolutional neural network model to comprehensively analyze the model response adaptability index and the model interaction adaptability index to determine the overall adaptability of current large-scale pre-trained models, classifying them into "highly adapted" or "lowly adapted" types.

[0071] The dual evaluation mechanism comprehensively measures the model's static response capability and dynamic interactive adaptability, improving the comprehensiveness and scientific nature of model selection. It integrates convolutional neural network models to achieve intelligent classification of evaluation results, improves evaluation accuracy and the level of intelligence in model switching decisions, quickly identifies insufficient model adaptability, and ensures the stability and reliability of the system's interactive experience.

[0072] When the classification result is classified as a low-fit type, the system searches for an optimized model from the model resource pool based on the current large-scale pre-trained model and the dual evaluation results for switching. When the current large-scale pre-trained model is determined to be a low-fit type, the system uses evaluation data such as model number, model response fit index, and model interaction fit index as input variables into the fuzzy logic inference system, and combines them with the fit status numbers of other models in the model resource pool as output variables to complete the input-output fuzzification process.

[0073] The system uses preset fuzzy rules (which describe the suitability of each model under different combinations of input data and the switching strategy) to reason about the fuzzy set and obtain the number corresponding to the optimal model. Then, the system switches from the model resource pool to the optimized model and achieves seamless migration of user dialogue data and contextual information, ensuring an uninterrupted user experience.

[0074] By using a fuzzy logic reasoning model, fuzzy processing and flexible control are achieved for model optimization switching decisions, improving the flexibility and adaptability of the switching strategy. The switching process is based on dynamic evaluation data, fully considering the current model operating status and system requirements, ensuring the rationality and efficiency of the switching behavior. Seamless switching technology ensures the continuity of the user question-and-answer process and avoids the loss of dialogue context or system response interruption caused by model switching.

[0075] In this invention, the system pre-sets multiple large-scale pre-trained models based on the needs of different application scenarios, forming a diverse model resource pool. Each large-scale pre-trained model in the model resource pool has undergone targeted training and optimization, and is used to deal with user question-answering requests of different domains and types.

[0076] For example, general domain models such as the GPT series (GPT-3, GPT-4) are mainly suitable for open domain question answering, encyclopedic knowledge question answering, and everyday dialogue generation, and have large-scale knowledge graphs and strong generalization capabilities.

[0077] Healthcare-related models such as BioGPT, Med-PaLM (Google), and ChatDoctor are primarily used for medical literature retrieval, clinical diagnostic assistance, and health consultation. They can understand and generate specialized text content involving medical terminology and clinical pathways. When users input symptom descriptions, medical terminology, or health consultations, healthcare-related models are prioritized to generate professional answers.

[0078] Legal domain models such as LawGPT and ChatLaw are used for scenarios such as legal and regulatory searches, legal consultations, and contract review, and have the ability to understand legal texts, cite provisions, and analyze legal principles. When user questions involve contracts, legal consultations, or case descriptions, the system will prioritize using legal domain models to provide answers.

[0079] Financial models such as FinGPT and BloombergGPT are used for scenarios such as financial information interpretation, investment advice, and financial analysis. They have the ability to analyze financial data, generate reports, and predict market trends. When users raise questions related to the stock market, wealth management, and financial policies, they automatically match financial models to provide professional answers.

[0080] Education and learning domain models such as EduGPT and ChatEDU are used in scenarios such as intelligent education tutoring, problem explanation, and learning plan development. They can explain knowledge in multiple subjects and adapt to the educational needs of different grades. When facing users' needs for homework tutoring and course explanation, education domain models are given priority.

[0081] When receiving the user's initial question-and-answer request, the system converts it into a request vector. Simultaneously, it transforms each large-scale pre-trained model in the model resource pool into an answer vector. The system then selects the answer vector with the highest similarity to the request vector and uses the corresponding large-scale pre-trained model to provide the initial response. After the user's initial question-and-answer request, the system uses Natural Language Understanding (NLU) and intent recognition to semantically vectorize the input content, forming the request vector.

[0082] Each large-scale pre-trained model in the model resource pool has its own pre-defined answer vector, representing the model's domain coverage and answer features. By comparing the request vector and the answer vector (e.g., calculating cosine similarity), the system selects the best-matching model, ensuring that the domain-appropriate model is used in the first round of question answering, thus improving the accuracy and professionalism of the answers. For example, if a user inputs, "I've had a cold recently and I keep coughing, what medicine should I take?", the system identifies it as involving the medical and health field. After vector matching, it automatically selects ChatDoctor as the current large-scale pre-trained model to generate a professional health guidance answer. If a user inputs, "What legal regulations must a company's articles of association follow?", the system identifies it as a legal consultation question and matches it with the LawGPT model to generate relevant legal clauses and interpretations for the first question answer.

[0083] The statistical characteristics of confidence level refer to: within a specified question-and-answer round window, obtaining the confidence level information of the answers in each round, and then calculating the confidence entropy value and the confidence skewness value respectively. The logic for triggering the dual-fit assessment mechanism is as follows: comparing the confidence entropy value with a preset standard disorder threshold, and simultaneously comparing the confidence skewness value with a standard threshold range composed of preset negative and positive thresholds. If the confidence entropy value is less than or equal to the preset standard disorder threshold and the confidence skewness value falls within the standard threshold range, the dual-fit assessment mechanism is not triggered. If the confidence entropy value is not less than or equal to the preset standard disorder threshold and the confidence skewness value does not fall within the standard threshold range, then the dual-fit assessment mechanism is triggered.

[0084] This invention extracts the confidence information of the model's output responses within a specified question-and-answer round window, and calculates the confidence entropy and confidence skewness values ​​respectively, thereby achieving a dynamic evaluation of the stability and confidence distribution of responses from a large-scale pre-trained model. The confidence entropy value reflects the degree of disorder in the model's output results, effectively measuring the uncertainty of the model's responses within the current interaction window; the confidence skewness value reveals the skewness of the confidence distribution, identifying the model's response bias and consistency level. Through this dual-indicator analysis, the system is provided with fine-grained model state awareness, enhancing the real-time early warning capability for potential model failure risks.

[0085] By jointly comparing the confidence entropy value with a preset standard disorder threshold and the confidence skewness value with the standard threshold range, a dynamic monitoring and triggering mechanism is constructed in the intelligent question-answering system. When the confidence feature of the model output is detected to not meet the preset stability requirements, the system automatically triggers a dual adaptability evaluation mechanism to comprehensively assess the current model's answer adaptability and interaction adaptability. This intelligent judgment logic based on statistical features effectively avoids the model being in a low-adaptability state for a long time, improves the intelligence level of system model switching and optimization, and ensures the stability of the question-answering process and the consistency of the user interaction experience.

[0086] The assessment of model response fit and model interaction fit refers to:

[0087] The model response fit of the current large-scale pre-trained model is evaluated, and a model response fit index is generated; the model interaction fit of the current large-scale pre-trained model is evaluated, and a model interaction fit index is generated.

[0088] The logic for obtaining the model's response fit index is as follows:

[0089] Get the mean confidence score of the answers within the specified question-and-answer round window and record it as . This reflects the average confidence level of the model's answers; the answer confidence entropy value is denoted as... The confidence distribution of the model's responses is measured to be somewhat disordered; the average cosine similarity between the response vector and the historical dialogue summary vector in each round is used as the consistency score and denoted as... This measures whether the response is consistent with the historical context; implicit feedback based on user behavior is mapped to the average score of user feedback in the range [0,1] and denoted as... This reflects the implicit satisfaction or acceptance of users with the model's answers. It can be obtained by assigning values ​​to various user behavior indicators and then calculating the average value.

[0090] To emphasize the interaction and coupling between indicators, an interaction term between the confidence factor and the entropy factor is designed. :

[0091] ; This is an adjustment coefficient used to strengthen the influence of confidence level. Its value ranges from [1,3]. The larger the value, the stronger the influence of confidence level. The preset entropy sensitivity coefficient determines the strength of the influence of the confidence entropy value on the denominator exponential function; the larger the value, the more sensitive the response to entropy changes. The preset entropy inflection point threshold determines when the entropy value begins to have a significant inhibitory effect, controlling the timing of model fitness suppression when the entropy value is high. This is the preset maximum entropy value used for normalization; To control for the enhancement effect of high-confidence responses, the model is encouraged to positively evaluate high-confidence answers. The denominator uses a sigmoid function to dynamically adjust the entropy value's inhibitory effect on the overall score. Higher entropy indicates a more dispersed and uncertain distribution of responses, leading to decreased fit. and By flexibly setting the influence range and intensity of entropy values, the model's fit is significantly reduced when the entropy value is too large, and the "illusion" or low-confidence responses that the model may produce are reasonably suppressed.

[0092] Design consistency and feedback factor interaction terms :

[0093] ; This is a preset amplification factor used to control the amplification factor of the consistency score, adjust the gain strength of the consistency score in the overall score, and control the influence of consistency on model fit. The influence factor of user feedback is preset, and the weight of the average score of user feedback on the overall score is adjusted to control the moderating effect of user feedback on model fitness. To amplify the impact of consistency on model evaluation and ensure that the model's responses have good contextual coherence, we introduce... And by The intensity of its influence is controlled to reflect the crucial role of user behavior feedback in model evaluation. A logarithmic function is used to balance the overall output value, avoiding evaluation "overflow" caused by consistency or excessively high feedback scores, thus ensuring the stability and rationality of the system scoring.

[0094] Design dynamic penalty items This is used to illustrate the negative impact of model stability issues:

[0095] ; and Both are preset fluctuation sensitivity coefficients, and their sum is one. adjust The contribution of the dynamic penalty term determines the degree of impact of volatility on the overall fit index. adjust The contribution of the dynamic penalty term determines the degree to which a sudden drop in confidence suppresses the fit index. The standard deviation of confidence level measures the stability of the model's confidence level; a larger value indicates greater fluctuation in the response. The average confidence decline rate of responses in adjacent rounds measures the continuity of model confidence. A large value indicates that the model performance is unstable and prone to sudden drops in confidence. The dynamic penalty term directly reflects the volatility and continuity of the model's response confidence.

[0096] The combined effect of volatility and decline rate: Simultaneously examining the volatility of confidence levels and the average decline rate, a dual penalty is applied to the model's "jitter" and "collapse" performance. Through... and By controlling the ratio of the two factors, the negative impact on the system score can be flexibly adjusted to prevent system risks caused by poor model stability.

[0097] The formula for calculating the model response fit index is:

[0098] ; The MAAI (Model-Ability-Adaptation Index) is used to measure the model's response fit. A higher MAAI indicates that the current large-scale pre-trained model performs well in terms of response confidence, confidence distribution stability, response consistency, and user feedback satisfaction. It also demonstrates high stability, good response continuity, and no significant confidence fluctuations or sudden drops. The model's current response capabilities are highly aligned with the interaction requirements, exhibiting good question understanding and contextual continuity. User feedback is positive, with implicit or explicit user behavior showing high satisfaction and smooth interaction. The system can continue to maintain the current model for handling question-and-answer requests without needing to switch or optimize.

[0099] The logic for obtaining the model interaction adaptation index is as follows:

[0100] Within a specified question-and-answer round window, the total number of model inputs and outputs is divided by the window duration to obtain the interaction activity level. The number of times historical information was cited in each round of responses was obtained and the average value was calculated to obtain the context preservation degree. The change fitness is calculated by obtaining the ratio of the number of topic changes in all rounds to the total number of inputs. The interaction responsiveness is obtained by calculating the average time taken for each round of responses. The formula for calculating the model interaction adaptation index is:

[0101] ;

[0102] This is a preset interaction activity adjustment factor. The preset gating steepness adjustment factor, The preset adaptive gating threshold, This is the preset power-law adjustment factor. The preset response time adjustment factor, This is the overall gain adjustment factor. The interaction complexity term is calculated using the following formula: ; The Model Interaction Adaptation Index is a formula for calculating the Model Interaction Adaptation Index. Its structure consists of nested fractions, power-law adjustment, and logarithmic gain, reflecting the comprehensive evaluation and dynamic control capabilities of multi-dimensional indicators. It integrates key indicators such as Interaction Activity (IA), Context Preservation (CR), Adaptability to Change (AD), and Interaction Response (RS) in a non-linear manner, measuring the overall interaction performance of the model while also achieving dynamic adjustment and balance.

[0103] In the first part, the power-law adjustment of IA (Interactive Intelligence) enhances the contribution of interactive activity to the model's interactive adaptability, indicating that frequent user-model interaction is one of the indicators of a well-functioning model. The square root of CR (Contextual Retention) smooths out context preservation, preventing excessive fluctuations from affecting the overall score and ensuring the model has continuous contextual understanding and responsiveness. The exponential amplification of AD (Adaptive Attitude) addresses the fact that change adaptability determines the model's responsiveness to topic changes and shifts in user intent; the exponential function rapidly enhances sensitivity to dialogue changes. The Sigmoid gating mechanism (denominator part) further contributes to this. : Controlling the model's score suppression or activation at different stages of fitness variation, when AD exceeds the adaptive gating threshold. At this time, the suppression effect on the denominator is significantly weakened, encouraging models with good adaptability to obtain higher scores. Overall power-law adjustment: controls the weight of the overall score, allowing the system to flexibly adjust the intensity of adaptive evaluation in different scenarios.

[0104] In the second part, the comprehensive interaction complexity item measures overall interaction complexity and system load, integrating three core indicators: activity level, context preservation, and adaptability, thus improving the comprehensiveness and accuracy of the evaluation. The interaction responsiveness control item... Shorter response time, smaller RS, smaller denominator, and greater overall gain reflect the model's advantage of fast response. Logarithmic function smooth growth avoids exponential increases in scores, maintaining stable score growth and reflecting the improved adaptability resulting from optimized model interaction performance.

[0105] Adjustment Mechanism: Different adjustment parameters, such as interaction activity adjustment factors and gating steepness adjustment factors, respectively adjust the sensitivity, gating threshold, and score gain of different sub-items and the overall score, making the model evaluation adjustable, flexible, and adaptable to different types of question-answering systems and business scenario requirements. A higher MIAI indicates that the current model has higher interaction adaptability in the application scenario, stronger long-term interaction stability, adaptability to conversational contexts, and delivers a good user experience. This can prevent user churn or conversation interruption due to insufficient model adaptability, improving the overall service quality of the system. The evaluation results can be directly used for dynamic decision-making, determining whether the current model should continue to maintain interaction or whether to switch to a better model.

[0106] By analyzing the overall fit of the current large-scale pre-trained convolutional neural network model based on the model response fit index and the model interaction fit index, the model is classified into high-fit or low-fit types.

[0107] This system intelligently analyzes the overall fit of current large-scale pre-trained models using a pre-trained convolutional neural network (CNN) model. The CNN model uses the model response fit index and the model interaction fit index as input features. Through multi-layer convolution and feature extraction operations, it automatically identifies and analyzes the nonlinear relationships and coupling characteristics between different indices. Local features are extracted through convolutional layers, and pooling layers are used to reduce feature dimensionality. Further feature fusion and comprehensive evaluation are achieved in fully connected layers, ultimately outputting the comprehensive fit judgment result of the current large-scale pre-trained models.

[0108] Based on the evaluation results of the convolutional neural network models, the system classifies the current large-scale pre-trained models into highly adapted or poorly adapted types. Highly adapted models indicate that the current model meets the system's application requirements in terms of answering ability, interaction performance, and overall stability, and is suitable for continuing question-and-answer interaction. Poorly adapted models indicate that the current model has insufficient adaptability and needs to be optimized and switched based on the evaluation results. This method achieves accurate classification and dynamic optimization of model adaptability, improving the overall question-and-answer quality and user interaction experience of the system.

[0109] When the classification result is a low-fit type, the ID of the current large-scale pre-trained model, the model response fit index, and the model interaction fit index are obtained and used together as input variables of fuzzy logic. The IDs of the other models in the model resource pool are used as output variables of fuzzy logic. The input variables are fuzzified and their values ​​are converted into fuzzy sets. The output variables are also fuzzified and their values ​​are converted into fuzzy sets. Fuzzy rules are formulated to describe the fit degree of the other models in the model resource pool under different data type combinations. The fuzzified input variables are inferred through fuzzy rules to obtain the ID of the optimized model in the model resource pool.

[0110] In this invention, the input variables include the ID of the current large-scale pre-trained model, the model response fit index, and the model interaction fit index. The system first needs to fuzzify these input values ​​for subsequent fuzzy inference processing. Fuzzification converts precise numerical values ​​into linguistic values ​​(such as "high," "medium," and "low"), and defines a corresponding membership function for each linguistic value. The membership function is used to determine the degree to which an input value belongs to a certain linguistic value. Commonly used membership functions include trigonometric functions, trapezoidal functions, and Gaussian functions. This invention preferably uses trigonometric membership functions because of their simple structure, high computational efficiency, and suitability for practical applications.

[0111] Example of triangular membership function design: Taking the model response fit index as an example, it is divided into three linguistic values: "low," "medium," and "high." The triangular membership function can be defined as follows: The peak of the "low" membership function is located at 20%, and the base range is 0 to 40%. The peak of the "medium" membership function is located at 50%, and the base range is 20% to 80%. The peak of the "high" membership function is located at 80%, and the base range is 50% to 100%. Through the membership function, the system can determine the membership degree of a certain actual input value under different linguistic values. For example, if the model response fit index is 55%, then the membership degree under the "medium" linguistic value is 0.8, and the membership degree under the "high" linguistic value is 0.2.

[0112] After the input variables are fuzzified, they form fuzzy sets, which serve as the input conditions for the subsequent fuzzy inference engine. The output variables are the association information between the candidate model numbers and their suitability levels in the model resource pool. These numbers also need to be converted into fuzzy sets using membership functions; commonly used linguistic values ​​include "preferred," "considerable," and "not recommended." For each model, a membership degree is pre-set based on its historical performance, domain adaptability, and interactive responsiveness. For example, if a model has performed well in multiple historical applications, the system assigns it a high membership degree of "preferred."

[0113] Fuzzy rules embody the system's experiential knowledge and decision-making logic. For example: Rule 1: "If the model's response fit index is low and the model's interaction fit index is low, switch to the preferred recommendation model." Rule 2: "If the model's response fit index is medium and the model's interaction fit index is high, maintain the current model or consider prioritizing the recommended model." Rule 3: "If the model's response fit index is high and the model's interaction fit index is high, maintain the current model." The rule base can be expanded and optimized according to actual application needs, and the system's reasoning ability can be improved through continuous training.

[0114] The fuzzy inference engine comprehensively judges the input fuzzy set based on a fuzzy rule base, employing commonly used max-min inference or weighted average methods. In this invention, max-min inference is preferred. Its basic principle is to determine the rule activation strength based on the degree of matching between the input conditions and the rule premises, and then generate an output fuzzy set based on the rule's conclusion. The system synthesizes the output results of all activated rules to obtain a fuzzy set of output variables, representing the fitness level and recommendation degree of all candidate models.

[0115] Defuzzification transforms the output fuzzy set into specific decision values, determining which optimization model to switch to. Common methods include the centroid method and the maximum membership method. This invention prefers the maximum membership method, selecting the candidate model with the highest membership degree as the final output. For example, if "Model 3" has the highest membership degree in the "preferred recommendation" set, the system ultimately selects "Model 3" as the current optimization model, completing the model switching operation.

[0116] Example 1: Low response fit and low interaction fit, recommend a high fit model:

[0117] Scenario Description: The current large-scale pre-trained model is designated "Model 1". The system detects that the model's response fit index is 35%, and its interaction fit index is 40%. According to the preset membership function: a response fit index of 35% corresponds to a "low" level with a membership degree of 0.8. An interaction fit index of 40% also corresponds to a "low" level with a membership degree of 0.7. The input variables are fuzzified to form a "low-low" state. The system searches for rules that meet the input conditions: if both the model's response fit index and interaction fit index are low, it switches to the "preferred recommendation" model. The model resource pool contains four models: "Model 2", "Model 3", "Model 4", and "Model 5". "Model 3" has a membership degree of 0.9 in the "preferred recommendation" state, while the membership degrees of the other models are all below 0.5. Based on the maximum membership degree method, "Model 3" with the highest membership degree is selected. The system ultimately selects "Model 3" as the optimized model number, completing the switch.

[0118] Example 2: For medium-level response adaptation and low-interaction adaptation, a comprehensive balance model is recommended:

[0119] Scenario Description: The current large-scale pre-trained model is designated "Model 4". The system detects that the model's response fit index is 60%, and its interaction fit index is 35%. According to the membership function mapping: a response fit index of 60% corresponds to a "medium" level with a membership degree of 0.6. An interaction fit index of 35% corresponds to a "low" level with a membership degree of 0.75. System search rule: If the model's response fit index is medium and its interaction fit index is low, then switch to either a "considerable" model or a "preferred" model. The resource pool model status is as follows: "Model 2" has a membership degree of 0.85 in the "considerable" state, and "Model 3" has a membership degree of 0.7 in the "preferred" state. According to the rule, the "considerable" model is matched first, and "Model 2" with the highest membership degree is selected. The system determines "Model 2" as the optimized model number.

[0120] Example 2: An intelligent question-answering system based on a large-scale pre-trained model, comprising:

[0121] The model resource pool stores a variety of large-scale pre-trained models;

[0122] The confidence monitoring module monitors the confidence information of the responses output by the large-scale pre-trained models of the current application.

[0123] The interaction anomaly detection module extracts the statistical features of confidence scores within a specified question-and-answer round window and determines whether the dual adaptability assessment mechanism is triggered.

[0124] The adaptability assessment module is used to assess the adaptability of the model's response and the adaptability of the model's interaction when the dual adaptability assessment mechanism is triggered.

[0125] The type classification module classifies the current large-scale pre-trained models into highly adapted or poorly adapted types based on the dual evaluation results.

[0126] The optimization switching module, when the classification result is a low-fit type, searches for an optimized model from the model resource pool for switching based on the current large-scale pre-trained model and the dual evaluation results;

[0127] The context migration module is used to pass the user's dialogue context when switching models, enabling seamless migration;

[0128] The question-answering execution module is used to perform intelligent question-answering interaction based on the current model.

[0129] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0130] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0131] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0132] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0133] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or modifications that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An intelligent question and answer optimization method based on a large-scale pre-training model, characterized in that, The method comprises the following steps: Pre-set a plurality of large-scale pre-training models to form a model resource pool, each pre-training model is used for different application scenarios, receive the first input of the user's question and answer request information, and generate an answer based on the current application of the large-scale pre-training model; Based on the answer confidence information output by the current application of the large-scale pre-training model, within a specified question and answer round window, extract the statistical characteristics of the confidence and judge whether to trigger the dual adaptability evaluation mechanism; When the dual adaptability evaluation mechanism is triggered, the model answer adaptation degree evaluation and the model interaction adaptation degree evaluation are performed on the current large-scale pre-training model, and based on the dual evaluation results, the current large-scale pre-training model is divided into a high adaptation type or a low adaptation type; When the division result is a low adaptation type, an optimized model is found from the model resource pool based on the current large-scale pre-training model and the dual evaluation results to switch; Within the specified question and answer round window, the answer confidence information of each round is obtained, and then the confidence entropy value and the confidence skewness value are calculated; The confidence entropy value is compared with the preset standard confusion degree threshold, and the confidence skewness value is compared with the standard threshold interval composed of the preset negative threshold and the positive threshold, if the confidence entropy value is less than or equal to the preset standard confusion degree threshold and the confidence skewness value falls within the standard threshold interval, the dual adaptability evaluation mechanism is not triggered, if the confidence entropy value is not less than or equal to the preset standard confusion degree threshold and the confidence skewness value falls within the standard threshold interval, the dual adaptability evaluation mechanism is triggered.

2. The intelligent question and answer optimization method based on a large-scale pre-training model according to claim 1, characterized in that, When receiving the first input of the user's question and answer request information, it is converted into a request vector, and each large-scale pre-training model in the model resource pool is converted into an answer vector, the answer vector with the highest similarity to the request vector is selected, and the corresponding large-scale pre-training model is answered for the first time.

3. The intelligent question and answer optimization method based on a large-scale pre-training model according to claim 1, characterized in that, The model answer adaptation degree evaluation and the model interaction adaptation degree evaluation refer to: The model answer adaptation degree evaluation is performed on the current large-scale pre-training model to generate a model answer adaptation index, and the model interaction adaptation degree evaluation is performed on the current large-scale pre-training model to generate a model interaction adaptation index.

4. The intelligent question and answer optimization method based on a large-scale pre-training model according to claim 3, characterized in that, The acquisition logic of the model answer adaptation index is: obtain the average value of the answer confidence in the specified question and answer turn window, and record it as obtain the entropy value of the answer confidence, and record it as the average value of the cosine similarity between the answer vector of each turn and the historical dialogue summary vector as the consistency score, and record it as the average value of the implicit feedback based on the user behavior, which is mapped to the user feedback average score in the range of [0, 1], and record it as ; To emphasize the interaction and coupling between indicators, a confidence factor and an entropy factor interaction term is designed : ; is an adjustment coefficient for strengthening the confidence influence, with a value range of [1, 3], is a preset entropy sensitivity coefficient, is a preset entropy inflection point threshold value for controlling the occurrence of entropy reversal inhibition, is a preset maximum entropy value for normalization; design consistency and feedback factor interaction term : ; is a preset amplification coefficient, used to control the amplification multiple of the consistency score, is a preset user feedback influence factor; Designing dynamic penalty terms : ; and are preset fluctuation sensitivity coefficients and the sum of the two is one, is a confidence fluctuation standard deviation, is an average of adjacent round answer confidence drop rates; The model answer adaptation index is calculated by the formula: ; is the model answer adaptation index.

5. The intelligent question and answer optimization method based on a large-scale pre-training model according to claim 3, characterized in that, The acquisition logic of the model interaction adaptation index is: The total number of model inputs and outputs within the specified question and answer turn window is divided by the window duration to obtain interaction activity , the number of times of obtaining each turn answer reference history information is averaged to obtain context retention , the ratio of the number of times of changing topics in all turns to the total number of inputs is obtained to obtain change adaptability , the time consumption of each turn answer is obtained and averaged to obtain interaction response , the model interaction adaptability index calculation formula is: ; is a preset interaction activity adjustment factor, is a preset gating steepness adjustment factor, is a preset adaptive gating threshold, is a preset power exponent adjustment factor, is a preset response time adjustment factor, is an overall gain adjustment factor, is an interaction complexity synthesis term, calculated by the following formula: ; is a model interaction adaptation index.

6. The intelligent question and answer optimization method based on a large-scale pre-training model according to claim 5, characterized in that, Through the pre-trained convolutional neural network model, the overall adaptation degree of the current large-scale pre-training model is analyzed based on the model answer adaptation index and the model interaction adaptation index, and it is divided into a high adaptation type or a low adaptation type.

7. The intelligent question and answer optimization method based on a large-scale pre-training model according to claim 6, characterized in that, When the division result is the low adaptation type, the number of the current large-scale pre-training model, the model answer adaptation index and the model interaction adaptation index are obtained and used as input variables of the fuzzy logic, the numbers of the remaining models in the model resource pool are used as output variables of the fuzzy logic, the input variables are fuzzified, the values of the input variables are converted into fuzzy sets, the output variables are fuzzified, the output variables are converted into fuzzy sets, fuzzy rules are formulated, the adaptation degrees of the remaining models in the model resource pool under different data category combinations are described, the fuzzified input variables are inferred through the fuzzy rules, and the number of the optimized model in the model resource pool is obtained.

8. The intelligent question-answering system based on large-scale pre-training model, used for implementing the intelligent question-answering optimization method based on large-scale pre-training model according to any one of claims 1-7, characterized in that, Comprise: A model resource pool for storing a plurality of large-scale pre-training models; A confidence monitoring module for monitoring answer confidence information of an output answer of a current large-scale pre-training model; An interaction anomaly detection module for extracting statistical features of the confidence within a specified question and answer round window and judging whether to trigger a double adaptation evaluation mechanism; An adaptation evaluation module for performing model answer adaptation degree evaluation and model interaction adaptation degree evaluation on the current large-scale pre-training model when the double adaptation evaluation mechanism is triggered; A type division module for dividing the current large-scale pre-training model into a high adaptation type or a low adaptation type based on double evaluation results; An optimization switching module for finding an optimized model from the model resource pool based on the current large-scale pre-training model and the double evaluation results to switch when the division result is the low adaptation type; A context migration module for transferring user dialogue context when the model is switched to realize seamless migration; A question and answer execution module for performing intelligent question and answer interaction based on the current model.

Images