An interactive question-answering system and question-answering method based on multi-model parallel reasoning

Abstract

The present invention discloses an interactive question-answering system and a question-answering method based on multi-model parallel reasoning. The system includes an interactive module, a model database, and an answer integration module; the interactive module is used to receive user input, send it to the answer integration module, and display information; the model database is used to store information sets of n large models; the answer integration module includes a weight calculation submodule, an answer integration submodule, and a statistical information calculation and visualization submodule; the weight calculation submodule calculates the weight of each large model for the user's question based on the question raised by the user and the information of each large model; the answer integration submodule calculates the final integrated answer based on the answers given by each large model, the weight of each model, and the user's preferred setting parameters; the statistical information calculation and visualization submodule calculates statistical information and performs visualization processing. The present invention can facilitate user intervention in the final integrated answer and trace the answer to its source, so that the final answer is more compatible with user needs.

Description

Technical Field

[0001] The present invention is directed to the field of large-scale model applications, and specifically relates to an interactive question-answering system and a question-answering method based on multi-model parallel reasoning. Background Art

[0002] The release of GPT-3 has made society and the market recognize the value and potential of large language models (hereafter referred to as large models). Over the past few years, companies, laboratories, and research institutes around the world have independently or collaboratively launched hundreds of large model products. Initially released large models are typically trained on massive amounts of heterogeneous data from various sources, such as news reports, social media, and encyclopedias. The trained models focus on capturing a broad range of knowledge and language patterns, and are therefore known as general models. Building on these general models, training data is concentrated within specific domains. For example, training data for specialized medical models may come from medical records, research papers, and clinical images. These trained models significantly outperform general models in specialized domains, and are thus known as specialized models. For ordinary users, large models can provide significant assistance in their work, study, and daily lives. However, the current variety of large models varies greatly, and their application scenarios vary widely. Even high-performing models can exhibit significant output fluctuations in certain scenarios, making them difficult for users to trust. The time required for users to pose questions to numerous large models, collect and evaluate the answers, is prohibitively expensive and significantly impacts user productivity and learning efficiency.

[0003] To enable users to quickly obtain answers with high confidence, we currently use a collaborative approach involving multiple large models, implemented in three ways: fusion, ensemble, and cooperation. Ensemble collaboration can be categorized into three types: pre-inference integration, mid-inference integration, and post-inference integration.

[0004] (1) Pre-inference integration method

[0005] Pre-inference integration involves integrating large language models before inference, selecting the most appropriate large model for the task or input. This is achieved by training an external router. This router selects an appropriate large model for the inference task based on certain criteria or rules.

[0006] (2) Integration methods in reasoning

[0007] The integration method in inference combines the outputs of multiple large models during the inference process, that is, in the decoding step.

[0008] (3) Post-inference integration

[0009] The post-inference integration method is to let multiple large models perform inference separately and generate their own outputs, and then integrate these outputs after the inference is completed. Specifically, multiple large models are sorted according to performance indicators (such as parameter quantity, BERTScore, BLEURT, BARTScore, etc.), and then integrated through cascading, selection, or fusion models.

[0010] The existing integration method has the following shortcomings:

[0011] 1) Although pre-inference integration can screen and integrate large language models in advance, avoiding poor performance caused by using inappropriate large language models during inference, the selected single large model may not necessarily provide the most comprehensive and accurate answers. In addition, the selection of this large model is significantly affected by the router. When the router performs poorly, the final output answer will be of poor quality.

[0012] 2) While the final answer obtained through post-inference integration combines answers from different large models, users cannot intervene in the integration process according to their needs, which may result in the loss of important information. Furthermore, users cannot trace the source of the final answer, making it difficult to ask additional questions. Summary of the Invention

[0013] To address the shortcomings of the existing technology, the present invention proposes an interactive question-answering system and method based on multi-model parallel reasoning, combining pre-reasoning and post-reasoning integration. The specific technical solution is as follows:

[0014] An interactive question-answering system based on multi-model parallel reasoning, including an interactive module, a model database, and an answer integration module;

[0015] The interactive module is used to receive user input, send it to the answer integration module, and display the information;

[0016] The model database is used to store information sets of n large models;

[0017] The answer integration module includes a weight calculation submodule, an answer integration submodule and a statistical information calculation and visualization submodule; wherein,

[0018] The weight calculation submodule has a built-in model weighting algorithm adjusted by user preferences, which calculates the weight of each large model for the question raised by the user based on the question raised by the user and the information of each large model;

[0019] The answer integration submodule has a built-in answer integration algorithm adjusted by user preferences, which calculates the final integrated answer based on the answers given by each large model to the question raised by the user, the weights of each model passed by the weight calculation submodule, and the user's preference setting parameters;

[0020] The statistical information calculation and visualization submodule calculates the statistical information of each word in the final integrated answer in all model answers based on the final integrated answer and the answers of each model to the question raised by the user, and displays it in the interactive module after visualization.

[0021] Furthermore, the specific execution process of the weight calculation submodule is as follows:

[0022] S1.1: Initialize the user ratings of all large models to unbiased ratings, and adjust the ratings of the corresponding large models to the user input values ​​based on the user input;

[0023] S1.2: Use machine learning methods to transform each large model tag t in the set tags of all large model tags in the model database i Convert to vector representation v i , build model tag vector set tags v ; where tags={t1,t2,…,t i ,…,t x}, tags v ={v1,v2,…, v i ,…,v x}; In addition, for each large model M in the model database i Corresponding to all labels, build the model label matrix , where a is the number of all labels;

[0024] S1.3: Extract keywords from user questions using a neural network-based approach in natural language processing. Give the question a classification label based on the set of tags consisting of all large model tags. Then, select the k tags with the highest probability based on the tag probability values ​​in the algorithm results to construct a question label matrix. , and record the probability value of each label ;

[0025] S1.4: Match the classification labels of the extracted user questions with the labels of all large models in the model database and calculate the similarity matrix S i And sort the successfully matched models from high to low according to the matching degree, and normalize the matching degree to obtain the matching weight value ;

[0026] S1.5: Based on the user's choice, use the public rating of the large model on the network or the user-defined rating to build a collection of the ratings of all large models and perform normalization. The model score is recorded as score norm ={s1,…,s i ,…,s n}, calculate the weight w of the i-th large model for the question raised by the current user i , and the calculated weight w i Passed to the answer integration submodule for subsequent calculations.

[0027] Furthermore, in step S1.4, the specific matching process of matching the extracted classification labels of user questions with the labels of all large models in the model database is as follows:

[0028] The question label vector V q With each large model M in the model database i The model label matrix Calculate the weighted cosine vector similarity matrix S i , if S i Each row in the table has a similarity threshold h that is not less than the user-set similarity threshold h. s The value of , then the model M i Matching is successful, and S is recorded i In each row, there is no less than h s The number of values ​​is used as the model fit.

[0029] Furthermore, in S1.5, the weights w of each large model i The calculation formula is as follows:

[0030]

[0031] Among them, z is the preference coefficient set by the user, which is used to control the tendency of the overall control system for decision-making.

[0032] Furthermore, the process of the answer integration submodule calculating the final integrated answer is as follows:

[0033] S2.1: First, read the answers output by each large model for this problem from the model database and standardize them one by one;

[0034] S2.2: Map the answers output by each large model into a high-dimensional semantic space to generate corresponding semantic embedding vectors. Then, calculate the semantic similarity between all answer pairs based on cosine similarity and construct a symmetric similarity matrix, denoted as Sim(i, j), where i and j represent different answers and Sim(i, j) represents the semantic similarity between answer i and answer j.

[0035] S2.3: Comprehensive weights w of each large model i And semantic similarity, calculate the final integrated weight of each answer:

[0036]

[0037]

[0038] in, represents the integration weight of the i-th answer, n represents the number of answers, is a hyperparameter for adjusting the influence of semantic similarity, with a value range of [0,1]; is the semantic similarity threshold; is a piecewise function;

[0039] S2.4: Select a large model with context understanding, reasoning and text generation capabilities as the final answer integration model, generate a logically coherent and grammatically fluent answer by inputting the original answers and final integration weights of each large model, and receive keywords related to the question entered by the user and the semantic similarity threshold to adjust the semantic tendency of the final integrated answer; the answer integration sub-module also maps each word in the final integrated answer back to the original answer.

[0040] Furthermore, the interaction module specifically includes:

[0041] Initialization phase: receiving questions raised by users, default model data or custom model data selected by users, and displaying model information in categories;

[0042] The interactive phase of answer integration: receives user input related to the question, semantic similarity threshold, modified model information, and preference coefficient, displays the integrated answer obtained from the answer integration module, and displays the user's question parameters and information about the various large models used in the integrated answer;

[0043] After the final integrated answer is determined: receive the user's word-marking operation and display the traceability and statistical information related to the final integrated answer.

[0044] Furthermore, the information set of each large model in the model database is represented by a tuple:

[0045] M i =(name i , source i , parameters i , score_original i , score_client i ,domain_orginal i, domain_client i , tags_original i , tags_client i , weight i ,description i , answer i ,…)

[0046] Among them, name i Indicates the name of the i-th large model; source i Indicates the source of the i-th large model (which company, institution or university published it); parameters i Indicates the number of parameters of the i-th large model; score_original i Indicates the public score of the i-th large model on the network; score_client i Indicates the user's rating of the i-th large model; domain_orginal i Indicates the application domain targeted by the publisher of the i-th large model; domain_client i It is the application field that the user considers suitable for the i-th large model based on his own needs; tags_original i Indicates the label that the publisher of the i-th large model has marked; tags_client i is the label that the user marks for the i-th large model according to his or her own preference; weight i Represents the weight score of the i-th large model, which is obtained by the model weighting algorithm adjusted by user preferences; description i Indicates the publisher's description of the i-th large model, which can also be modified by the user; answer i Used to store the answers obtained by the i-th large model to questions raised by users.

[0047] Furthermore, the semantic similarity threshold The initial value is 0.5, increasing If the value is set, the more relevant answers will be selected during the answer integration process, reducing If the value is taken, more potential relevant answers will be introduced into the answer integration process.

[0048] Furthermore, the standardization process specifically includes unifying upper and lower case, removing punctuation marks, and removing stop words.

[0049] An interactive question-answering method based on multi-model parallel reasoning is implemented by an interactive question-answering system based on multi-model parallel reasoning. The method includes the following steps:

[0050] Step 1: The interactive module sets model information, loads default values, selects a model cluster, receives user questions, and sends them to all large models in the model cluster for answers;

[0051] Step 2: The weight calculation submodule in the answer integration module classifies and locates the user's questions and calculates the weight of each large model in the model database for the user's questions;

[0052] Step 3: The answer integration submodule in the answer integration module outputs an integrated answer based on the answers given by each model to the question posed by the user, the weights of each model passed by the weight calculation submodule, and the user's preference parameters;

[0053] Step 4: The output integrated answer is displayed to the user in real time through the interactive module, and the user's instruction is received to confirm whether it is the final integrated answer; if it is, step 5 is executed; if not, the user's modified preference setting parameters are received, and the answer integration submodule re-outputs the integrated answer;

[0054] Step 5: The statistical information calculation and visualization submodule in the answer integration module calculates the statistical information of each word in the final integrated answer in all model answers based on the final integrated answer and the answers of each large model, and then performs visualization and displays it in the interactive module.

[0055] The beneficial effects of the present invention are as follows:

[0056] (1) In the system and method of the present invention, each model receives the user's question and gives an inferred answer each time the user asks a question. The system assigns weights to all large models based on the user's question and the various parameter indicators of the models in the system. The weights are only for the question asked by the user this time. The present invention can consider and retain valuable answers provided by some models whose overall scores are not high enough.

[0057] (2) The system and method of the present invention enable users to intervene in the answer integration process according to their own needs (such as setting keywords, setting word frequency thresholds, etc.). The integrated answer will change with the user's intervention, making the final answer more compatible with the user's needs.

[0058] (3) The system and method of the present invention can mark each word or sentence in the final integrated answer, so that users can check at any time which model or models any word or sentence in the final integrated answer comes from, making it easier for users to trace the source.

[0059] (4) The system and method of the present invention provide users with a visual display of answer differences, allowing users to easily view information such as the consistency, difference, and difference points between the answers of any model and the final integrated answer, thereby reducing the time cost of reading and evaluating the answers of each large model and improving usage efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is a schematic diagram of the composition of the interactive question-answering system based on multi-model parallel reasoning of the present invention, and its relationship with users and model clusters.

[0061] Figure 2 Schematic diagram of the initialization state of the interaction module.

[0062] Figure 3 Schematic diagram of the interactive process for integrating answers.

[0063] Figure 4 Schematic diagram of the answer tracing and visualization process.

[0064] Figure 5 This is a flow chart of the interactive question-answering method of the present invention, wherein the solid arrow represents the first-time running process, and the dotted arrow represents the process of the user intervening in the answer based on preference. DETAILED DESCRIPTION

[0065] The present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments, and the purpose and effects of the present invention will become more apparent. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0066] Technical term explanation:

[0067] GloVe: Global Vectors for Word Representation, word embedding of global vectors

[0068] Word2vec: Word to vector, a toolkit for obtaining word vectors launched by Google in 2013;

[0069] BERT model, Bidirectional Encoder Representations from Transformers, a pre-trained language model;

[0070] Sentence-BERT: Sentence Bidirectional Encoder Representations from Transformers, sentence embedding representation model;

[0071] GPT: Generative Pre-trained Transformer, generative pre-trained transformer.

[0072] On the one hand, the present invention provides an interactive question-answering system based on multi-model parallel reasoning, such as Figure 1 As shown in the figure, the system consists of three parts: interaction module, model database and answer integration module.

[0073] 1. Interaction Module

[0074] The interaction module is used to receive user input and send it to the answer integration module. At the same time, it obtains the final integrated answer from the answer integration module and displays it. The user input is different at different stages of the question and answer, and is divided into:

[0075] Initialization phase: The interaction module receives questions raised by the user, the default model data or custom model data selected by the user, and sends them to the answer integration module, and classifies and displays each model and its parameters in the model group.

[0076] Interactive stage of answer integration: When the user is not satisfied with the integrated answer, the interactive module receives the keywords related to the question, semantic similarity threshold, modified model information, preference coefficient, etc. input by the user, and sends them to the answer integration module. Then, the regenerated integrated answer is obtained from the reply integration module and displayed, while displaying the user's question parameters and the information of each model used in the integrated answer.

[0077] After the final integrated answer is determined: the interactive module receives the user's word-marking operation and visually displays the traceability information and statistical information of the marked words.

[0078] After the system starts, the interactive module is initialized. Figure 2 As shown, the interactive interface display of the initialization phase. Area A is the input interaction area, which serves as the entrance to the user's questions and adopts the form of a text input box to provide the original input data for the subsequent question-and-answer process. Area B is used for model database management. It uses a dual-mode switching mechanism and supports two loading modes: default model data and custom model database. The default model data directly calls the system's preset model scores, labels and other metadata; the custom model data allows users to modify the basic model information (such as scoring weights, field labels), and the modified results will be used as input parameters for the model answer weight calculation. Area C is used to display model information, including static information such as model name and version number, as well as configurable information such as field labels and scoring weights. Area D is used for model filtering, providing filtering dimensions such as fields, labels, and parameter quantities to realize multi-condition combination filtering functions. All are selected by default.

[0079] like Figure 3As shown in the figure, during the interactive phase of answer integration, after receiving the question raised by the user, the interactive module obtains the integrated answer from the answer integration module and displays it. Area A is used to display the final integrated answer, and the integration process follows the model weights and screening conditions configured by the user. Area B is used for parameter intervention operations. First, the parameters used in the generation of the final integrated answer are displayed, and a modification function is provided to meet the user's needs for parameter modification. At the same time, the user input is received, including keywords related to the user's question, semantic similarity threshold, modified model information, preference coefficient, and other parameters. After the parameters are modified, they are sent to the answer integration module for calculation. The answer content displayed in Area A is updated in real time as the parameters change.

[0080] like Figure 4 As shown, when the final integrated answer is determined, that is, when the user is satisfied, the interactive module provides a word-marking and tracing function, receives the user's word-marking operation, and starts the data tracing and statistical process. Area B is used to display the word's tracing information and statistical information, and at the same time, visualizes some appropriate statistical information.

[0081] 2. Model Database

[0082] The model database is used to store the information set of n large models M={M1, M2, M3,…,M i ,…,M n}, any large model M i The information is represented by a tuple:

[0083] M i =(name i , source i , parameters i , score_original i , score_client i ,domain_orginal i , domain_client i , tags_original i , tags_client i , weight i ,description i , answer i ,…)

[0084] Among them, name i Indicates the name of the i-th large model; source i Indicates the source of the i-th large model (which company, institution or university published it); parameters iIndicates the number of parameters of the i-th large model; score_original i Indicates the public score of the i-th large model on the network; score_client i Indicates the user's rating of the i-th large model; domain_orginal i Indicates the application domain targeted by the publisher of the i-th large model; domain_client i It is the application field that the user considers suitable for the i-th large model based on his own needs; tags_original i Indicates the label that the publisher of the i-th large model has marked; tags_client i is the label that the user marks for the i-th large model according to his or her own preference; weight i Represents the weight score of the i-th large model, which is obtained by the model weighting algorithm adjusted by user preferences; description i Indicates the publisher's description of the i-th large model, which can also be modified by the user; answer i Used to store the answers obtained by the i-th large model to questions raised by users.

[0085] 3. Answer Integration Module

[0086] The answer integration module is divided into a weight calculation submodule, an answer integration submodule, and a statistical information calculation and visualization submodule.

[0087] 3.1. Weight calculation submodule

[0088] The input of the weight calculation submodule is the question raised by the user and the information of each large model, and the output is the weight w of each large model for the question raised by the user. i The weight calculation submodule has a built-in model weighting algorithm that is adjusted by user preferences to calculate the weight w of each large model for the question raised by the user. i The specific implementation process is as follows:

[0089] (1) Initialization configuration

[0090] First, score all users of the large model score_client i Initialized to 5.0 (representing no preference), and the score of the corresponding large model is adjusted according to the user input.

[0091] (2) Vector representation of model labels

[0092] Each large model tag t in the set tags consisting of all large model tags in the model database is converted into iConvert to vector representation v i , build model tag vector set tags v ; where tags={t1,t2,…, t i ,…,t x}, tags v ={v1,v2,…, v i ,…,v x}; In addition, for each large model M in the model database i Corresponding to all labels, build the model label matrix , where a is the number of all labels.

[0093] (3) Question tag extraction

[0094] The key words in the user's question are extracted through a neural network-based method in natural language processing (such as the BERT model). The classification label of the question is given based on the set tags composed of all large model labels. The k labels with the highest probability are selected according to the label probability values ​​in the algorithm results (k is entered by the user and defaults to 3. ), build the question label matrix , and record the probability value of each label .

[0095] (4) Tag matching

[0096] The classification labels of the extracted user questions are matched with the labels of all large models in the model database. Specifically, the question label vector V q With each model M in the model database i The model label matrix Calculate the weighted cosine vector similarity matrix S i , if S i Each row in the table has a similarity threshold h that is not less than the user-set similarity threshold h. s The value of , then the model M i Matching is successful, and S is recorded i In each row, there is no less than h s The number of values ​​is used as the model matching degree; finally, the successfully matched models are sorted from high to low according to the matching degree, and the matching degree is normalized to obtain the matching weight value .

[0097] Among them, the calculation formula of the similarity matrix is:

[0098]

[0099] in, To calculate the Hadamard product, For size The matrix represents the similarity of each question label to each model label.

[0100] (5) Model weight calculation

[0101] Based on the user's choice, the public rating of the model on the network or the user-defined rating is used to build a collection of all model ratings and normalize them. The model rating is recorded as score norm ={s1,…,s i ,…,s n}, calculate the weight value w of the i-th model for the question raised by the current user i , and calculate the weights w of each model i Passed to the answer integration submodule for subsequent calculations.

[0102] The weight value w of the i-th model for the question raised by the current user i The calculation formula is as follows:

[0103]

[0104] Among them, z is the preference coefficient set by the user, which is used to control the tendency of the overall control system for decision-making.

[0105] 3.2 Answer Integration Submodule

[0106] The input of the answer integration submodule is the answers given by each large model to the questions raised by the user, and the weights w of each model passed by the weight calculation submodule. i The answer integration submodule is built with an answer integration algorithm that is regulated by user preferences. The specific execution process is as follows:

[0107] 1) Text preprocessing

[0108] First, the answers output by each model for this question are read from the model database and standardized one by one to eliminate the influence of format and grammar and ensure the accuracy of semantic calculation, including unifying capitalization, removing punctuation, removing stop words, etc.

[0109] 2) Semantic weight calculation

[0110] The Sentence-BERT model is used to map each model's output answer into a high-dimensional semantic space, generating a corresponding semantic embedding vector. The cosine similarity is then used to calculate the semantic similarity between all answer pairs, constructing a symmetric similarity matrix, denoted as Sim(i, j), where i and j represent different answers and Sim(i, j) represents the semantic similarity between answer i and answer j.

[0111] 3) Integrated weight calculation

[0112] The final integration weight of each answer is calculated by combining the model weight and semantic similarity. Assuming there are n answers, the integration weight of the i-th answer is The calculation formula is:

[0113]

[0114]

[0115] Among them, w i It represents the model weight of the i-th model obtained by the model weighting algorithm adjusted by user preferences. The initial value is set to the weight obtained by the weight calculation module, reflecting its reliability. Users can subsequently adjust the model weight according to their own preferences to intervene in the content of the final integrated answer.

[0116] This is a hyperparameter to adjust the influence of semantic similarity, and its value range is [0,1]. Users can adjust it according to task requirements. For tasks that require highly consistent answers (such as calculation problems), increase For tasks that require a comprehensive diversity of perspectives (such as open questions), reduce value.

[0117] is the semantic similarity threshold, which is used to filter out answer pairs with low semantic similarity to reduce the interference of irrelevant information. The initial value is recommended to be set to 0.5, which can be adjusted through experimental data to adapt to specific scenario requirements. Users can adjust Value to control the integration of answers and improve The system will more strictly filter out highly relevant answers, which is suitable for scenarios with high consistency requirements. The system can introduce more potential relevant answers, which is suitable for questions that require inspiring discussions.

[0118] 4) Answer Generation

[0119] A large model with contextual understanding, reasoning, and text generation capabilities is selected as the final answer integration model (specific criteria: large-scale multi-task language understanding of at least 80%; context window size of at least 32kb; GPT is selected in this example). By inputting the original answers of each model and the final integrated weights, a logically coherent and grammatically fluent answer is generated. Users can also manipulate the semantic bias of the final integrated answer by setting keywords. In the generated answer, the answer integration submodule maps each word in the final integrated answer back to the original answer (preserving the mapping relationship between the original answer and the final integrated answer), enabling answer tracing. That is, each word in the final integrated answer can be traced back to its primary contributing model. This tracing information allows users to understand the influence of each model on the final integrated answer and can adjust various parameters to their own preferences.

[0120] 3.3 Statistical Information Calculation and Visualization Submodule

[0121] The input of the statistical information calculation and visualization submodule is the final integrated answer and the answers of each model to the questions asked by the user. The output is the statistical information of each word in the final integrated answer in all model answers, such as the percentage of answers using the word, the number of times the word appears in each answer, the number of synonyms / antonyms, etc. For appropriate data forms, corresponding visualization methods such as pie charts, bar charts, word cloud charts, etc. are used for display.

[0122] On the other hand, the present invention also provides an interactive question-answering method based on multi-model parallel reasoning, such as Figure 5 As shown, the method includes the following steps:

[0123] Step 1: The interactive module sets model information, loads default values, selects a model cluster, receives user questions, and sends them to all large models in the model cluster for answers;

[0124] Step 2: The weight calculation submodule in the answer integration module classifies and locates the user's questions and calculates the weight of each large model in the model database for the user's questions;

[0125] Step 3: The answer integration submodule in the answer integration module outputs an integrated answer based on the answers given by each model to the question posed by the user, the weights of each model passed by the weight calculation submodule, and the user's preference parameters;

[0126] Step 4: The output integrated answer is displayed to the user in real time through the interactive module, and the user's instruction is received to confirm whether it is the final integrated answer; if it is, step 5 is executed; if not, the user's modified preference setting parameters are received, and the answer integration submodule re-outputs the integrated answer;

[0127] Step 5: The statistical information calculation and visualization submodule in the answer integration module calculates the statistical information of each word in the final integrated answer in all model answers based on the final integrated answer and the answers of each large model, and then performs visualization and displays it in the interactive module.

[0128] Those skilled in the art will understand that the foregoing descriptions are merely preferred embodiments of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art will still be able to modify the technical solutions described in the foregoing examples or substitute equivalents for some of the technical features therein. Any modifications, equivalent substitutions, etc. made within the spirit and principles of the invention shall be included within the scope of protection of the invention.

Claims

1. An interactive question-answering system based on multi-model parallel reasoning, characterized by: Includes interactive module, model database and answer integration module; The interactive module is used to receive user input, send it to the answer integration module, and display the information; The model database is used to store information sets of n large models; The answer integration module includes a weight calculation submodule, an answer integration submodule and a statistical information calculation and visualization submodule; wherein, The weight calculation submodule has a built-in model weighting algorithm adjusted by user preferences, which calculates the weight of each large model for the question raised by the user based on the question raised by the user and the information of each large model; The answer integration submodule has a built-in answer integration algorithm that is adjusted by user preferences. It calculates the final integrated answer based on the answers given by each large model to the question posed by the user, the weights of each model passed by the weight calculation submodule, and the user's preference settings. The process of calculating the final integrated answer by the answer integration submodule is as follows: S2.1: First, read the answers output by each large model for this problem from the model database and standardize them one by one; S2.2: Map the answers output by each large model into a high-dimensional semantic space to generate corresponding semantic embedding vectors. Then, calculate the semantic similarity between all answer pairs based on cosine similarity and construct a symmetric similarity matrix, denoted as Sim(i, j), where i and j represent different answers and Sim(i, j) represents the semantic similarity between answer i and answer j. S2.3: Comprehensive weights w of each large model i And semantic similarity, calculate the final integrated weight of each answer: ； ； in, represents the integration weight of the i-th answer, n represents the number of answers, is a hyperparameter for adjusting the influence of semantic similarity, with a value range of [0,1]; is the semantic similarity threshold; is a piecewise function; S2.4: Select a large model with contextual understanding, reasoning, and text generation capabilities as the final answer integration model. By inputting the original answers of each large model and the final integrated weight, a logically coherent and grammatically fluent answer is generated. Keywords related to the question posed and a semantic similarity threshold are input by the user to adjust the semantic orientation of the final integrated answer. The answer integration submodule also maps each word in the final integrated answer back to the original answer. The statistical information calculation and visualization submodule calculates the statistical information of each word in the final integrated answer in all model answers based on the final integrated answer and the answers of each model to the question raised by the user, and displays it in the interactive module after visualization.

2. The interactive question-answering system based on multi-model parallel reasoning according to claim 1 is characterized in that: The specific execution process of the weight calculation submodule is as follows: S1.1: Initialize the user ratings of all large models to unbiased ratings, and adjust the ratings of the corresponding large models to the user input values ​​based on the user input; S1.2: Use machine learning methods to transform each large model tag t in the set tags of all large model tags in the model database i Convert to vector representation v i , build model tag vector set tags v ; Among them, tags={t1,t2,…,t i ,…,t x }, tags v ={v1,v2,…, v i ,…,v x }; In addition, for each large model M in the model database i Corresponding to all labels, build the model label matrix , where a is the number of all labels; S1.3: Extract keywords from user questions using a neural network-based approach in natural language processing. Give the question a classification label based on the set of tags consisting of all large model tags. Then, select the k tags with the highest probability based on the tag probability values ​​in the algorithm results to construct a question label matrix. , and record the probability value of each label ; S1.4: Match the classification labels of the extracted user questions with the labels of all large models in the model database and calculate the similarity matrix S i And sort the successfully matched models from high to low according to the matching degree, and normalize the matching degree to obtain the matching weight value ; S1.5: Based on the user's choice, use the public rating of the large model on the network or the user-defined rating to build a collection of the ratings of all large models and perform normalization. The model score is recorded as score norm ={s1,…,s i ,…,s n }, calculate the weight w of the i-th large model for the question raised by the current user i , and the calculated weight w i Passed to the answer integration submodule for subsequent calculations.

3. The interactive question-answering system based on multi-model parallel reasoning according to claim 2, characterized in that: In step S1.4, the specific matching process of matching the extracted classification labels of user questions with the labels of all large models in the model database is as follows: The question label matrix V q With each large model M in the model database i The model label matrix Calculate the weighted cosine vector similarity matrix S i , if S i Each row in the table has a similarity threshold h that is not less than the user-set similarity threshold h. s The value of , then the model M i Matching is successful, and S is recorded i In each row, there is no less than h s The number of values ​​is used as the model fit.

4. The interactive question-answering system based on multi-model parallel reasoning according to claim 2, characterized in that: In S1.5, the weights w of each large model i The calculation formula is as follows: ； Among them, z is the preference coefficient set by the user, which is used to control the tendency of the overall control system for decision-making.

5. The interactive question-answering system based on multi-model parallel reasoning according to claim 1 is characterized in that: The interaction module specifically includes: Initialization phase: receiving questions raised by users, default model data or custom model data selected by users, and displaying model information in categories; The interactive phase of answer integration: receives user input related to the question, semantic similarity threshold, modified model information, and preference coefficient, displays the integrated answer obtained from the answer integration module, and displays the user's question parameters and information about the various large models used in the integrated answer; After the final integrated answer is determined: receive the user's word-marking operation and display the traceability and statistical information related to the final integrated answer.

6. The interactive question-answering system based on multi-model parallel reasoning according to claim 1, characterized in that: The information set of each large model in the model database is represented by a tuple: M i =(name i , source i , parameters i , score_original i , score_client i , domain_orginal i , domain_client i , tags_original i , tags_client i , weight i , description i ,answer i ,…) Among them, name i Indicates the name of the i-th large model; source i Indicates the source of the i-th large model; parameters i Indicates the number of parameters of the i-th large model; score_original i Indicates the public score of the i-th large model on the network; score_client i Indicates the user's rating of the i-th large model; domain_orginal i Indicates the application domain targeted by the publisher of the i-th large model; domain_client i It is the application field that the user considers suitable for the i-th large model based on his own needs; tags_original i Indicates the label that the publisher of the i-th large model has marked; tags_client i is the label that the user marks for the i-th large model according to his or her own preference; weight i Represents the weight score of the i-th large model, which is obtained by the model weighting algorithm adjusted by user preferences; description i Indicates the publisher's description of the i-th large model, which can also be modified by the user; answer i Used to store the answers obtained by the i-th large model to questions raised by users.

7. The interactive question-answering system based on multi-model parallel reasoning according to claim 1, characterized in that: Semantic similarity threshold The initial value is 0.5, increasing If the value is set, the more relevant answers will be selected during the answer integration process, reducing If the value is taken, more potential relevant answers will be introduced into the answer integration process.

8. The interactive question-answering system based on multi-model parallel reasoning according to claim 1, characterized in that: The standardization process specifically includes unifying upper and lower case, removing punctuation marks, and removing stop words.

9. An interactive question-answering method based on multi-model parallel reasoning, characterized in that: The method is implemented by the interactive question-answering system based on multi-model parallel reasoning according to claim 1, and the method comprises the following steps: Step 1: The interactive module sets model information, loads default values, selects a model cluster, receives user questions, and sends them to all large models in the model cluster for answers; Step 2: The weight calculation submodule in the answer integration module classifies and locates the user's questions and calculates the weight of each large model in the model database for the user's questions; Step 3: The answer integration submodule in the answer integration module outputs an integrated answer based on the answers given by each model to the question posed by the user, the weights of each model passed by the weight calculation submodule, and the user's preference parameters; Step 4: The output integrated answer is displayed to the user in real time through the interactive module, and the user's instruction is received to confirm whether it is the final integrated answer; If yes, execute step 5; if no, receive the preference setting parameters modified by the user, and the answer integration submodule re-outputs the integrated answer; Step 5; The statistical information calculation and visualization submodule in the answer integration module calculates the statistical information of each word in the final integrated answer in all model answers based on the final integrated answer and the answers of each large model, and then displays it through the interactive module after visualization.

Images