Interactive question-and-answer system and method based on multi-model parallel inference
The interactive question-and-answer system with multi-model parallel inference addresses output variability by integrating user preferences and source tracing, ensuring efficient and relevant answers.
Patent Information
- Application Number
- JP2026038385
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-09
- Filing Date
- 2026-03-10
- Publication Date
- 2026-11-19
AI Technical Summary
Existing large-scale language models exhibit considerable output variability in specific application scenarios, making it difficult for users to obtain highly confident answers efficiently, and existing integration methods either rely heavily on a single model's quality or lack user intervention in the answer integration process.
An interactive question-and-answer system employing multi-model parallel inference that integrates pre-inference and post-inference methods, using an interaction module, model database, and answer integration module to calculate model weights and user preferences, allowing for user intervention in the response integration process and tracing the source of answers.
The system provides highly confident answers by retaining valuable model outputs, enabling user intervention for improved relevance, and facilitating source tracing, thus reducing time costs and improving efficiency.
Smart Images

Figure 2026182746000001_ABST
Abstract
Description
[Technical Field]
[0001] This invention relates to application fields of large-scale models, and more specifically to an interactive question-and-answer system and method based on multi-model parallel inference. [Background technology]
[0002] The announcement of GPT-3 made society and the market recognize the value and potential of large-scale language models (hereinafter abbreviated as "large-scale models"). In recent years, companies, laboratories, and research institutes around the world have successively announced hundreds of large-scale model products, either independently or collaboratively. Early large-scale models were typically trained on heterogeneous data from a vast amount of multi-source data spanning a wide range of sources, such as news reports, social media, and encyclopedias. Models trained in this way focused primarily on acquiring broad knowledge and language patterns, and this type of model is called a general-purpose model. Based on general-purpose models, models trained by concentrating training data on specific fields, such as medical records, medical research papers, and clinical images, perform significantly better in their respective fields and are called specialized models. For general users, large-scale models can provide broad support in work, learning, and daily life. However, there are now many types of large-scale models, each suited to different application scenarios, and even high-performance models still exhibit considerable output variability in specific application scenarios, making it difficult for users to easily accept the results. On the other hand, for users to ask questions to numerous large-scale models one by one and then collect and evaluate the answers would incur excessive time costs, seriously impacting the user's productivity and learning efficiency.
[0003] To enable users to quickly obtain highly confident answers, methods that coordinate multiple large-scale models are now employed, and these implementation methods can be broadly classified into three types: merge, ensemble, and cooperate. Here, large-scale model coordination using the ensemble method is further classified into three types: pre-inference ensemble, in-inference ensemble, and post-inference ensemble.
[0004] (1) Pre-inference aggregation method The pre-inference integration method is a method that integrates large-scale language models before performing inference operations, and selects the most suitable large-scale model for the current task or input from among many large-scale models. In specific implementations, an external router is trained, and the role of this router is to select a large-scale model suitable for the current inference task based on predetermined criteria or rules before inference is performed.
[0005] (2) Aggregation method during inference The in-inference augmentation method is a method that combines the outputs of multiple large-scale models during the inference process, specifically during the decoding step.
[0006] (3) Post-inference aggregation method Post-inference integration is a method in which multiple large-scale models are each made to perform inference, generate their respective outputs, and then these outputs are integrated after the inference is complete. Specifically, this method involves ranking multiple large-scale models based on performance metrics (e.g., parameter count, BERTScore, BLEURT, BARTScore, etc.) and then integrating them through cascading, selection, or fusion models.
[0007] Existing integrated systems have the following drawbacks:
[0008] 1) Pre-inference aggregation can avoid performance degradation due to the use of inappropriate large-scale language models in the reasoning process by selecting and deciding on the aggregation of large-scale language models in advance. However, it has the drawback that the single large-scale model selected may not necessarily provide the most comprehensive and accurate answer, and the selection of large-scale models is heavily influenced by the router. Therefore, if the router's performance is poor, the quality of the final output answer will be low.
[0009] 2) The final answer obtained through post-inference aggregation integrates answers from different large-scale models, but because users cannot intervene in the integration process based on their own needs, important information for the user may be excluded. Furthermore, a drawback is that users cannot trace the source of the final answer generation process, making it difficult to ask additional questions. [Overview of the Initiative] [Problems that the invention aims to solve]
[0010] In response to the shortcomings of the prior art, the present invention proposes an interactive question-and-answer system and method based on multi-model parallel inference, combining both pre-inference and post-inference integration methods. The specific technical solution is as follows: [Means for solving the problem]
[0011] An interactive question-and-answer system based on multi-model parallel inference, the interactive question-and-answer system includes an interaction module, a model database, and an answer integration module. The interaction module is used to receive user input, transmit it to the response integration module, and display information. The aforementioned model database is used to store information sets for n large-scale models. The aforementioned response integration module includes a weight calculation submodule, a response integration submodule, and a statistical information calculation / visualization submodule. The aforementioned weight calculation submodule incorporates a model weighting algorithm that is adjusted according to user preferences, and calculates the weights of each large model for the user-submitted question based on the user-submitted question and information about each large model. The aforementioned answer integration submodule incorporates an answer integration algorithm that is adjusted according to user preferences, and calculates the final integrated answer based on the answers provided by each large model to the questions submitted by the user, the weights of each model transmitted from the weight calculation submodule, and the user preference setting parameters. The process by which the aforementioned answer integration submodule calculates the final integrated answer includes the following steps 2.1 to 2.4: In step 2.1, first, the answers output from each large-scale model to the question are read from the model database, and each answer is standardized one by one. In step 2.2, the responses output by each large-scale model are mapped to a high-dimensional semantic space, corresponding semantic embedding vectors are generated, and then the semantic similarity between all response pairs is calculated based on cosine similarity, and a symmetric similarity matrix is constructed as the symbol Sim(i,j), where i and j represent different responses, and Sim(i,j) represents the semantic similarity between response i and response j. In step 2.3, the weights W of each large model were added. i Integrating semantic similarity,
number
[0012] Further, the specific execution process of the weight calculation sub-module includes the following step 1.1 to step 1.5, In step 1.1, the user evaluation scores of all large models are initialized as non-preference evaluation scores, and the evaluation score of the corresponding large model is adjusted to the user input value based on the user's input, In step 1.2, by a machine learning method, for each large model tag t in the set tags consisting of all large model tags in the model database i to vector representation v i to construct a model tag vector set tags v , where tags={t1,t2,…,t i ,…,t x}, tags v ={v1,v2,…,v i ,…,v x}, and for all tags corresponding to each large model M in the model database i , a model tag matrix is constructed TIFF2026182746000007.tif956, where a is the total number of tags. In step 1.3, keywords are extracted from the user's question based on a neural network-based method in natural language processing, classification tags are assigned to the question based on a set of tags consisting of all large-scale model tags, and k tags with the highest probability are selected according to the ranking of the probability values of the tags in the algorithm result to form a question tag matrix. Construct TIFF2026182746000008.tif844 and determine the probability value of each tag. Recorded TIFF2026182746000009.tif846, In step 1.4, the classification tags of the extracted user questions are matched with the tags held by all large-scale models in the model database, and a similarity matrix S is created. i The matching weights are calculated, the models that successfully match are sorted in descending order of matching degree, and the matching degree is normalized to obtain the matching weight values. Obtaining TIFF2026182746000010.tif865, In step 1.5, based on the user's selection, the evaluation scores of all large models are constructed as a set using either publicly available evaluation scores on the network of large models or user-defined evaluation scores, and then normalized, and the evaluation scores of the models are scored norm ={s1,…,s i ,…,s n} is written, and the weights of the i-th large model for the question currently submitted by the user w i The calculated weight w i The data is transmitted to the answer integration submodule and used for subsequent calculations.
[0013] Furthermore, in step 1.4, in the specific matching process in which the classification tags of the extracted user questions are matched with the tags held by all large-scale models in the model database, Question Tag Matrix V q And, each large-scale model M in the model database i Model tag matrix For TIFF2026182746000011.tif89, the weighted cosine vector similarity matrix S i The weighted cosine vector similarity matrix S is calculated. i In each of the rows, the similarity threshold h set by the user is... s If the above values exist, then Model M i The matching was determined to be successful, and the weighted cosine vector similarity matrix S i h s The number of values above these will be recorded as the degree of model matching.
[0014] Furthermore, in step 1.5, the weights w of each large model i The formula for calculating this is:
number
[0015] Furthermore, the interaction module specifically has an initialization phase, an integrated response interaction phase, and a phase after the final integrated response is identified. In the initialization phase, the system receives questions submitted by the user, default model data or custom model data selected by the user, and classifies and displays the model information. In the interaction phase of the integrated response, keywords related to the submitted question entered by the user, semantic similarity thresholds, modified model information, and preference coefficients are received, and the integrated response obtained from the response integration module is displayed, as well as the user's question parameters and information on each large-scale model used in the integrated response. In the subsequent steps following the identification of the final integrated answer, the user's selected vocabulary operations are received, and source trace information and statistical information related to the final integrated answer are displayed.
[0016] Furthermore, in the aforementioned model database, the information set for each large-scale model is a single tuple M i =(name i source i ,parameters i ,score_original i ,score_client i ,domain_orginal i ,domain_client i ,tags_original i ,tags_client i weight i description i , answer i As shown by, name i This indicates the name of the i-th large-scale model, source i This shows the source of the i-th large-scale model, parameters i This shows the parameter count of the i-th large-scale model, and score_original i This shows the public evaluation score on the network for the i-th large model, and score_client i This shows the evaluation score that the user assigned to the i-th large model, and is domain_original i This indicates the application area positioned by the presenter of the i-th large-scale model, domain_client i This indicates the application area where the user determined, based on their own needs, that the i-th large-scale model was suitable. i This indicates the tags assigned by the presenter of the i-th large-scale model, and tags_client i This indicates the tags that users have assigned to the i-th large model based on their preferences, and weight i This shows the weight score of the i-th large-scale model, calculated by a model weighting algorithm that is adjusted according to user preferences, and description iThis shows the presenter's explanation of the i-th large-scale model, which users can also modify themselves, and answer i This is used to store the answers obtained by the i-th large-scale model through inference in response to questions submitted by the user.
[0017] Furthermore, the threshold for semantic similarity The initial value for TIFF2026182746000013.tif54 is set to 0.5. Setting a high value for TIFF2026182746000014.tif54 will select more relevant responses during the response consolidation process. Setting a lower value for TIFF2026182746000015.tif54 will capture more potentially relevant answers during the integration process.
[0018] Furthermore, the standardization process specifically includes unifying uppercase and lowercase letters, removing punctuation marks, and removing stop words.
[0019] An interactive question-and-answer method based on multi-model parallel inference, the interactive question-and-answer method being executed by the above-described interactive question-and-answer system based on multi-model parallel inference, and comprising the following steps 1 to 5: In step 1, the interaction module sets model information, loads default values, selects a group of models, receives a user's question, and sends the user's question to all large models in the model cluster to get an answer. In step 2, the weight calculation submodule in the answer integration module classifies and positions the user's questions and calculates the weights for each large model in the model database for the user's questions. In step 3, the response integration submodule in the response integration module outputs an integrated response based on the responses provided by each model to the questions submitted by the user, the weights of each model transmitted from the weight calculation submodule, and the user's preference setting parameters. In step 4, the interaction module displays the outputted integrated response to the user in real time, receives user instructions, verifies whether it is the final integrated response, and if it is the final integrated response, executes step 5; if it is not the final integrated response, receives the user's modified preference setting parameters, and the response integration submodule re-outputs the integrated response. In step 5, the statistical information calculation and visualization submodule in the response integration module calculates and visualizes statistical information for all model responses for each vocabulary in the final integrated response based on the final integrated response and the responses of each large model, and then the interaction module displays the results. [Effects of the Invention]
[0020] The beneficial effects of the present invention are as follows: (1) The system and method of the present invention, each time a user submits a question, each model receives the user's question and generates an inference answer, and the system assigns weights to all large models based on the user's question and each parameter indicator of the models in the system, the weights being for the user's current question only, and the present invention retains valuable answers provided by models that do not have a high overall evaluation score. (2) The system and method of the present invention enable users to individually intervene in the response integration process (e.g., setting keywords, setting word frequency thresholds) according to their own needs, and the integrated response changes in response to the user's intervention, thereby increasing the degree of matching between the final response and the user's needs. (3) The system and method of the present invention can identify each word in the final integrated answer, allowing the user to search at any time to find out which or which models any word in the final integrated answer originates from, thereby facilitating source tracing for the user. (4) The system and method of the present invention provide users with a visual display of differences in answers, thereby enabling users to easily check information such as the degree of agreement, degree of difference, and differences between the answers of any model and the final integrated answer, thereby reducing the time cost of reading and evaluating the answers of each large model and improving utilization efficiency. [Brief explanation of the drawing]
[0021] [Figure 1] This diagram shows the configuration of an interactive question-and-answer system based on multi-model parallel inference according to the present invention, and a schematic diagram of its relationship to the user and the model cluster. [Figure 2] This is a schematic diagram of the initial state of the interaction module. [Figure 3] This is a schematic diagram of the interaction process for integrated responses. [Figure 4] This is a schematic diagram of the source tracing and visualization process for the answer. [Figure 5] This is a flowchart of the interactive question-and-answer method of the present invention, where the solid arrows in the figure represent the flow of the initial execution, and the dashed arrows represent the flow in which the user intervenes in the answer according to their preferences. [Modes for carrying out the invention]
[0022] The present invention will be described in detail below based on the drawings and preferred embodiments, but the object and effect of the present invention will become clearer, and the specific embodiments described herein should be understood to be used solely for the purpose of illustrating the present invention and not to limit it.
[0023] The interpretations of the technical terms are as follows: GloVe is Global Vectors for Word Representation, or word embedding using global vectors. Word2vec is a toolkit for obtaining word vectors, released as open source by Google in 2013. The BERT model, or Bidirectional Encoder Representations from Transformers, is a pre-trained language model. Sentence-BERT is a sentence embedding representation model called Sentence Bidirectional Encoder Representations from Transformers. GPT stands for Generative Pre-trained Transformer.
[0024] On the one hand, the present invention provides an interactive question-and-answer system based on multi-model parallel inference, and as shown in Figure 1, the system includes three parts: an interaction module, a model database, and an answer integration module.
[0025] 1. Interaction Module The interaction module is used to receive user input, send it to the answer integration module, and simultaneously retrieve and display the final integrated answer from the answer integration module. User input differs in content at different stages of the question-and-answer session and can be divided into the following stages:
[0026] During the initialization phase, the interaction module receives questions submitted by the user, default model data selected by the user, or custom model data, and sends this to the answer integration module, which then classifies and displays each model and its parameters within the model cluster.
[0027] During the integrated response interaction phase, if the user is not satisfied with the integrated response, the interaction module receives keywords related to the submitted question entered by the user, semantic similarity thresholds, modified model information, preference coefficients, etc., and sends them to the response integration module. Subsequently, the integrated response regenerated from the response integration module is retrieved and displayed, along with the user's question parameters and information on each model used in the integrated response.
[0028] After identifying the final integrated answer, the interaction module receives the user's selected vocabulary operations and visualizes the source trace information and statistics of the selected vocabulary.
[0029] After the system starts up, the interaction module performs initialization. As shown in Figure 2, this is a display of the interaction interface during the initialization phase. Area A is the input interaction area, which takes the form of a text input box as the entry point for user questions, providing the original input data for the subsequent question-and-answer flow. Area B is used for managing the model database and uses a dual-mode switching mechanism to support two types of load modes: default model data and custom model database. In the default model data load mode, metadata such as evaluation scores and tags of models pre-configured by the system are directly retrieved. In the custom model data load mode, the user is allowed to modify the basic information of the model (e.g., evaluation score weights, domain tags), and the modified results become input parameters for calculating the weights of the model's answers. Area C is used for displaying model information, which includes static information such as the model name and version number, as well as placeable information such as domain tags and evaluation score weights. Area D is used for model selection, providing selection dimensions such as domain, tags, and parameter amounts, realizing a selection function that combines multiple conditions. By default, all conditions are selected.
[0030] As shown in Figure 3, during the interaction phase of the integrated response, the interaction module receives the question submitted by the user, retrieves the integrated response from the response integration module, and displays it. Here, area A is used to display the final integrated response, and the integration process is carried out according to the model weights and selection criteria set by the user. Area B is used to perform parameter intervention operations, first displaying the parameters used in the generation process of the final integrated response, and providing a modification function to meet the user's parameter modification needs. It also receives parameters such as keywords related to the question entered by the user, semantic similarity thresholds, modified model information, and preference coefficients, modifies the parameters, and sends them to the response integration module for calculation, so that the response content displayed in area A is updated in real time according to the parameter changes.
[0031] As shown in Figure 4, after the final integrated answer is identified, i.e., after the user is satisfied, the interaction module provides a selected vocabulary source trace function, receives the user's selected vocabulary operation, and initiates the data source trace and statistical processing flow. Area B is used to display the source trace information and statistical information for the word in question, and also visualizes some appropriate statistical information.
[0032] 2. Model Database A model database is a set of information M = {M1, M2, M3, ..., M} of large-scale models. i ,…,M n Used to store}, where any one large model M i The information is in a single tuple M i =(name i source i ,parameters i ,score_original i ,score_client i ,domain_orginal i ,domain_client i ,tags_originali ,tags_client i weight i ,description i , answer i As shown by, Here's the name i This indicates the name of the i-th large-scale model, source i This indicates the source of the i-th large-scale model (which company, institution, or high school published it) and parameters. i This shows the parameter count of the i-th large-scale model, and score_original i This shows the public evaluation score on the network for the i-th large model, and score_client i This shows the evaluation score that the user assigned to the i-th large model, and is domain_original i This indicates the application area positioned by the presenter of the i-th large-scale model, domain_client i This indicates the application area where the user determined, based on their own needs, that the i-th large-scale model was suitable. i This indicates the tags assigned by the presenter of the i-th large-scale model, and tags_client i This indicates the tags that users have assigned to the i-th large model based on their preferences, and weight i This shows the weight score of the i-th large-scale model, calculated by a model weighting algorithm that is adjusted according to user preferences, and description i This shows the presenter's explanation of the i-th large-scale model, which users can also modify themselves, and answer i This is used to store the answers obtained by the i-th large-scale model through inference in response to questions submitted by the user.
[0033] III. Answer Integration Module The response integration module is divided into a weight calculation submodule, a response integration submodule, and a statistical information calculation and visualization submodule.
[0034] 3.1. Weight calculation submodule The inputs of the weight calculation submodule are the question submitted by the user and the information of each large model, and the output is the weight w of each large model for the question submitted by the user i , the weight calculation submodule incorporates a built-in model weighting algorithm that is adjusted according to user preferences, by which the weight w of each large model for the question submitted by the user i is calculated, and the specific implementation process is as follows.
[0035] (1) Initialization and configuration First, the user evaluation score score_client of all large models i is initialized to 5.0 (indicating no preference), and the evaluation score of the corresponding large model is adjusted to the user input value based on the user input.
[0036] (2) Vector representation of model tags By a machine learning method (for example, Word2vec or GloVe, etc.), for each large model tag t in the set tags consisting of all tags of all large models in the model database i is converted into a vector representation v i , and a model tag vector set tags v is constructed, where tags={t1,t2,…,t i ,…,t x}, tags v ={v1,v2,…,v i ,…,v x}, and for all tags corresponding to each large model M in the model database i , a model tag matrix TIFF2026182746000016.tif1056 is constructed, where a is the total number of tags.
[0037] (3) Extraction of question tags Based on neural network-based methods in natural language processing (e.g., the BERT model), keywords in the user's question are extracted, classification tags are assigned to the question based on the set tags consisting of all large model tags, and according to the ranking of the probability values of the tags in the algorithm result, k tags with the maximum probability (k is a value input by the user, its default value is 3, and k≦x) are selected to form a question tag matrix TIFF2026182746000017.tif844, and record the probability value of each tag TIFF2026182746000018.tif846.
[0038] (4) Tag Matching The extracted classification tags of the user's question are matched with the tags possessed by all large models in the model database. Specifically, for the question tag vector V q and the model M in the model database i of the model tag matrix TIFF2026182746000019.tif910, a weighted cosine vector similarity matrix S i is calculated, and in each row of S i , if there is a value greater than or equal to the similarity threshold h set by the user s , the model M i is recognized as having succeeded in matching, the number of values greater than or equal to h in each row of S i s is recorded as the model matching degree, finally, the models that have succeeded in matching are sorted in descending order of the matching degree, and the matching degree is normalized to obtain a matching weight value TIFF2026182746000020.tif864.
[0039] Here, the calculation formula of the similarity matrix is [Formula] represented by Here, TIFF2026182746000022.tif77 is an operation to calculate the Hadamard product, and S i This is a j x k matrix that shows the similarity of each question tag to each model tag.
[0040] (5) Calculation of model weights Based on the user's selection, a set of evaluation scores for all models is constructed using either the publicly available evaluation scores on the model network or user-defined evaluation scores, and then normalized to obtain the model evaluation scores as a score. norm ={s1,…,s i ,…,s n} is written, and the weights w of the i-th model for the question currently submitted by the user i The weights w of each model are calculated. i The data is transmitted to the answer integration submodule and used for subsequent calculations.
[0041] The weights w of the i-th model for the question submitted by the current user. i The formula for calculating this is:
number
[0042] 3.2 Answer Integration Submodule The input to the Answer Integration submodule is the answers generated from each large model for the questions submitted by the user, and the weights of each model transmitted from the Weight Calculation submodule. i These are user preference setting parameters, and their output is the final integrated answer, which is used to solve questions where the user cannot intervene in the answer integration process and the source trace of the final answer is not possible. The answer integration submodule contains an answer integration algorithm that is adjusted according to the user's preferences, and the specific execution process is as follows.
[0043] 1) Text preprocessing First, the output responses of each model to the question in question are read from the model database and standardized one by one. This eliminates the influence of formatting and grammar, ensures accuracy in semantic computation, and includes standardization of uppercase and lowercase letters, removal of punctuation, and removal of stop words.
[0044] 2) Calculation of semantic weights Using a Sentence-BERT model, the responses output by each model are mapped to a high-dimensional semantic space, and corresponding semantic embedding vectors are generated. Then, the semantic similarity between all response pairs is calculated based on cosine similarity, and a symmetric similarity matrix is constructed, denoted by the symbol Sim(i,j), where i and j represent different responses, and Sim(i,j) represents the semantic similarity between response i and response j.
[0045] 3) Calculation of integrated weights The final combined weight for each answer is calculated by comprehensively considering the model weights and semantic similarity. Assuming there are n answers, the combined weight α of the i-th answer is calculated. i The formula for calculating this is:
number
[0046] TIFF2026182746000025.tif65 is a hyperparameter that adjusts the influence of semantic similarity, with a possible range of [0,1]. Users can adjust this parameter according to the needs of the task, for tasks requiring a high degree of consistency in answers (e.g., computational questions). Increase the value of TIFF2026182746000026.tif65 to address tasks that require the integration of diverse perspectives (e.g., open questions). Reduce the value of TIFF2026182746000027.tif65.
[0047] TIFF2026182746000028.tif54 is a semantic similarity threshold used to reduce interference from irrelevant information by selecting and excluding answer pairs with low semantic similarity. It is recommended to set the initial value to 0.5, and it can be adjusted based on experimental data to suit the needs of specific scenarios. Users can: By adjusting the values in TIFF2026182746000029.tif54, you can control the integration of responses. Setting a higher value for TIFF2026182746000030.tif54 causes the system to more rigorously select highly relevant answers, making it suitable for scenarios with high consistency requirements, while, Setting a lower value for TIFF2026182746000031.tif54 allows the system to introduce more potential relevant answers, making it suitable for questions that require further discussion.
[0048] 4) Answer generation A large-scale model with contextual understanding, reasoning, and text generation capabilities is selected as the final integrated answer model (specifically, the large-scale multi-task language understanding portion must be 80% or more, and the context window size must be 32KB or more; in this embodiment, GPT is selected). By inputting the original answers from each model and the final integrated weights, a logically consistent and grammatically fluent answer is generated. Furthermore, the user can intervene in the semantic tendencies of the final integrated answer by setting keywords. In the generated answer, the answer integration submodule maps each vocabulary in the final integrated answer back to the original answer (maintaining the mapping relationship between the original answer and the final integrated answer), thereby realizing a source tracing function for the answer, meaning that each vocabulary in the final integrated answer can be traced back to its main contributing model. By tracing the information, the user can understand the influence each model had on the final integrated answer and intervene in the content of the final integrated answer by adjusting various parameters according to their preferences.
[0049] 3.3 Statistical Information Calculation and Visualization Submodule The inputs to the statistical information calculation and visualization submodule are the final integrated response and the responses of each model to user-submitted questions. The output is statistical information on each vocabulary in the final integrated response, such as the percentage of responses using that vocabulary, the number of occurrences in each response, and the number of synonyms and antonyms. For appropriate data formats, the information is displayed using corresponding visualization methods such as pie charts, bar graphs, and word clouds.
[0050] On the other hand, the present invention further provides an interactive question-and-answer method based on parallel inference of multiple models, which, as shown in Figure 5, includes the following steps: Step 1 involves the interaction module setting model information, loading default values, selecting a group of models, receiving a user's question, and sending that user's question to all large models in the model cluster to receive an answer. Step 2 involves the weight calculation submodule in the answer integration module classifying and positioning the user's questions and calculating the weights for each large model in the model database for the user's questions. Step 3 involves the response integration submodule in the response integration module outputting an integrated response based on the responses provided by each model to the questions submitted by the user, the weights of each model transmitted from the weight calculation submodule, and the user's preference setting parameters. The output integrated response is displayed to the user in real time through the interaction module, and the user's instructions are received to confirm whether it is the final integrated response. If the answer is "yes," step 5 is executed; if the answer is "no," the user's modified preference setting parameters are received, and the response integration submodule re-outputs the integrated response in step 4. Step 5 includes the following steps: the statistical information calculation and visualization submodule in the response integration module calculates and visualizes statistical information for all model responses for each vocabulary in the final integrated response based on the final integrated response and the responses of each large model, and then displays it using the interaction module.
[0051] As those skilled in the art will understand, the above description is merely a preferred embodiment of the invention and is not intended to limit the invention. While the invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical proposals described in the aforementioned embodiments or make equivalent substitutions for some of their technical features. Any modifications, equivalent substitutions, etc., made within the spirit and principles of the invention should be included within the scope of protection of the invention.
Claims
1. An interactive question-and-answer system based on multi-model parallel inference, the interactive question-and-answer system includes an interaction module, a model database, and an answer integration module. The interaction module is used to receive user input, transmit it to the response integration module, and display information. The aforementioned model database is used to store information sets for n large-scale models. The aforementioned response integration module includes a weight calculation submodule, a response integration submodule, and a statistical information calculation and visualization submodule. The aforementioned weight calculation submodule incorporates a model weighting algorithm that is adjusted according to user preferences, and calculates the weights of each large model for the user-submitted question based on the user-submitted question and information about each large model. The aforementioned answer integration submodule incorporates an answer integration algorithm that is adjusted according to user preferences, and calculates the final integrated answer based on the answers provided by each large model to the questions submitted by the user, the weights of each model transmitted from the weight calculation submodule, and the user preference setting parameters. The process by which the aforementioned answer integration submodule calculates the final integrated answer includes the following steps 2.1 to 2.4: In step 2.1, first, the answers output from each large-scale model to the question are read from the model database, and each answer is standardized one by one. In step 2.2, the responses output by each large-scale model are mapped to a high-dimensional semantic space, corresponding semantic embedding vectors are generated, and then the semantic similarity between all response pairs is calculated based on cosine similarity, and a symmetric similarity matrix is constructed as the symbol Sim(i,j), where i and j represent different responses, and Sim(i,j) represents the semantic similarity between response i and response j. In step 2.3, the weights W of each large-scale model are described above. i Integrating semantic similarity, [Math 1] This calculates the final combined weight for each answer. 【number】 represents the combined weight of the i-th response, and n represents the number of responses. 【number】 This is a hyperparameter that adjusts the influence of semantic similarity, and its possible range is [0, 1]. 【Mathematics】 This is the threshold for semantic similarity, 【number】 This is a segment function, In step 2.4, a single large-scale model with contextual understanding, reasoning, and text generation capabilities is selected as the final answer integration model. By inputting the original answers and final integration weights of each large-scale model, a logically consistent and grammatically fluent answer is generated. Furthermore, keywords and semantic similarity thresholds related to the submitted questions entered by the user are received to adjust the semantic tendencies of the final integrated answer. The answer integration submodule further maps each vocabulary in the final integrated answer back to the original answers. The aforementioned statistical information calculation and visualization submodule is characterized by calculating statistical information for all model responses for each vocabulary in the final integrated answer based on the final integrated answer and the answers from each model to the questions submitted by the user, and then visualizing the calculated statistical information before displaying it on the interaction module.
2. The specific execution process of the aforementioned weight calculation submodule includes the following steps 1.1 to 1.5: In step 1.1, the user evaluation scores of all large-scale models are initialized to unbiased evaluation scores, and the evaluation scores of the corresponding large-scale models are adjusted to the user input values based on the user input. In the aforementioned step 1.2, by a machine learning method, each large model tag t in the set tags consisting of all large model tags in the model database i is converted into a vector representation v i , and a model tag vector set tags v is constructed, where tags = {t 1 , t 2 , ..., t i , ..., t x}, tags v = {v 1 , v 2 , ..., v i , ..., v x}, and for all tags corresponding to each large model M i in the model database, a model tag matrix Construct a, where a is the total number of tags. In step 1.3, keywords from the user's question are extracted based on a neural network-based method in natural language processing, classification tags are assigned to the question based on a set of tags consisting of all large-scale models, and k tags with the highest probability are selected according to the ranking of the probability values of the tags in the algorithm result to form a question tag matrix. Construct and the probability value of each tag Record it, In step 1.4, the classification tags of the extracted user questions are matched with the tags held by all large-scale models in the model database, and a similarity matrix S is created. i The matching weights are calculated, the models that successfully match are sorted in descending order of matching degree, and the matching degree is normalized to obtain the matching weight values. Having obtained, In step 1.5, based on the user's selection, the evaluation scores of all large-scale models are constructed as a set using publicly available evaluation scores on the network of large-scale models or user-defined evaluation scores, and then normalized, and the evaluation scores of the models are scored norm = {s 1 , ..., s i , ..., s n } is written, and the weights of the i-th large model for the question currently submitted by the user w i The calculated weight lol i An interactive question-and-answer system based on multi-model parallel inference according to claim 1, characterized in that the data is transmitted to an answer integration submodule and used for subsequent calculations.
3. In step 1.4, the specific matching process involves matching the classification tags of the extracted user questions with the tags held by all large-scale models in the model database. Question Tag Matrix V q And, each large-scale model M in the model database i Model tag matrix For and , a weighted cosine vector similarity matrix S i The weighted cosine vector similarity matrix S is calculated. i In each of the rows, the similarity threshold h set by the user is... s If the above values exist, Model M i The matching was determined to be successful, and the weighted cosine vector similarity matrix S i h s The interactive question-and-answer system based on multi-model parallel inference according to claim 2, characterized in that the number of the above values is recorded as the degree of model matching.
4. In step 1.5, the weights w of each large model i The formula for calculating this is: [Math 2] As shown by, The interactive question-and-answer system based on multi-model parallel inference according to claim 2, characterized in that z is a preference coefficient set by the user and is used to control the overall tendency toward the system's decisions.
5. The interaction module specifically has an initialization phase, an integrated response interaction phase, and a phase after the final integrated response is identified. In the initialization phase, the system receives questions submitted by the user, default model data or custom model data selected by the user, and classifies and displays the model information. In the interaction phase of the integrated response, keywords related to the submitted question entered by the user, semantic similarity thresholds, modified model information, and preference coefficients are received, and the integrated response obtained from the response integration module is displayed, as well as the user's question parameters and information on each large-scale model used in the integrated response. The interactive question-and-answer system based on multi-model parallel inference according to claim 1, characterized in that, in the step after the identification of the final integrated answer, it receives user vocabulary manipulation and displays source trace information and statistical information related to the final integrated answer.
6. In the aforementioned model database, the information set for each large-scale model is a single tuple M i = (name i , source i , parameters i ,score_original i ,score_client i , domain_original i , domain_client i , tags_original i , tags_client i , weight i description i , answer i As shown by, ...) name i This indicates the name of the i-th large-scale model, source i This shows the source of the i-th large-scale model, parameters i This shows the parameter quantities of the i-th large-scale model, and score_original i This shows the public evaluation score on the network of the i-th large model, and score_client i This shows the evaluation score that the user assigned to the i-th large model, and domain_organal i This indicates the application area targeted by the presenter of the i-th large-scale model, domain_client i This indicates the application area where the user has determined, based on their own needs, that the i-th large-scale model is suitable, and tags_original i This indicates the tags assigned by the presenter of the i-th large-scale model, tags_client i This shows the tags that the user assigned to the i-th large-scale model based on their preferences, and weight i This indicates the weight score of the i-th large-scale model, calculated by a model weighting algorithm that is adjusted according to user preferences, and description i This shows the presenter's explanation of the i-th large-scale model, which users can also modify themselves, and answer i The interactive question-and-answer system based on multi-model parallel inference according to claim 1, characterized in that it is used to store answers obtained by the i-th large-scale model through inference in response to questions submitted by the user.
7. Threshold for semantic similarity The initial value is set to 0.
5. Setting a higher value will select more relevant responses during the response integration process. The interactive question-and-answer system based on multi-model parallel inference according to claim 1, characterized in that setting a lower value incorporates more potentially relevant answers during the integration process.
8. The interactive question-and-answer system based on multi-model parallel inference according to claim 1, characterized in that the standardization process specifically includes unifying uppercase and lowercase letters, removing punctuation marks, and removing stop words.
9. An interactive question-and-answer method based on multi-model parallel inference, the interactive question-and-answer method being performed by the interactive question-and-answer system based on multi-model parallel inference described in claim 1, and comprising the following steps 1 to 5: In step 1, the interaction module sets model information, loads default values, selects a group of models, receives a user's question, and sends the user's question to all large models in the model cluster to get an answer. In step 2, the weight calculation submodule in the answer integration module classifies and positions the user's questions and calculates the weights for each large model in the model database for the user's questions. In step 3, the response integration submodule in the response integration module outputs an integrated response based on the responses provided by each model to the questions submitted by the user, the weights of each model transmitted from the weight calculation submodule, and the user's preference setting parameters. In step 4, the interaction module displays the outputted integrated response to the user in real time, receives user instructions, and verifies whether it is the final integrated response. If it is the final integrated response, it executes step 5. If it is not the final integrated response, it receives the user's modified preference setting parameters, and the response integration submodule re-outputs the integrated response. Step 5 is characterized in that the statistical information calculation and visualization submodule in the response integration module calculates and visualizes statistical information for all model responses for each vocabulary in the final integrated response based on the final integrated response and the responses of each large-scale model, and then the interaction module displays the results.
