Multi-objective Optimization Method for Language Model-based Intelligent Question Answering System

Abstract

The present invention discloses a multi-objective optimization method for a language model intelligent question-answering system, which enables the language model to maximize the user experience and minimize the number of interactions to perform online learning and decision-making tasks under incomplete information. The language model intelligent question-answering system consists of multiple language generation models and a decision model. The language generation model is responsible for generating several alternative answers, and the decision model is responsible for selecting two of the answers as feedback. First, an initial strategy is designed to explore the performance of the language generation model, and the language generation model is simply sorted using user feedback. For the design of the decision model, a decision method that is adaptive to dynamic sorting is used, supplemented by a pair-by-pair elimination method to gradually eliminate language generation models that do not meet user needs. Compared with existing question-answering model optimization technologies, the present invention can not only ensure that the language generation model that best meets user needs is used to provide answers as much as possible to maximize user experience, but can also additionally ensure that the number of interactions required to find the language generation model is minimized.

Description

Technical Field

[0001] The present invention relates to a multi-objective optimization method for a language model intelligent question-answering system, belonging to the technical field of online learning optimization of language models. Background Art

[0002] With the rapid development of artificial intelligence and natural language processing technologies, language model-based intelligent question-answering systems are playing an increasingly important role in our daily lives and work. From personal assistants to customer service robots to information retrieval tools in academic research, language model-based intelligent question-answering systems are being widely adopted in various fields. However, in practical applications, language models often focus solely on answer accuracy. While this single-objective optimization approach can provide accurate answers to a certain extent, it ignores the user's needs for other performance metrics in real-world applications, such as reducing the number of interactions and improving the speed and relevance of answers. This one-sided pursuit of accuracy can lead to a poor user experience, especially in application scenarios that require efficient communication.

[0003] However, users' goal isn't just to obtain high-quality answers; minimizing the number of interactions required is equally important in certain scenarios. Existing systems often fail to balance these two requirements, resulting in lengthy user interactions. This not only increases user time but can also delay decision-making due to redundant or irrelevant answers, reducing the system's practicality and user experience. For example, in online medical consultation systems, users not only want accurate answers but also want to quickly obtain the information they need to make timely decisions. Summary of the Invention

[0004] Purpose of the Invention: Existing language model question-answering systems typically only consider answer accuracy to improve the user interaction experience, which greatly limits the practicality of existing language model question-answering systems. To address the problems and shortcomings of the existing technology, this invention considers using online learning to design an online decision-making method for language models, which can simultaneously ensure the maximum user interaction experience and minimize the number of interactions.

[0005] Technical Solution: A multi-objective optimization method for a language model-based intelligent question-answering system. The system includes multiple language generation models and a decision model. The generation model is responsible for generating candidate answers, while the decision model is responsible for selecting and returning an answer to the user. The multi-objective optimization method aims to maximize the user interaction experience while minimizing the number of interactions.

[0006] First, the language generation model and decision model are initialized. When a user asks a question, the language generation model generates several candidate answers based on the question. This process is determined by the pre-trained language generation model. Next, the decision model makes a decision, selecting two answers based on the current internal state of the decision model and returning them to the user. This process requires exploring unknown language generation model information or leveraging known language generation model information. The user then selects the better of the two answers. Based on this feedback, the decision model updates its internal state, a process that requires addressing potentially inaccurate user feedback. Finally, the language generation model updates its set of candidate answers, a process that requires addressing imbalanced information across the language generation models.

[0007] A multi-objective optimization method for language model intelligent question answering systems, which needs to deal with exploring unknown language generation model information or utilizing known language generation model information, potential inaccurate user feedback, and imbalanced information among language generation models.

[0008] Initialize the language generation model and decision model:

[0009] Set the number N of language generation models.

[0010] Initialize an ensemble of language generation models but

[0011] Initialize the weight vector S=0 of N language generation models N .

[0012] Initialize the qualification vector Q of N language generation models = 1 N .

[0013] Initialize the historical feedback matrix W = 0 N×N .

[0014] Initialize the round number r=1.

[0015] The specific steps for the decision model to make a decision and explore unknown language generation model information or use known language generation model information are as follows:

[0016] Repeat the following steps until

[0017] Select the first language generation model that meets the conditions: the sequence number of the language generation model is i, and the decision model selection weight S i >0 and qualification Q i =1 a language generation model, if there is no model that meets the conditions, then Select one at random.

[0018] Calculate the set except i Historical information of the language generation model in where k=1,...,i-1,i+1,...,N,W m，k Represents the total number of times that user feedback model m is better than model k in all historical feedback.

[0019] By probability Sampling obtains a language generation model, where k=1, ..., i-1, i+1, ..., N. The serial number of the language generation model is j, and its corresponding answer is taken as the second answer.

[0020] The specific steps for the decision model to update its internal state and respond to potentially inaccurate user feedback are:

[0021] Repeat the following steps until the weight S of a language generation model among language generation model i and language generation model j is -r:

[0022] User asks a question.

[0023] Return the answers generated by the two language generation models numbered i and j to the user.

[0024] If the user feedback model i's answer is better than model j, then W i，j =W i，j +1, S i =S i +1, S j =S j -1; otherwise W j，i =W j，i +1, S i =S i -1, S j =S j +1.

[0025] The language generation model updates its candidate answer set. The specific steps to deal with the imbalanced information of each language generation model are as follows:

[0026] If S i =-r,Q i =0; if S j =-r,Q j =0

[0027] For language generation model i and model j, let the maximum acceptable failure probability be δ. renew like Update Update

[0028] If the qualification vector Q of all language generation models is 0N ,but Language Generation Model

[0029] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the multi-objective optimization method for the language model intelligent question-answering system as described above is implemented.

[0030] A computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the steps of the multi-objective optimization method for a language model intelligent question-answering system as described above.

[0031] Beneficial Effects: Compared to existing technologies, the multi-objective optimization method for language model intelligent question-answering systems provided by this invention offers the following advantages: Its design not only focuses on answer accuracy but also optimizes the number of interactions, achieving an effective balance between accuracy and user interaction burden. By introducing a dynamic learning mechanism, enhancing noise immunity, and performing information balancing, this invention makes language model intelligent question-answering systems more flexible in practical applications, significantly improving the system's practicality and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 This is a flowchart of the language model intelligent question answering system in an embodiment of the present invention;

[0033] Figure 2 is a flow chart of a method for exploring unknown language generation model information or utilizing known language generation model information in an embodiment of the present invention;

[0034] Figure 3 is a flow chart of a method for dealing with potentially inaccurate user feedback in an embodiment of the present invention;

[0035] Figure 4 4 is a flow chart of a method for dealing with imbalanced information of various language generation models in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The present invention is further illustrated below with reference to specific examples. It should be understood that these examples are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, modifications of various equivalent forms of the present invention made by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0037] Take, for example, intelligent medical question-and-answer systems. Existing intelligent medical question-and-answer systems typically prioritize answer accuracy to enhance the user experience. This significantly limits their practicality. For example, in intelligent medical question-and-answer systems, users not only want accurate answers but also want to quickly obtain the information they need to make timely decisions. However, existing systems often optimize for only a single objective, resulting in a poor user experience.

[0038] The multi-objective optimization technology for the language model intelligent question answering system disclosed in the present invention can effectively address the above problems, thereby greatly expanding the practicality of the intelligent question answering system.

[0039] A multi-objective optimization method for language model-based intelligent question-answering systems. These systems consist of multiple language generation models and a decision model. The generation model generates candidate answers, while the decision model selects and returns the answer to the user. Multi-objective optimization aims to maximize the user interaction experience while minimizing the number of interactions.

[0040] First, the language generation model and decision model are initialized. When a user asks a question in the intelligent medical question-answering system, the language generation model first generates several candidate answers based on the knowledge base and previous experience. This process is determined by the pre-trained language generation model. Next, the decision model makes a decision. Based on the current internal state of the decision model, it selects two answers that are most likely to be helpful to the user and returns them to the user. This process requires exploring unknown language generation model information or leveraging known language generation model information. The user then selects the answer they feel is better based on the selected answer. Based on this feedback, the decision model updates its internal state. This process also requires addressing potential inaccurate user feedback. Finally, the language generation model updates its set of candidate answers based on the preferences corresponding to the user's feedback to answer the user's subsequent medical consultation questions. This process requires addressing imbalanced information among the language generation models.

[0041] Multi-objective optimization technology for language model intelligent question answering systems, which requires dealing with exploring unknown language generation model information or utilizing known language generation model information, potential inaccurate user feedback, and unbalanced information among language generation models.

[0042] Initialize the language generation model and decision model:

[0043] Set the number N of language generation models.

[0044] Initialize an ensemble of language generation models but

[0045] Initialize the weight vector S=0 of N language generation modelsN .

[0046] Initialize the qualification vector Q of N language generation models = 1 N .

[0047] Initialize the historical feedback matrix W = 0 N×N .

[0048] Initialize the round number r=1.

[0049] The decision model makes a decision. The specific steps for exploring unknown language generation model information or using known language generation model information are as follows:

[0050] Repeat the following steps until the collection

[0051] 1. Select the first language generation model that meets the conditions: the sequence number of the language generation model is i, and the decision model selection weight is S i >0 and qualification Q i =1 a language generation model, if there is no model that meets the conditions, then Select one at random.

[0052] 2. Calculate the set other than i Historical information of the language generation model in Among them, k=1,…,i-1,i+1,…,N.

[0053] 3. Based on probability Sampling obtains a model, where k = 1,…,i-1,i+1,…,N. The serial number of the language generation model is j, and its corresponding answer is taken as the second answer.

[0054] The specific steps for the decision model to update its internal state and respond to potentially inaccurate user feedback are:

[0055] Repeat the following steps until the weight S of a language generation model in model i and model j is -r:

[0056] 1. The user asks a question.

[0057] 2. Return the answers generated by the two language generation models numbered i and j to the user.

[0058] 3. If the user feedback model i’s answer is better than model j, then W i,j =W i,j +1, S i =S i +1,S j =S j -1; otherwise

[0059] Wj,i =W j,i +1, S i =S i -1,S j =S j +1

[0060] 4. The language generation model updates its candidate answer set. The specific steps to deal with the imbalanced information of each language generation model are as follows:

[0061] 5. If S i =-r,Q i =0; if S j =-r,Q j =0

[0062] 6. For model i and model j, if renew like Update Update

[0063] If the qualification vector Q of all language generation models is 0 N ,but

[0064] Next, the specific implementation of the present invention will be described in detail using the intelligent medical question-answering system as an example.

[0065] Figure 1 The following is a workflow diagram of the intelligent medical question-and-answer system proposed in this invention. The system consists of multiple language generation models and a decision model. The language generation models are responsible for generating candidate answers, while the decision model is responsible for selecting and returning the answer to the user. Step 1: Initialize N candidate language generation models, each with different medical knowledge and expertise. Step 2: Determine whether only one candidate language model remains. If so, the process ends, indicating that the system has determined the model most suitable for the current medical scenario. Step 3: Each language model generates several candidate answers based on the user's medical question, such as diagnostic recommendations or treatment plans. Step 4: The decision model, based on the current decision model's internal state, selects two medical recommendations that best meet the user's needs from all the candidate answers and returns them to the user. Step 5: Based on the quality of the two returned medical recommendations, the user selects the one they are most satisfied with and provides feedback to the system. Step 6: Based on the user's feedback, the decision model updates its internal state. Step 7: The decision model determines whether the candidate answer set needs to be updated. If not, the system returns to step 3 to answer the user's next question. If so, the system proceeds to step 8: In step 8, the candidate model is updated based on the user feedback and internal state, ensuring that subsequent medical recommendations are more tailored to the user's needs.

[0066] Figure 2This is a workflow diagram of the decision model proposed by the present invention making decisions in the intelligent medical question-answering system and selecting a model that meets the conditions from all language generation models. For example, at a certain moment, after a user asks a medical question to the intelligent medical system, several language generation models in the system have generated multiple alternative medical suggestions. At this time, the decision model should select the two best suggestions from these alternative suggestions as output and return them to the user. The first selected generation model is numbered i, and step 41 of the decision model establishes the judgment condition with weight S i >0 and qualification Q i =1, which means that the model has sufficient credibility and qualifications. Step 42 determines whether there is a language generation model that meets the conditions. If so, go to step 43, and step 43 selects the medical advice generated by the model that meets the conditions as the first medical advice; if there is no model that meets the conditions in step 42, go to step 44, and step 44 selects the medical advice generated by the model that meets the conditions from the set. After selecting the first answer, step 45 calculates the set except i. Historical information of the language generation model in Where k = 1, ..., i-1, i+1, ..., N. The purpose of this step is to evaluate the relative merits of other models in providing medical advice in history. Step 46 is to use the probability of The second model j is sampled, where k=1, ..., i-1, i+1, ..., N, and the answer generated by the model is selected as the second medical suggestion. Finally, step 47 returns the two selected medical suggestions to the user.

[0067] Figure 3 This is a workflow diagram of the decision model proposed by the present invention updating its internal state in the intelligent medical question-answering system. Take the case where a user at a certain moment receives two medical suggestions returned by the intelligent medical system and selects one of the suggestions that better meets his needs. At this time, the decision model should update its internal state based on user feedback. The serial numbers of the models corresponding to the two selected answers are recorded as i and j. Step 61 determines whether the weight S of a language generation model in model i and model j is -r. If so, the process ends. Step 62 jumps to step 63 and step 65 respectively based on user feedback that i is better than j or j is better than i. If i is better than j, step 63 updates the historical feedback matrix W i，j =W i，j +1, step 64 updates the weight vector S i =S i +1, S j =S j -1; if j is better than i, then step 65 updates the historical feedback matrix W j，i =W j，i +1, step 66 updates the weight vector Si =S i -1, S j =S j +1.

[0068] Figure 4 This is the workflow diagram of the intelligent medical question-answering system proposed by the present invention to update its candidate answer set. Step 81 determines which model, model i or j, will be temporarily suppressed as a candidate after the internal state is updated. For example, if S i = -r, then step 82 sets the qualification Q of model i i =0, indicating that the model will not participate in answer generation in the next question; if S j = -r, then step 83 sets the qualification Q of model j j = 0. Step 84 determines whether the candidate model needs to be updated. The specific determination is based on two conditions: Condition 1: If condition 1 is met, step 85 updates the selected language model to Condition 2: If condition 2 is met, step 86 updates the selected language model to Step 87 determines whether the qualification vectors of all language generation models are 0, that is, condition 3: Q = 0 N If condition 3 is met, step 88 will initialize the qualifications of all remaining candidate language models, i.e. Ensure that the smart medical system can continue to provide effective medical advice to users.

[0069] Obviously, those skilled in the art should understand that the various steps of the multi-objective optimization method of the intelligent medical question-answering system of the above-mentioned embodiment of the present invention can be implemented using a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. Alternatively, they can be implemented using program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be performed in a different order than herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the embodiments of the present invention are not limited to any specific combination of hardware and software.

Claims

1. A multi-objective optimization method for a language model intelligent question answering system, characterized in that: The language model intelligent question-answering system includes multiple language generation models and a decision model; the language generation model is responsible for generating alternative answers, and the decision model is responsible for selecting answers and returning them to the user; the multi-objective optimization refers to maximizing the user interaction experience and minimizing the number of interactions; First, the language generation model and decision model are initialized. When a user asks a question, the language generation model generates several alternative answers based on the question. This process is determined by the pre-trained language generation model. Next, the decision model makes a decision and selects two answers based on the current internal state of the decision model and returns them to the user. This process requires exploring unknown language generation model information or leveraging known language generation model information. The user then selects the better of the two answers. Based on this feedback, the decision model updates its internal state. This process also requires dealing with potentially inaccurate user feedback. Finally, the language generation model updates its set of alternative answers. This process also requires dealing with imbalanced information among the language generation models. The initialization language generation model and decision model include: Set the number N of language generation models; Initialize an ensemble of language generation models but Initialize the weight vector S=0 of N language generation models N ; Initialize the qualification vector Q of N language generation models = 1 N ; Initialize the historical feedback matrix W = 0 N×N ; Initialize the round number r = 1; The specific steps for the decision model to make a decision and explore unknown language generation model information or use known language generation model information are as follows: Repeat the following steps until Select the first language generation model that meets the conditions: the sequence number of the language generation model is i, and the decision model selection weight S i >0 and qualification Q i =1 a language generation model, if there is no model that meets the conditions, then Randomly select one; Calculate the set except i Historical information of the language generation model in where k=1,…,i-1,i+1,…,N,W M,k Indicates the total number of times that user feedback model m is better than model k in all historical feedback; By probability Sampling obtains a language generation model, where k = 1,…,i-1,i+1,…,N. The sequence number of the language generation model is j, and its corresponding answer is the second answer; The specific steps for the decision model to update its internal state and respond to potentially inaccurate user feedback are: Repeat the following steps until the weight S of a language generation model among language generation model i and language generation model j is -r: Users raise questions; Return the answers generated by the two language generation models numbered i and j to the user; If the user feedback model i's answer is better than model j, then W i,j =W i,j +1, S i =S i +1,S j =S j -1; otherwise W j,i =W j,i +1, S i =S i -1,S j =S j +1.

2. The multi-objective optimization method for the language model intelligent question answering system according to claim 1, characterized in that: The language generation model updates its candidate answer set. The specific steps to deal with the imbalanced information of each language generation model are as follows: If S i =-r,Q i =0; if S j =-r,Q j =0 For language generation model i and model j, if renew like renew If the qualification vector Q of all language generation models is 0 N ,but Language Generation Model 3. A computer device, characterized in that: The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the multi-objective optimization method for the language model intelligent question-answering system as described in any one of claims 1 to 2 is implemented.

4. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, the steps of the multi-objective optimization method for a language model intelligent question answering system as described in any one of claims 1-2 are implemented.

Images