User medical inquiry quality improvement method and system based on preference optimization

Abstract

The present invention discloses a method and system for improving the quality of user medical inquiries based on preference optimization, and relates to the field of medical artificial intelligence technology, including: collecting medical questions and dividing them into a training set and a test set; constructing a question rewriting model based on a large language model, inputting the training set into the model, and obtaining rewritten medical questions; inputting the questions into a medical question-answering machine to obtain original answers, obtaining doctors' answers, and comparing the two answers for comprehensiveness and accuracy to obtain scores; using a direct preference optimization algorithm and an adaptive momentum optimization algorithm to optimize the question rewriting model according to the scores to obtain a final question rewriting model; inputting a test set into the model to obtain a final rewritten medical question, and inputting the final answer into a medical question-answering machine; the present invention can more quickly and accurately convert complex user medical questions into questions that are more easily understood by medical machines, help different medical machines better understand user questions, and help users obtain more accurate and effective answers.

Description

Technical Field

[0001] The present invention relates to the field of medical artificial intelligence technology, and in particular to a method and system for improving the quality of user medical inquiries based on preference optimization. Background Art

[0002] With the rapid development of artificial intelligence technology, medical question-answering robots have become one of the important technical applications in the medical field. These robots are designed to provide real-time and accurate medical consultation services through natural language processing (NLP) technology. Users can ask these robots health-related questions through text or voice, and the robots need to give professional and accurate medical answers based on the questions.

[0003] Users often have unclear expressions when interacting with robots, including various non-standard expressions, colloquialisms, and non-professional medical terms, which makes it difficult to understand the questions and give accurate answers. In addition, the questions that medical question-answering robots need to answer cover a wide range of medical fields, from self-management of common symptoms to diagnostic advice for complex diseases. Unclear questions make it difficult to accurately link to the corresponding medical knowledge. Summary of the invention

[0004] The present invention provides a method and system for improving the quality of user medical inquiries based on preference optimization, so as to overcome the technical problem that the existing medical question-answering robot cannot accurately handle unclear medical questions, resulting in the medical question-answering robot being unable to give accurate answers.

[0005] In order to achieve the above object, the technical solution of the present invention is:

[0006] A method for improving the quality of user medical inquiries based on preference optimization, comprising:

[0007] S1: Collect users' medical questions and divide the medical questions into a training set and a test set;

[0008] S2: constructing a question rewriting model based on the large language model, inputting the training set into the question rewriting model for rewriting, and obtaining a rewritten medical question;

[0009] S3: inputting the rewritten medical question into the medical question answering machine to obtain the original answer, obtaining the doctor's answer to the rewritten medical question, comparing the two answers in terms of comprehensiveness and accuracy, obtaining a comprehensiveness score and an accuracy score for each answer to the question, and combining the comprehensiveness score and the accuracy score to obtain a final score;

[0010] S4: optimizing the question rewriting model according to the final score using a direct preference optimization algorithm and an adaptive momentum optimization algorithm to obtain an optimized question rewriting model; the optimized question rewriting model is used to obtain a final rewritten medical question;

[0011] S5: Input the test set into the optimized question rewriting model to obtain a final rewritten medical question, and input the final rewritten medical question into a medical question answering machine to obtain a final answer to the question.

[0012] Further, S3 inputs the rewritten medical question into the medical question answering machine to obtain the original answer, obtains the doctor's answer to the rewritten medical question, compares the two answers in terms of comprehensiveness and accuracy, obtains the comprehensiveness score and accuracy score of each answer to the question, and combines the comprehensiveness score and accuracy score to obtain the final score, including:

[0013] S31. Input the rewritten medical question into a medical question-answering machine to obtain an original answer set:

[0014] ,

[0015] in , It indicates a rewritten answer to the oth question of the same question;

[0016] S32. Obtain the doctor's answer to the rewritten medical question, compare the original answer with the doctor's answer in terms of comprehensiveness and accuracy, and obtain a comprehensiveness score and an accuracy score of the original answer to each question:

[0017] The comprehensiveness score of the original answer is shown in formula (1):

[0018] (1)

[0019] in, is the objective fact that the mth question must contain, and the objective fact is determined from the doctor's answer to the question, It means that there are objective facts that must be included in the answer;

[0020] The accuracy score of the original answer is shown in formula (2),

[0021] (2)

[0022] in, Indicates that the original answer contradicts the objective facts that must be included;

[0023] S33, combining the comprehensiveness score with the accuracy score to obtain a final score, as shown in formula (3),

[0024] (3)

[0025] in, is a fixed hyperparameter, represents the mth medical question, Indicates the oth rewrite of the question.

[0026] Further, S4 uses a direct preference optimization algorithm and an adaptive momentum optimization algorithm to optimize the question rewriting model according to the final score to obtain an optimized question rewriting model; the optimized question rewriting model is used to obtain a final rewritten medical question, including:

[0027] S41, randomly generating sample pairs according to the final scores, and using the BT model to perform preference modeling according to the sample pairs to obtain a preference model;

[0028] S42, rewriting the final score using a reinforcement learning algorithm, and substituting it into the preference model formula to obtain a final preference model;

[0029] S43, taking the maximum likelihood of the final preference model as the target, using a direct preference optimization algorithm to calculate the loss function of the final preference model;

[0030] S44. Using an adaptive momentum optimization algorithm, update the parameters in the final preference model according to the loss function, optimize the question rewriting model, and obtain an optimized question rewriting model.

[0031] Furthermore, sample pairs are randomly generated according to the final scores, and preference modeling is performed according to the sample pairs using the BT model to obtain a preference model, including:

[0032] S411: Randomly generate sample pairs according to the final score, each sample pair is , indicating the same user question Different rewrites of, among which, Rating ratio of corresponding answers high, is a positive sample, is a negative sample;

[0033] S412, using the BT model, modeling the preference positive samples according to the sample pairs to obtain a preference model, as shown in formula (4),

[0034] (4)

[0035] in, is the sigmoid activation function.

[0036] Furthermore, the final score is rewritten using a reinforcement learning algorithm and substituted into the preference model to obtain a final preference model, including:

[0037] S421, rewrite the final score using a reinforcement learning algorithm, as shown in formula (5),

[0038] (5)

[0039] in, is the partition function; Represents the problem rewriting model, represents the optimized problem rewriting model, is a fixed hyperparameter; Representation Problem Rewriting Model Input User Question After that, output sample The probability distribution of Represents the optimized problem rewriting model input user problem After that, output sample The probability distribution of

[0040] S422. Substitute formula (5) into formula (4) to obtain the final preference model, as shown in formula (6).

[0041] (6).

[0042] Further, taking the maximum likelihood of the final preference model as the goal, a direct preference optimization algorithm is used to calculate the loss function of the final preference model, including:

[0043] The loss function is shown in formula (7):

[0044] (7)

[0045] in, Represents the loss function.

[0046] Further, an adaptive momentum optimization algorithm is used to update the parameters in the final preference model according to the loss function, and the question rewriting model is optimized to obtain an optimized question rewriting model, including:

[0047] Using an adaptive momentum optimization algorithm, the parameters in the final preference model are updated according to the loss function, as shown in formulas (8)-(13),

[0048] (8)

[0049] (9)

[0050] (10)

[0051] (11)

[0052] (12)

[0053] (13)

[0054] in, Rewrite the parameters of the model for the problem in question, represents the gradient, represents the estimate of the first moment of the gradient, represents the estimated value of the second-order moment of the gradient, represents the weighted average, represents the weighted average, Indicates that the updated problem rewrites the model parameters.

[0055] Furthermore, a system for implementing a method for improving the quality of user medical inquiries based on preference optimization includes:

[0056] A question collection module, used to collect medical questions of users and divide the medical questions into a training set and a test set;

[0057] A question rewriting module uses a question rewriting model with basic question rewriting capabilities and rewrites the user's question to obtain a rewritten question;

[0058] The medical question-answering module, i.e., the medical question-answering machine, is used to collect answers corresponding to the rewritten questions and input the answers into the feedback collection module; and to make answers based on the final rewritten questions and feed them back to the user;

[0059] The feedback collection module is used to collect the standard answers of doctors to each user's medical questions, extract the facts that must be included in the standard answers, evaluate the comprehensiveness and accuracy of the answers of each medical question-and-answer module based on these facts, use the score of the answer as the score of the corresponding rewritten question, and obtain the final score, which is fed back to the question rewriting module;

[0060] The optimization module is used to optimize the question rewriting module according to the final score through a direct preference optimization algorithm to obtain a question rewriting module that meets the doctor's preferences.

[0061] Beneficial effects: The present invention provides a method for improving the quality of user medical inquiries based on preference optimization, which obtains the answers of a question-answering machine and the answers of a doctor, compares them, obtains a comparative score, and uses a direct preference optimization algorithm and an adaptive estimation momentum estimation algorithm to train the original medical rewriting model according to the score to obtain a question rewriting model that meets the standard answer preferences. The question rewriting model and the answering process of the medical question-answering machine can be made more in line with the preferences of users and doctors, and can more quickly and accurately convert complex user medical questions into questions that are easier to be understood by medical machines, which can help different medical machines better understand user questions and help users get more accurate and effective answers. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0063] Figure 1 A method flow chart of the method for improving the quality of user medical inquiries based on preference optimization of the present invention;

[0064] Figure 2 The system block diagram of the user medical query quality improvement system based on preference optimization of the present invention. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] This embodiment provides a method for improving the quality of user medical inquiries based on preference optimization, such as Figure 1 As shown, including:

[0067] S1: Collect users' medical questions and divide the medical questions into a training set and a test set;

[0068] S2: constructing a question rewriting model based on the large language model, inputting the training set into the question rewriting model for rewriting, and obtaining a rewritten medical question;

[0069] S3: inputting the rewritten medical question into the medical question answering machine to obtain the original answer, obtaining the doctor's answer to the rewritten medical question, comparing the two answers in terms of comprehensiveness and accuracy, obtaining a comprehensiveness score and an accuracy score for each answer to the question, and combining the comprehensiveness score and the accuracy score to obtain a final score;

[0070] S4: optimizing the question rewriting model according to the final score using a direct preference optimization algorithm and an adaptive momentum optimization algorithm to obtain an optimized question rewriting model; the optimized question rewriting model is used to obtain a final rewritten medical question;

[0071] S5: Input the test set into the optimized question rewriting model to obtain a final rewritten medical question, and input the final rewritten medical question into a medical question answering machine to obtain a final answer to the question.

[0072] Specifically, firstly, the medical questions of users are collected, and the medical questions are divided into a training set and a test set, so as to obtain the medical questions of users to the greatest extent and improve the accuracy of the model; secondly, a question rewriting model is constructed based on a large language model, and the training set is input into the question rewriting model for rewriting to obtain rewritten medical questions, which can reduce the time of construction and training, make the preliminary rewritten medical questions closer to actual needs, and provide a basis for subsequent optimization; the rewritten medical questions are input into a medical question-answering machine to obtain the original answers, obtain the doctor's answers to the rewritten medical questions, compare the comprehensiveness and accuracy of the two answers respectively, obtain the comprehensiveness score and accuracy score of each answer to the question, and combine the comprehensiveness score and accuracy score to obtain the final score, calculate the comprehensiveness and accuracy of the medical question-answering machine's answers compared with the doctor's answers, and can comprehensively evaluate the two aspects. The accuracy of the formula is considered, so that the medical machine can answer more comprehensively, more accurately, and closer to objective facts in the subsequent optimization process; the direct preference optimization algorithm and the adaptive momentum optimization algorithm are used to optimize the question rewriting model according to the final score to obtain the optimized question rewriting model; the optimized question rewriting model is used to obtain the final rewritten medical question. This solution directly uses human preference data to optimize the language model, thereby avoiding the complexity and instability of reward model fitting and RL optimization in the RLHF process. This method simplifies the preference learning process, reduces the complexity of implementation, and has the potential to improve the efficiency and accuracy of the model in generating outputs based on human preferences; finally, the test set is input into the optimized question rewriting model to obtain the final rewritten medical question, and the final rewritten medical question is input into the medical question-answering machine to obtain the final answer to the question.

[0073] In a specific embodiment, the scheme for collecting medical questions of users and dividing the medical questions into a training set and a test set is:

[0074] Collect the user's medical questions. This embodiment uses an open source data set to obtain the following set of user questions:

[0075] , where the ratio of training set to test set is 3:1.

[0076] In this embodiment, an open source data set is selected to obtain the user's medical problems to the greatest extent and improve the accuracy of the model.

[0077] In a specific embodiment, a question rewriting model is constructed based on a large language model, and the training set is input into the question rewriting model for rewriting, and the solution to the rewritten medical question is:

[0078] In this embodiment, an open-source large language model with medical knowledge is used. The model has a medical rewriting function and a medical question-answering function. The user's question is preliminarily rewritten using the question rewriting model to obtain a set of preliminarily rewritten medical questions: ,

[0079] The mth element is , For the problem Article o of the rewrite.

[0080] This embodiment uses an open-source large language model with medical rewriting and medical question-answering functions, which can reduce the time for construction and training, make the preliminary rewritten medical questions closer to actual needs, and provide a basis for subsequent optimization.

[0081] In a specific embodiment, the rewritten medical question is input into a medical question-answering machine to obtain the original answer, the doctor's answer to the rewritten medical question is obtained, the two answers are compared in terms of comprehensiveness and accuracy, the comprehensiveness score and accuracy score of each answer are obtained, and the final score is obtained by combining the comprehensiveness score and the accuracy score:

[0082] S31. Input the rewritten medical question into a medical question-answering machine to obtain an original answer set:

[0083] ,

[0084] in , It indicates a rewritten answer to the oth question of the same question;

[0085] S32. Obtain the doctor's answer to the rewritten medical question, compare the original answer with the doctor's answer in terms of comprehensiveness and accuracy, and obtain a comprehensiveness score and an accuracy score of the original answer to each question:

[0086] The comprehensiveness score of the original answer is shown in formula (14):

[0087] (14)

[0088] in, The objective facts that must be included in the mth question are decomposed into several facts by the large language model, and then the objective facts contained in the answer are screened out by the doctor. It means that there are objective facts that must be included in the answer;

[0089] The accuracy score of the original answer is shown in formula (15),

[0090] (15)

[0091] in, Indicates that the original answer contradicts the objective facts that must be included;

[0092] S33, combining the comprehensiveness score with the accuracy score to obtain a final score, as shown in formula (16),

[0093] (16)

[0094] in, is a fixed hyperparameter.

[0095] The medical question-answering machine used in this embodiment is an open source medical question-answering model. In this solution, the comprehensiveness and accuracy of the answers of the medical question-answering machine are calculated compared with the answers of doctors. The accuracy of the two methods can be comprehensively considered, and in the subsequent optimization process, the medical machine can answer more comprehensively, more accurately, and closer to objective facts.

[0096] In a specific embodiment, the direct preference optimization algorithm and the adaptive momentum optimization algorithm are used to optimize the question rewriting model according to the final score to obtain an optimized question rewriting model; the optimized question rewriting model is used to obtain the final rewritten medical question solution:

[0097] S41, randomly generate sample pairs according to the final score, and use the BT model to perform preference modeling according to the sample pairs to obtain a preference model:

[0098] S411: Randomly generate sample pairs according to the final score, each sample pair is , indicating the same user question Different rewrites of, among which, Rating ratio of corresponding answers high, is a positive sample, is a negative sample;

[0099] S412, using the BT model, modeling the preference positive samples according to the sample pairs to obtain a preference model, as shown in formula (17),

[0100] (17)

[0101] in, is the sigmoid activation function;

[0102] S42. Rewrite the final score using a reinforcement learning algorithm and substitute it into the preference model to obtain a final preference model:

[0103] S421. Rewrite the final score using a reinforcement learning algorithm, as shown in formula (18):

[0104] (18)

[0105] in, is the partition function; Represents the problem rewriting model, represents the optimized problem rewriting model, is a fixed hyperparameter; Representation Problem Rewriting Model Input User Question After that, output sample The probability distribution of Represents the optimized problem rewriting model input user problem After that, output sample The probability distribution of

[0106] S422. Substitute formula (18) into formula (17) to obtain the final preference model, as shown in formula (19).

[0107] (19)

[0108] S43, taking the maximum likelihood of the final preference model as the target, using the direct preference optimization algorithm to calculate the loss function of the final preference model, the loss function is shown in formula (20),

[0109] (20)

[0110] S44, using an adaptive momentum optimization algorithm, updating the parameters in the final preference model according to the loss function, optimizing the question rewriting model, and obtaining an optimized question rewriting model:

[0111] Using an adaptive momentum optimization algorithm, the parameters in the final preference model are updated according to the loss function, as shown in formulas (21)-(26),

[0112] (twenty one)

[0113] (twenty two)

[0114] (twenty three)

[0115] (twenty four)

[0116] (25)

[0117] (26)

[0118] in, Rewrite the parameters of the model for the problem in question, represents the gradient, represents the estimate of the first moment of the gradient, represents the estimated value of the second-order moment of the gradient, represents the weighted average, represents the weighted average, Indicates that the updated problem rewrites the model parameters.

[0119] In this scheme, the BT model is a classic human preference model. It is a probabilistic model used to predict the results of two competitors (such as individuals or teams). It is often used to process paired comparison data and is usually used to estimate and compare the relative capabilities of individuals or projects. This model is suitable for unordered data. In this scheme, it only relies on the difference between two different answers, so the BT model is used for preference modeling; in this scheme, the reinforcement learning algorithm PPO is used to rewrite the preference score. The PPO algorithm is a policy optimization algorithm widely used in reinforcement learning. It belongs to a type of policy gradient method, which aims to stabilize the training process by limiting the difference between the new strategy and the old strategy. PPO introduces a technique called "proximal policy optimization" to avoid excessive policy updates, thereby reducing instability during training. and sample complexity; the DPO algorithm is used to calculate the loss function. The key to the DPO algorithm is to directly use human preference data to optimize the language model, thereby avoiding the complexity and instability of reward model fitting and RL optimization in the RLHF process. This method simplifies the preference learning process, reduces the complexity of implementation, and has the potential to improve the efficiency and accuracy of the model in generating outputs based on human preferences; finally, the adaptive momentum optimization algorithm is used to optimize the parameters in the rewritten model according to the loss function. The adaptive momentum optimization algorithm combines the ideas of momentum method and adaptive learning rate to achieve fast and stable update of model parameters. In the field of deep learning, the adaptive momentum optimization algorithm has been widely used and has achieved remarkable optimization effects.

[0120] This embodiment also provides a user medical inquiry quality improvement system based on preference optimization, such as Figure 2 As shown, including:

[0121] A question collection module, used to collect medical questions of users and divide the medical questions into a training set and a test set;

[0122] A question rewriting module uses a question rewriting model with basic question rewriting capabilities and rewrites the user's question to obtain a rewritten question;

[0123] The medical question-answering module, i.e., the medical question-answering machine, is used to collect answers corresponding to the rewritten questions and input the answers into the feedback collection module; and to make answers based on the final rewritten questions and feed them back to the user;

[0124] The feedback collection module is used to collect the standard answers of doctors to each user's medical questions, extract the facts that must be included in the standard answers, evaluate the comprehensiveness and accuracy of the answers of each medical question-and-answer module based on these facts, use the score of the answer as the score of the corresponding rewritten question, and obtain the final score, which is fed back to the question rewriting module;

[0125] The optimization module is used to optimize the question rewriting module according to the final score through a direct preference optimization algorithm to obtain a question rewriting module that meets the doctor's preferences.

[0126] Specifically, first, a question collection module is used to collect users' questions, a question rewriting module is used to preliminarily rewrite users' questions to obtain rewritten questions, and then the rewritten questions are input into the medical question and answer module to obtain answers to the rewritten questions; the answers to the rewritten questions are input into the feedback collection module, and the doctor's standard answers to each question are collected through the feedback collection module, the comprehensiveness and accuracy of the answers of the medical question and answer module are evaluated, and a final score is obtained; finally, an optimization module is used to optimize the question rewriting module according to the final score through a direct preference optimization algorithm to obtain a question rewriting module that meets the doctor's preferences, the question is input into the optimized question rewriting module to obtain the final question, and then the question is input into the medical question and answer module to obtain the final answer.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for improving the quality of user medical inquiries based on preference optimization, characterized in that: include: S1: Collect users' medical questions and divide the medical questions into a training set and a test set; S2: constructing a question rewriting model based on the large language model, inputting the training set into the question rewriting model for rewriting, and obtaining a rewritten medical question; S3: inputting the rewritten medical question into the medical question answering machine to obtain the original answer, obtaining the doctor's answer to the rewritten medical question, comparing the original answer with the doctor's answer to the rewritten medical question in terms of comprehensiveness and accuracy, obtaining a comprehensiveness score and an accuracy score of the original answer, and combining the comprehensiveness score and the accuracy score to obtain a final score of the original answer; S4: using a direct preference optimization algorithm and an adaptive momentum optimization algorithm to optimize the question rewriting model according to the final score to obtain an optimized question rewriting model; the optimized question rewriting model is used to obtain a final rewritten medical question; the optimized question rewriting model is a question rewriting model that meets the doctor's preferences: S41, randomly generate sample pairs according to the final score, and use the BT model to calculate the sample q i Perform preference modeling to obtain a preference model; S42. Rewrite the final score using a reinforcement learning algorithm and substitute it into the preference model to obtain a final preference model: S421, rewrite the final score using a reinforcement learning algorithm, as shown in formula (5), Among them, Z(x) is the partition function; R0 represents the problem rewriting model, R1 represents the optimized problem rewriting model, and β is a fixed hyperparameter; R0(q g |q i ) represents the question rewriting model input user question q i After that, the output sample q g The probability distribution of R1(q g |q i ) represents the optimized problem rewriting model input user question q i After that, the output sample q g The probability distribution of S422. Substitute formula (5) into formula (4) to obtain the final preference model, as shown in formula (6). S43, taking the maximum likelihood of the final preference model as the target, using a direct preference optimization algorithm to calculate the loss function of the final preference model; S44, using an adaptive momentum optimization algorithm to update the parameters in the final preference model according to the loss function, optimizing the question rewriting model, and obtaining an optimized question rewriting model; S5: Input the test set into the optimized question rewriting model to obtain a final rewritten medical question, input the final rewritten medical question into the medical question-answering machine to obtain a final answer to the question, the final rewritten medical question is a medical question that meets the doctor's preferences, and the final answer is a medical answer obtained based on the rewritten medical question.

2. The method for improving user medical inquiry quality based on preference optimization according to claim 1, characterized in that: S3 inputs the rewritten medical question into the medical question answering machine to obtain the original answer, obtains the doctor's answer to the rewritten medical question, compares the two answers in terms of comprehensiveness and accuracy, obtains the comprehensiveness score and accuracy score of each answer to the question, and combines the comprehensiveness score and accuracy score to obtain the final score, including: S31, input the rewritten medical question into the medical question answering machine to obtain the original answer set A, A={A1,A2,...,A m }, where A m ={a m1 , a m2 , ..., a mo }, a mo It indicates a rewritten answer to the same question No. o; S32. Obtain the doctor's answer to the rewritten medical question, compare the original answer with the doctor's answer in terms of comprehensiveness and accuracy, and obtain a comprehensiveness score and an accuracy score of the original answer to each question: The comprehensiveness score of the original answer is shown in formula (1): Among them, Must_Have_m is the objective fact that must be included in the mth question, and the objective fact is determined from the doctor's answer to the question, and entails means that the objective fact that must be included appears in the answer; The accuracy score of the original answer is shown in formula (2): s comt (a mo )=|{x∈Must_Have_m|a mo contradicts x}| (2) Among them, contradicts means that the original answer contradicts the objective facts that must be included; S33, combining the comprehensiveness score with the accuracy score to obtain a final score, as shown in formula (3), s(q mo |q m )=s comp (a mo )-αs comt (a mo ) (3) Where α is a fixed hyperparameter, q m represents the mth medical question, q mo It indicates the oth rewrite of the question; The doctor's answer to the rewritten medical question contains objective facts about the rewritten medical question. The final score is used to compare the comprehensiveness and accuracy of the original answer with the doctor's answer, and the question rewriting model is optimized based on the final score to obtain a question rewriting model that meets the doctor's preferences.

3. The method for improving user medical inquiry quality based on preference optimization according to claim 1, characterized in that: Randomly generate sample pairs according to the final score, and use the BT model to perform preference modeling according to the sample pairs to obtain a preference model, including: S411, randomly generate sample pairs according to the final score, each sample pair is {q g ,q b }, indicating the same user question q i Different rewrites of , where q g The score ratio of the corresponding answer is q b High, Q g is a positive sample, q b is a negative sample; S412, using the BT model, modeling the preference positive samples according to the sample pairs to obtain a preference model, as shown in formula (4), p(q g >q b |q i )=σ(exp(s mo (what g |q i ))-exp(s mo (what b |q i ))) (4) Among them, σ is the sigmoid activation function.

4. The method for improving user medical inquiry quality based on preference optimization according to claim 1, characterized in that: Taking the maximum likelihood of the final preference model as the goal, a direct preference optimization algorithm is used to calculate the loss function of the final preference model, including: The loss function is shown in formula (7): Among them, L DPO represents the loss function.

5. The method for improving user medical inquiry quality based on preference optimization according to claim 1, characterized in that: Using an adaptive momentum optimization algorithm, updating the parameters in the final preference model according to the loss function, optimizing the question rewriting model, and obtaining an optimized question rewriting model, including: Using an adaptive momentum optimization algorithm, the parameters in the final preference model are updated according to the loss function, as shown in formulas (8)-(13), m t =β1m t-1 +(1-β1)g t (9) Among them, θ t Rewriting the parameters of the model for the problem, g t represents the gradient, m t represents the estimated value of the first-order moment of the gradient, v t represents the estimated value of the second-order moment of the gradient, represents the weighted average, represents the weighted average, θ t+1 Indicates that the updated problem rewrites the model parameters.

6. A system for implementing the method for improving the quality of user medical inquiries based on preference optimization as described in any one of claims 1 to 5, characterized in that: include: A question collection module, used to collect medical questions of users and divide the medical questions into a training set and a test set; A question rewriting module uses a question rewriting model with basic question rewriting capabilities and rewrites the user's question to obtain a rewritten question; The medical question-answering module, i.e., the medical question-answering machine, is used to collect answers corresponding to the rewritten questions and input the answers into the feedback collection module; and to make answers based on the final rewritten questions and feed them back to the user; The feedback collection module is used to collect the standard answers of doctors to each user's medical questions, extract the facts that must be included in the standard answers, evaluate the comprehensiveness and accuracy of the answers of each medical question-and-answer module based on these facts, use the score of the answer as the score of the corresponding rewritten question, and obtain the final score, which is fed back to the question rewriting module; The optimization module is used to optimize the question rewriting module according to the final score through a direct preference optimization algorithm to obtain a question rewriting module that meets the doctor's preferences.

Images