Construction method of medical model and medical inquiry system

Abstract

The invention provides a construction method of a medical model and a medical inquiry system, and belongs to the artificial intelligence technology and the medical industry, and the method comprises the steps: collecting and preprocessing data, and loading a pre-training model; finely adjusting the model; inputting a patient question to the medical model, generating a plurality of answers from the medical model, performing medical feedback and labeling on each answer by a professional doctor, and training based on the medical feedback and labeling of the doctor to obtain a medical reply effect evaluation model; model parameters are updated by combining reinforcement learning with a medical reply effect evaluation model, so that the ability of the medical model is improved from the perspective of profession, the model is more humanized, and the model performance is improved; evaluating model performance; the fine-tuned model is evaluated using a validation set and a test set to determine whether its performance meets requirements. According to the method, reinforcement learning improvement is carried out through fine adjustment of the large model by professional medical data and training of the medical reply effect evaluation model, and finally a medical large model which is more accepted by a physician is obtained and applied.

Description

Technical Field

[0001] The present invention relates to artificial intelligence technology and the medical industry, and in particular to a method for constructing a medical model and a medical consultation system. Background Art

[0002] Most of the popular big models on the market are general-purpose big models with a large number of parameters, and the effect is not ideal when applied to professional fields. By fine-tuning the general big model with professional knowledge in the professional field, the effect is good in the professional field and the training cost is not high. In addition, because medical data has high requirements for security, it cannot be provided to external models for training. Therefore, it is particularly important to fine-tune the big model based on internal medical data and train your own exclusive medical big model. Therefore, fine-tuning technology has been widely used in the implementation of big models in the medical industry.

[0003] Among them, the application with publication number CN116579402A provides a solution, and the fine-tuning method, device, equipment and storage medium of the large model in the banking business are mainly methods for fine-tuning the large model for the banking business. Receive occasion requests in the banking business, wherein the occasion requests include the data types that need to be processed in the banking business; select the corresponding large model and build a fine-tuning auxiliary model according to the data type and occasion request, and connect the fine-tuning auxiliary model to the large model; receive labeled data samples, input the large model and the fine-tuning auxiliary model, fix the first parameter of the large model, and train the fine-tuning auxiliary model; when the large model and the fine-tuning auxiliary model converge, send the second parameter of the fine-tuning auxiliary model to the large model; fine-tune the large model according to the second parameter.

[0004] The application with publication number CN116629235A provides a solution, a method, device, electronic device and medium for fine-tuning a large-scale pre-trained language model: obtain a pre-trained large-scale language model, use a task instruction template to transform the input data of the corresponding task, obtain input text and output text for fine-tuning training, and use the output text as the correct answer corresponding to the input text; input the input text into the pre-trained large-scale language model for prediction, and obtain the prediction result output by the pre-trained large-scale language model; based on the prediction result and the correct answer corresponding to the input text, calculate the loss function, and use the loss function to update the parameters of the pre-trained large-scale language model until the pre-trained large-scale language model converges.

[0005] The application with publication number CN116595131A provides a method system for medical question and answer using a large language model: obtaining the user's medical conversation history content; inputting the medical conversation history content and a first prompt instruction into the large language model, and obtaining multiple first responses based on the diversified sampling decoding of the large language model; inputting the medical conversation history content and multiple second prompt instructions into the large language model, and obtaining multiple second responses corresponding to each of the second prompt instructions; inputting the medical conversation history content, the multiple first responses and the multiple second responses into the large language model, generating a reply content of the medical conversation and sending it to the user.

[0006] Since there is too much professional knowledge in the medical field, and the general domain knowledge of LLMs often cannot meet such specialized needs, if it is directly used for intelligent diagnosis, it is very likely to lead to inaccuracies in diagnostic accuracy, drug recommendations, and medical advice, and even endanger the patient's life. Therefore, it is very necessary to input professional medical domain knowledge and diagnostic case data into the big model for specialized learning.

[0007] Most of the popular large models on the market are general-purpose large models with a large number of parameters, and the effect of applying them to the medical field is not ideal. In addition, because medical data has high requirements for security, it cannot be provided to external models for training. Therefore, it is particularly important to fine-tune the large model based on internal medical data and train your own exclusive medical large model. Therefore, fine-tuning technology is widely used in the implementation of large models in the medical industry, but the professionalism and humanization level of the model after fine-tuning is unstable. Reinforcement learning training can be added to enhance the professionalism and humanization of the model and "align" professional physicians. Therefore, fine-tuning technology is widely used in the implementation of large models in the medical industry. Summary of the invention

[0008] To solve the above problems, the present invention proposes a method for constructing a medical model and a medical consultation system. By fine-tuning the large model with professional medical data and training the medical response effect evaluation model for reinforcement learning improvement, a medical large model that is more recognized by doctors is finally obtained and has a great difference in application.

[0009] The present invention proposes a method for constructing a medical model, comprising the following steps:

[0010] Step 1: Data collection and preprocessing, including: collecting relevant data in the medical field, including medical records, symptom descriptions, medical knowledge, and performing data cleaning and labeling;

[0011] Step 2: Load the pre-trained model, including: loading the pre-trained open source ChatGLM-6B large model and corresponding parameters onto the computing device;

[0012] Step 3: Fine-tune the model to obtain the medical model, including: fine-tuning the ChatGLM-6B large model according to the selected fine-tuning strategy; adding a trainable embedding coding layer on the basis of the ChatGLM-6B large model to add prefixes to the prompt words for information filtering, adapting to downstream medical consultation tasks without changing the parameter weights of the original model, so that the model can better understand and handle medical problems;

[0013] Step 4: Input the patient's question into the medical model obtained after fine-tuning in step 3, generate multiple answers from the medical model, and professional doctors provide medical feedback and annotate each answer generated by the fine-tuned medical model. Based on the doctors' medical feedback and annotations, a medical response effect evaluation model is trained;

[0014] Step 5: Use reinforcement learning combined with the medical response effect evaluation model to update model parameters, improve the medical model capabilities from a professional perspective, make it more humane, and improve model performance;

[0015] Step 6: Evaluate model performance; use the validation set and test set to evaluate the fine-tuned model to determine whether its performance meets the requirements;

[0016] Step 7: Apply the model: Apply the fine-tuned medical model to the medical consultation system.

[0017] The present invention has the following beneficial effects:

[0018] The above-mentioned efficient parameter adjustment strategy greatly reduces the pressure on computing resources compared to fine-tuning all parameters, and large medical models can be deployed on consumer-grade graphics cards. Combined with reinforcement learning, interactive learning with doctors, and medical response effect evaluation models are used to update model parameters, so as to improve the medical model capabilities from a professional perspective and make them more humane, improve model performance, and optimize and adjust large medical models that can be deployed locally. The adjusted large medical model is applied to the medical consultation system to assist doctors in diagnosis and treatment, improve work efficiency and reduce work pressure. Patients can experience accurate disease diagnosis and treatment plans online even at home. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a flow chart of a method in an embodiment of the present invention;

[0020] Figure 2 It is a schematic diagram of the structure of the medical consultation system in an embodiment of the present invention.

[0021] Among them, the accompanying drawings are marked as: model fine-tuning module 100, reinforcement learning module 200, and medical consultation module 300. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in each embodiment of the present invention described below can be combined with each other as long as they do not conflict with each other. To achieve the above-mentioned purpose, the present invention adopts the following technical scheme.

[0023] Figure 1 This is a flow chart of a method for building a medical model. The method includes:

[0024] Step 1: Data collection and preprocessing: Collect relevant data in the medical field, including medical records, symptom descriptions, medical knowledge, etc., and perform data cleaning and labeling. The specific steps include:

[0025] Step 11: Collect data: The collected data include public data sets, medical guidelines and paper data, and records of diagnosis and treatment conversations between doctors and patients.

[0026] Step 12: Cleaning data: Clean and preprocess the collected data to remove noise and useless information to ensure data quality and accuracy. First, preliminarily filter some low-quality text paragraphs based on rules. These rules mainly include low-density text, abnormal symbols, low Chinese ratio, sensitive words, etc. Then use the textcnn model to filter suspected advertising content, then use simhash to deduplicate similar text fragments, and finally use the ngram language model and large language model evaluation method to filter low-quality text.

[0027] Step 13: Format data: Organize and store the dataset in JSON file format.

[0028] For example, the dataset format is {"question":"Here is the question, that is, the question","answer":"Here is the answer, that is, the doctor's answer to the question"}, for example: {"question":"For a patient with stomach disease, how to regulate at home, what to eat, how to take care of the patient, the stomach has been removed.","answer":"This situation is considered to be caused by gastritis, and can be treated with omeprazole, cisapride, belladonna tablets. It is recommended to eat regularly and avoid spicy and cold food."}

[0029] Step 2: Load the pre-trained model: Load the pre-trained open source large model and corresponding parameters onto the computing device.

[0030] Step 21: Download the trained open source large model ChatGLM-6B model, some basic packages, and related dependencies and libraries from GitHub.

[0031] Step 22: Select a suitable deep learning framework, such as pytorch, TensorFlow, Transformers, etc., load the downloaded model and install it.

[0032] Step 23: Initialize the downloaded model and its weights.

[0033] Step 3: Fine-tune the model: Fine-tune the model according to the selected fine-tuning strategy. If computing resources are abundant, full parameter fine-tuning can be performed, that is, all parameters are fine-tuned. If computing resources are prioritized, efficient parameter fine-tuning is performed, and only some important parameters will be trained.

[0034] Parameter fine-tuning example: Based on the original model, add a trainable embedding encoding layer to add prefixes to the prompt words for information filtering, so that the model can better understand the intention of the prompt words; among the existing parameters, select some parameters as learnable parameters to create prefixes for each prompt word, freeze the original model parameters, and only train the prefix of each prompt word, so as to help the model better understand and handle medical consultation tasks. The final prefixes obtained by training are interpretable, and we can check these prefixes to see how the model helps us optimize the prompt words; efficient parameter fine-tuning greatly reduces the pressure on computing resources compared to fine-tuning all parameters.

[0035] More specifically, the ChatGLM-6B large model is fine-tuned and trained according to the selected fine-tuning strategy; based on the ChatGLM-6B large model, a trainable embedding coding layer is added to adapt to downstream medical consultation tasks without changing the parameter weights of the original model, so that the model can better understand and handle medical problems; among the original model parameters, the layers with large model parameter gradient changes are selected as learnable parameters, and the impact of these layers on the overall performance of the model is verified through ablation. Based on these selected layers, prompt coding prefixes are created, and the original model parameters are frozen. During the training process, only each prompt coding prefix is ​​trained, which keeps the original model structure stable while helping the model to better understand and handle medical consultation tasks. The prefixes obtained through training are finally interpretable;

[0036] More specifically, a continuous learnable embedding tensor for medical consultation tasks is spliced ​​on the front end of the embedding vector input by ChatGLM-6B to adapt to the medical consultation task. During training, only the embedding tensor is optimized by gradient descent, while the original model parameters are frozen, which keeps the original model structure stable while helping the model to better understand and handle medical consultation tasks. The prefix obtained by training is finally interpretable.

[0037] Step 4: Input the patient's question to the fine-tuned model after step S30, and multiple answers can be generated from the fine-tuned medical model. Professional doctors can provide medical feedback and annotate each answer generated by the fine-tuned medical model. Based on the medical feedback and annotations of doctors, a medical response effect evaluation model is trained.

[0038] Step 41: Based on the fine-tuned medical model, given the input patient problem P, n model outputs can be generated:

[0039] ,

[0040] in, is the nth answer generated by the medical model;

[0041] Step 42: The doctor gives n answer vectors Sort by effect and get the sorted answer vector ,Will Use the permutation and combination method to divide it into C(n, 2) groups, and then have C(n, 2) professional doctors evaluate it and combine it with the patient's problem P to get the doctor's feedback containing the patient's problem (P, , );

[0042] Step 43: Repeat steps S41 to S42 for m patient questions to obtain C(n, 2)×m doctor feedback comparison pairs containing patient questions, and construct a doctor feedback data set;

[0043] Step 44: Based on the doctor feedback data set, a medical response effect evaluation model can be obtained through training. The calculation formula of the loss function of the medical response effect evaluation model is:

[0044] ,

[0045] in, For the model parameters In the case of medical response effect evaluation model for patient question P and model response answer The scalar output score of is the label of the doctor feedback dataset, E represents The expected value on the dataset, is the activation function; E represents Expected value on the dataset.

[0046] Step 5: Use reinforcement learning combined with the medical response effect evaluation model to update the model parameters, improve the medical model capabilities from a professional perspective, make it more humane, "align" professional physicians, and improve model performance: Specifically, the following steps are included:

[0047] Step 51: backing up a fine-tuned medical benchmark model obtained in step S30, and naming the backed-up model as an RL model;

[0048] Step 52: Randomly extract a patient question P from the data set collected in S10, and use the RL model to generate a machine reply Reply;

[0049] Step 53: The extracted patient question P and its corresponding answer Reply are used as the input of the primary medical response effect evaluation model obtained in step S40 to obtain the score output by the medical response effect evaluation model. ;

[0050] Step 54: Based on the reinforcement learning algorithm, the score output by the TCM response effect evaluation model in step S53 is used Update the parameters of the RL model.

[0051] During the training of the reinforcement learning algorithm, the maximum combined objective function is as follows:

[0052] ,

[0053] in, Represents the strategy to be learned by the RL model, which is initially composed of initialization, It is a supervised training model trained on the basis of labeled questions and answers. is the distribution of data used for reinforcement learning, is the distribution of training data during pre-training, is the reward coefficient and r is the loss coefficient of pre-training.

[0054] Step 55: After repeating S42 to S44 multiple times, the RL model obtained is the trained medical model.

[0055] Step 6: Evaluate model performance: Evaluate the fine-tuned model using the test set or validation set to determine whether its performance meets the requirements.

[0056] During the training process, the model is evaluated regularly using the validation set, and the hyperparameters or fine-tuning strategies are adjusted based on the evaluation results. This helps improve the performance and generalization ability of the model. After fine-tuning is completed, the fine-tuned model is evaluated using the test set to determine whether its performance meets the requirements.

[0057] Step 7: Apply the model: Apply the fine-tuned model to the medical consultation system.

[0058] The fine-tuned model provides an interface in the medical consultation system, through which the medical consultation system can call the medical big model. Users do not need to directly operate the big model, but can directly input some questions in the medical consultation visualization system, and the results obtained through the trained big model are returned to the user. The specific steps include:

[0059] Step 71: User input: The user can input his or her own symptoms or medical knowledge that he or she wants to consult.

[0060] Step 72: Model reasoning: The questions input by the user are sent to the medical model, which can be used to assist doctors in diagnosing, formulating treatment plans, etc. through model reasoning.

[0061] Step 73: Output results: The questions input by the user can be used to obtain diagnosis results, treatment plans, etc. through model reasoning.

[0062] The present invention also provides a medical consultation system, such as Figure 2 As shown, the results obtained based on the above method include:

[0063] The model fine-tuning module 100 is used to fine-tune the open source big model and input unique data to train a unique medical big model suitable for oneself;

[0064] Reinforcement learning module 200, used to interactively learn with doctors through reinforcement learning, update model parameters by combining reinforcement learning with a medical response effect evaluation model, improve the medical model capabilities from a professional perspective, make them more humane, and improve model performance;

[0065] The medical consultation module 300 is used to apply the medical model to diagnose and treat the user.

Claims

1. A method for constructing a medical model, characterized in that: The steps include: Step 1: Data collection and preprocessing, including: collecting relevant data in the medical field, including medical records, symptom descriptions, medical knowledge, and performing data cleaning and labeling; Step 2: Load the pre-trained model, including: loading the pre-trained open source ChatGLM-6B large model and corresponding parameters onto the computing device; Step 3: Fine-tune the model to obtain the medical model, including: fine-tuning the ChatGLM-6B large model according to the selected fine-tuning strategy; adding a trainable embedding coding layer on the basis of the ChatGLM-6B large model to add prefixes to the prompt words for information filtering, adapting to the downstream medical consultation tasks without changing the parameter weights of the original model, so that the model can better understand and handle medical problems; Step 4: Input the patient's question into the medical model obtained after fine-tuning in step 3, generate multiple answers from the medical model, and professional doctors provide medical feedback and annotate each answer generated by the fine-tuned medical model. Based on the doctors' medical feedback and annotations, a medical response effect evaluation model is trained; Step 5: Use reinforcement learning combined with the medical response effect evaluation model to update the model parameters, improve the medical model capabilities from a professional perspective, make it more humane, and improve model performance; Step 6: Evaluate model performance; use the validation set and test set to evaluate the fine-tuned model to determine whether its performance meets the requirements; Step 7: Apply the model: Apply the fine-tuned medical model to the medical consultation system.

2. The method according to claim 1, characterized in that Step 1 includes: Step 11: Collect data: The collected data include public data sets, medical guidelines and paper data, and records of diagnosis and treatment conversations between doctors and patients; Step 12: Cleaning data: Clean and preprocess the collected data to remove noise and useless information; Step 13: Format data: Organize and store the dataset in JSON file format.

3. The method according to claim 2, characterized in that Specifically, step 3 is to splice a continuous learnable embedding tensor for medical consultation tasks at the front end of the embedding vector input by ChatGLM-6B to adapt to the medical consultation task; during training, only the embedding tensor is optimized by the gradient descent method, and the original model parameters are frozen, which keeps the original model structure stable while helping the model to better understand and handle medical consultation tasks. The prefix obtained by training is finally interpretable.

4. The method according to claim 3, characterized in that Step 4 includes: Step 41: Based on the fine-tuned medical model, given the input patient problem P, generate n model outputs: ， in, is the nth answer generated by the medical model; Step 42: The doctor gives n answer vectors Sort by effect and get the sorted answer vector ,Will The permutation and combination method is used to divide the data into C(n, 2) groups, and C(n, 2) professional doctors evaluate them respectively. Combined with the patient's problem P, the doctor's feedback containing the patient's problem is obtained (P, , );in, is the ath answer generated by the medical model for the patient question P. It is the bth answer generated by the medical model for the patient's question P. The answers that doctors rate as having good effects are ranked first, and the answers that have poor effects are ranked last. Step 43: Repeat steps S41 to S42 for m patient questions to obtain C(n, 2)×m pairs of doctor evaluation results containing patient questions, and construct a doctor feedback data set; Step 44: Based on the doctor feedback data set, a medical response effect evaluation model is obtained through training. The calculation formula of the loss function of the medical response effect evaluation model is: ， in, , The model parameters are In the case of medical response effect evaluation model for patient question P and model response answer The scalar output score of is the label of the doctor feedback dataset, E represents The expected value on the dataset, is the activation function.

5. The method according to claim 4, characterized in that Step 5 includes: Step 51: Back up a fine-tuned medical model, and name the backed up model as an RL model; Step 52: Randomly extract a patient question P from the collected data set, and use the RL model to generate a machine reply Reply; Step 53: The extracted patient question P and its corresponding answer Reply are used as inputs to the medical response effect evaluation model to obtain the score output by the medical response effect evaluation model. ; Step 54: Based on the reinforcement learning algorithm, use the score output by the medical response effect evaluation model Update the parameters of the RL model; during the training of the reinforcement learning algorithm, the maximum combined objective function is as follows: ， in, Represents the strategy to be learned by the RL model, which is initially composed of initialization, It is a supervised training model trained on the basis of labeled questions and answers. is the distribution of data used for reinforcement learning, is the distribution of training data during pre-training, is the reward coefficient, r is the loss coefficient of pre-training; Step 55: The RL model obtained by repeating steps 42 to 44 multiple times, the final RL model obtained is the trained medical model.

6. The method according to claim 5, characterized in that Step 7 includes: Step 71: User input: The user can input his or her own symptoms or medical knowledge that he or she wants to consult; Step 72: Model reasoning: Send the questions input by the user into the medical model, which can be used to assist doctors in diagnosis and formulation of treatment plans through model reasoning; Step 73: Output results: The questions input by the user are used to derive diagnosis results and treatment plans through model reasoning.

7. A medical consultation system based on any one of the methods of claims 1-6, characterized in that: include: Model fine-tuning module, used to fine-tune the open source large model and input unique data to train your own unique medical model; The reinforcement learning module is used to interact with doctors through reinforcement learning. It uses reinforcement learning combined with the medical response effect evaluation model to update model parameters, improve the medical model capabilities from a professional perspective, make it more humane, and improve model performance; The medical consultation module is used to apply medical models to diagnose and treat users.

8. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: Executable instructions are stored thereon, and when the instructions are executed by a processor, the processor implements the method according to any one of claims 1 to 6.

Images