Medical question answering system based on large language model

Abstract

The invention discloses a medical question-answering system based on a large language model, belongs to the field of large language models in artificial intelligence in the field of computers, and particularly relates to the medical question-answering system based on the large language model. The objective of the invention is to solve the problems of low answering accuracy and poor medical problem solving ability of a general large language model in the medical field. The system comprises a large language base module used for selecting an open source large language model of a general field as a base model; the knowledge instance sampling module is used for sampling knowledge instances from the knowledge graph; the question-answer data generation module is used for generating question-answer data according to the knowledge instances by using a large language base model; the large language base model fine tuning module is used for performing fine tuning on the large language base model by using the generated question and answer data to obtain a fine-tuned large language base model; and the reply generation module is used for answering the to-be-processed medical knowledge question based on the fine-tuned large language base model.

Description

Technical Field

[0001] The present invention belongs to the field of large language models in artificial intelligence in the computer field, and specifically relates to a medical question-answering system based on a large language model. Background Art

[0002] Since November 2022, large language models represented by ChatGPT have attracted great interest due to their outstanding performance in understanding instructions and generating human-like responses. Compared with smaller models, large language models have shown strong generalization capabilities in various natural language processing tasks due to their large-scale data training and complex deep learning architecture, and have unique emergent capabilities in solving zero-shot tasks or complex tasks.

[0003] Despite the many advantages of large language models, they are not specifically designed for the medical field. When dealing with such specialized field problems, the general domain knowledge in large language models is often insufficient to answer them, and relevant knowledge in specific medical fields is crucial. The lack of such knowledge may cause large models to produce suboptimal diagnostic accuracy, drug recommendations, and medical advice, the results of which may endanger patients.

[0004] In the medical field, it is difficult to obtain high-quality training data, and data sets involving medical question-answering tasks are relatively rare. The data provided on existing medical question-answering platforms often lack strict industry standard review, and their professionalism is somewhat different from the authoritative standards in the medical field. Although data can be collected from these platforms through crawler technology, due to the high noise and unstable quality of these data, fine-tuning using such data usually cannot meet the accuracy and professionalism requirements of the output answers.

[0005] Therefore, how to build a high-quality medical question-and-answer dataset and effectively improve the performance of large language models in medicine, a highly professional field that relies heavily on the accuracy of generated knowledge, has become an urgent problem to be solved in the current technical field. It is also of great significance to the further expansion of large language models in future application fields such as intelligent medicine and clinical decision support. Summary of the invention

[0006] The purpose of the present invention is to solve the problem that the general large language model has low answer accuracy in the medical field and poor ability to solve medical problems, and to propose a medical question-answering system based on a large language model.

[0007] A medical question-answering system based on a large language model comprises: a large language base module, a knowledge instance sampling module, a question-answering data generation module, a large language base model fine-tuning module and a reply generation module;

[0008] The large language base module is used to select an open source large language model in a general field as a base model;

[0009] The knowledge instance sampling module is used to sample knowledge instances from the knowledge graph;

[0010] The question-answer data generation module is used to generate question-answer data according to knowledge instances using a large language base model;

[0011] The large language base model fine-tuning module is used to fine-tune the large language base model using the generated question-answering data to obtain a fine-tuned large language base model;

[0012] The response generation module is used to answer the pending medical knowledge questions based on the fine-tuned large language base model.

[0013] The beneficial effects of the present invention are:

[0014] The present invention uses the CMeKG knowledge graph to construct medical knowledge question-answer pairs, and performs supervised fine-tuning of the LLaMA-7B base model. The ability to handle tasks and the ability to express in medical scenarios are significantly improved, especially in terms of usability and fluency, without significantly reducing the security of the answers.

[0015] The large language base module of the present invention is used to select an open source large language model in a general field as a base model; the knowledge instance sampling module is used to sample knowledge instances from the knowledge graph; the question and answer data generation module is used to use the large language base model to generate question and answer data according to the knowledge instances; the large language base model fine-tuning module is used to use the generated question and answer data to fine-tune the large language base model to obtain a fine-tuned large language base model; the reply generation module is used to answer the medical knowledge questions to be processed based on the fine-tuned large language base model; the answer accuracy of the general large language model in the medical field is improved, and the ability to solve medical problems is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Flowchart of the method for fine-tuning large language models for medicine;

[0017] Figure 2 Schematic diagram for constructing multi-type medical question-answering data. DETAILED DESCRIPTION

[0018] Specific implementation method 1: In this implementation method, a medical question answering system based on a large language model includes:

[0019] Large language base module, knowledge instance sampling module, question and answer data generation module, large language base model fine-tuning module and response generation module;

[0020] The large language base module is used to select an open source large language model in a general field as a base model;

[0021] The knowledge instance sampling module is used to sample knowledge instances from the knowledge graph;

[0022] The question-answer data generation module is used to generate question-answer data according to knowledge instances using a large language base model;

[0023] The large language base model fine-tuning module is used to fine-tune the large language base model using the generated question-answering data to obtain a fine-tuned large language base model;

[0024] The response generation module is used to answer the pending medical knowledge questions based on the fine-tuned large language base model.

[0025] Construct the evaluation index SUS for medical question answering tasks;

[0026] The SUS indicator is used to evaluate the ability of the fine-tuned model in the medical field.

[0027] Specific implementation method 2: This implementation method is different from the specific implementation method 1 in that the large language base module is used to select an open source large language model in a general field as the base model; the specific process is:

[0028] The large language base model is an open source large language model, and the open source large language model is LLaMA, BLOOM or ChatGLM.

[0029] The open source large language model has scalability and multi-task processing capabilities, and supports fine-tuning of the model to adapt to the complex needs of the medical field.

[0030] Based on the existing large language model as the basic model, the selected model needs to have strong natural language processing capabilities and be able to process in multiple language environments. At the same time, considering the particularity of the medical field, it is necessary to select a model with strong scalability and multi-task processing capabilities to ensure that it can adapt to the complex needs of the medical field. Open source large language models include LLaMA, BLOOM, ChatGLM, etc.

[0031] The other steps and parameters are the same as those in the first specific implementation.

[0032] Specific implementation method three: This implementation method is different from specific implementation methods one or two in that the question and answer data generation module is used to generate question and answer data based on knowledge instances using a large language base model;

[0033] The specific process is:

[0034] Randomly sample knowledge instances from the medical knowledge graph CMeKG;

[0035] Each knowledge instance is a group of a disease and corresponding knowledge, a drug and corresponding knowledge, a symptom and corresponding knowledge (each knowledge instance is a disease and corresponding knowledge, a drug and corresponding knowledge, or a symptom and corresponding knowledge);

[0036] The format of each knowledge instance is as follows:

[0037] Type:

[0038] Knowledge:

[0039] The Type field records the type of knowledge instance (disease, symptom, or drug)

[0040] The Knowledge field records the content of the knowledge instance (knowledge corresponding to the disease, knowledge corresponding to the symptom, or knowledge corresponding to the drug);

[0041] The knowledge instances are sampled from the medical knowledge graph CMeKG, providing medical knowledge about diseases, drugs, symptoms, etc.

[0042] Examples of disease knowledge in the knowledge base are as follows:

[0043] Type: Disease;

[0044] Knowledge: {"class":"100 common diseases","key words":"liver cancer","drug treatment":["regorafenib","antiviral drugs effective for hepatitis B or C","sorafenib"],"high-incidence areas":["sub-Saharan Africa"],"risk factors":["obesity","high HBV DNA","chronic alcoholism","male","chronic hepatitis B infection","family history of liver cancer","chronic hepatitis C cirrhosis","core promoter mutation","cirrhosis","HCV superinfection","senile heart valve disease","hepatitis B e antigen","diabetes"],"site of onset":["liver"],"auxiliary examinations":["liver function test"],"medical history":["long-term history of chronic hepatitis B"]}

[0045] Examples of drug knowledge in the knowledge base are as follows:

[0046] Type: Drug;

[0047] Knowledge:{"class":"Western medicine","Keyword":"Metformin","Properties":["Sugar-coated or film-coated tablets, white after removing the coating"],"English name":["Yifan Tablets","Gervazhi"],"Classification":["Biguanides","Antituberculosis drugs"],"Specifications":["0.25g"],"OTC type":["Class B OTC","Class A OTC"],"Indications":["Diabetes","Obesity"],"Generic name":["Yifan Tablets"],"Ingredients":["Rifampicin and Isoniazid","Isoniazid","Isoniazid 0.1g","Isoniazid 150mg","This product is a compound preparation","Rifampicin","Rifampicin 300mg","Rifampicin 0.15g","Metformin hydrochloride","Hydrochloric acid"]}

[0048] Examples of symptom knowledge in the knowledge base are as follows:

[0049] Type: Symptom;

[0050] Knowledge: {"Keyword":"hair loss","Examination":["Hair mineral examination"],"Related diseases":["Alopecia areata","Chronic fatigue syndrome"],"Related symptoms":["light brown hair","dry and brittle hair","hardened skin"],"Department":["Internal Medicine","Dermatology and Venereology","Radiotherapy and Chemotherapy"],"Part of the disease":["Head"]}

[0051] The "Type" field in the sampled knowledge includes "Disease", "Drug" and "Symptom".

[0052] The sampling step combines random sampling with targeted sampling based on medical disease, drug, and symptom classifications to cover a variety of medical scenarios.

[0053] The other steps and parameters are the same as those in the first or second embodiment.

[0054] Specific implementation method 4: This implementation method is different from any one of specific implementation methods 1 to 3 in that the knowledge instance sampling module is used to sample knowledge instances from the knowledge graph;

[0055] The specific process is:

[0056] 3.1. Classify the knowledge instances randomly sampled from the medical knowledge graph CMeKG (disease, symptom or drug) to obtain the type of knowledge instance; the specific process is:

[0057] Determine the type of the randomly sampled knowledge instance (disease, symptom, or drug) according to the Type field in the randomly sampled knowledge instance;

[0058] 32. Write question templates for different types of knowledge instances;

[0059] 3. Write prompts according to the question template and generate single-type question-answering data through OpenAI API;

[0060] 3.4. Integrate single-type knowledge instances to obtain cascade type instances, write prompts according to the cascade type, and generate cascade type question and answer data through OpenAI API.

[0061] The other steps and parameters are the same as those in Specific Embodiments 1 to 3.

[0062] Specific implementation method 5: This implementation method is different from specific implementation methods 1 to 4 in that question templates are written for different types of knowledge instances in the above three or two;

[0063] The specific process is:

[0064] Question templates written for the type of "Type" being "disease";

[0065] An example is:

[0066] Question template 1: What disease is this? ;

[0067] Question template 2: What are the symptoms of the disease? ;

[0068] Question template 3: How to treat diseases? ;

[0069] Question templates written for the type of "Type" is "drug";

[0070] An example is:

[0071] Question template 1: What diseases are the drugs suitable for treating? ;

[0072] Question template 2: What are the common side effects of drugs? ;

[0073] Question templates written for the type of "Type" is "Symptom";

[0074] An example is:

[0075] Question template 1: What disease might the symptom be a manifestation of? ;

[0076] Question template 2: What should I do if symptoms occur?

[0077] The other steps and parameters are the same as those in Specific Embodiments 1 to 4.

[0078] Specific implementation method 6: This implementation method is different from specific implementation methods 1 to 5 in that in the above-mentioned 33, prompts are written according to question templates, and single-type question-answering data is generated through OpenAI API; the specific process is:

[0079] 331. Write prompts according to each question template;

[0080] The following is an example of a prompt for a disease type question template:

[0081] prompt: Based on the given disease information, please generate questions and answers about the disease between doctors and patients. The types of questions include but are not limited to: 1. What is this disease? 2. What are the symptoms of the disease? 3. How to treat the disease? : "class": "100 common diseases", "central word": "liver cancer", "drug treatment": ["regorafenib", "antiviral drugs effective for hepatitis B or C", "sorafenib"], "high-incidence areas": ["sub-Saharan Africa"], "high-risk factors": ["obesity", "HBV DNA is too high", "chronic alcoholism", "male", "chronic hepatitis B infection", "family history of liver cancer", "chronic hepatitis C cirrhosis", "core promoter mutation", "cirrhosis", "HCV superinfection", "senile heart valve disease", "hepatitis B e antigen", "diabetes"], "site of onset": ["liver"], "auxiliary examinations": ["liver function test"], "medical history": ["long-term history of chronic hepatitis B"];

[0082] An example of a prompt for a medication type question template is as follows:

[0083] prompt: Based on the given drug information, please generate questions and answers about drugs between doctors and patients. Question types include but are not limited to: 1. What diseases are drugs suitable for treating? 2. What are the common side effects of drugs? :"class":"Western medicine","keyword":"Metformin","properties":["sugar-coated or film-coated tablets, white after removing the coating"],"English name":["Yifu Tablets","Glucose"],"classification":["biguanides","anti-tuberculosis drugs"],"specifications":["0.25g"],"OTC type":["Class B OTC","Class A OTC"],"indications":["diabetes","obesity"],"generic name":["Yifu Tablets"],"ingredients":["Rifampicin and Isoniazid","Isoniazid","Isoniazid 0.1g","Isoniazid 150mg","This product is a compound preparation","Rifampicin","Rifampicin 300mg","Rifampicin 0.15g","Metformin hydrochloride","hydrochloric acid"]

[0084] An example of a prompt for a symptom-type question template is as follows:

[0085] prompt: Based on the given symptom information, please generate questions and answers about the symptoms between doctors and patients. The types of questions include but are not limited to: 1. What disease may the symptoms be a manifestation of? 2. What should be done when the symptoms occur? : "Central word": "hair loss", "examination": ["hair mineral examination"], "related diseases": ["alopecia areata", "chronic fatigue syndrome"], "related symptoms": ["light brown hair", "dry and brittle hair", "hardened skin"], "department": ["internal medicine", "dermatology and venereal diseases", "radiotherapy and chemotherapy department"], "site of onset": ["head"]

[0086] 332. Use ChatGPTAPI in OpenAIAPI, input each prompt into ChatGPTAPI, and obtain the corresponding question and answer data from the return value of ChatGPTAPI; the specific process is:

[0087] Extract the choices[0].message.content field from the chat_completion object returned by the ChatGPTAPI;

[0088] The format of the content (question and answer data) recorded in the choices[0].message.content field is:

[0089] question:

[0090] answer:

[0091] 3 are independent;

[0092] For diseases:

[0093] question:What is liver cancer?

[0094] answer: Liver cancer is a malignant tumor that originates from the liver, usually caused by factors such as chronic hepatitis or cirrhosis. The early symptoms are not obvious, but may cause symptoms such as pain in the liver area, indigestion, and fatigue.

[0095] For drugs:

[0096] Question:What diseases is metformin suitable for treating?

[0097] answer:Metformin is primarily used to treat diabetes, especially type 2 diabetes. It is also used to treat obesity and can help control weight.

[0098] For symptoms:

[0099] Question: I have recently found that my hair has fallen out a lot. What disease might this be?

[0100] Answer: Hair loss can be related to a variety of medical conditions, with alopecia areata and chronic fatigue syndrome being common causes. Alopecia areata usually causes sudden hair loss in a localized area, while chronic fatigue syndrome may be accompanied by symptoms such as dry, brittle hair. If you have persistent fatigue or other discomfort, further examination is recommended.

[0101] The other steps and parameters are the same as those in Specific Embodiments 1 to 5.

[0102] Specific implementation method seven: This implementation method is different from specific implementation methods one to six in that the specific process of using ChatGPTAPI in OpenAIAPI in the above 332 is:

[0103] Register an OpenAI account, create a ChatGPTAPI key, load the openai library through the Python client, configure the obtained ChatGPTAPI key to the openai.api_key parameter of the code, and configure the version name of GPT 3.5 to the model parameter.

[0104] For example, "gpt-3.5-turbo"; at the same time, fill in the content key value of the messages list with the prompt configuration obtained in step 332 in the code, and set the role key value of the messages list to "user"; call the openai.ChatCompletion.create() method to send a request to the OpenAI server, and wait for the server to generate text based on the request and return it; the generated response will return a chat_completion object, and the answer content is extracted from the choices[0].message.content field of the object.

[0105] The other steps and parameters are the same as those in Specific Embodiments 1 to 6.

[0106] Specific implementation eight: This implementation differs from specific implementations one to seven in that in the third and fourth embodiments, single-type knowledge instances are integrated to obtain cascade type instances, prompts are written according to the cascade type, and cascade type question and answer data are generated through OpenAI API;

[0107] The specific process is:

[0108] 341. Combine the related knowledge instances sampled from the medical knowledge graph CMeKG, and the combined types are disease-drug, disease-symptom or disease-drug-symptom; 342

[0110] 1) Targeting disease-drug combination; the specific process is:

[0111] 11) When the type is a disease-drug combination, extract the content of the "knowledge" field of the disease;

[0112] Get the value value1 of the key "central word" in the "knowledge" field;

[0113] The content of the "knowledge" field of the disease is stored in dictionary form;

[0114] 12) Match each key value in the "knowledge" field of the drug with value1;

[0115] If the match is successful, the key value in the "knowledge" field corresponding to the drug is added to the content of the "knowledge" field of the disease to obtain a new disease-drug combination;

[0116] If the match fails, no processing will be done;

[0117] 13) Repeat 12) until all key values ​​in the "knowledge" field of the drug are traversed to determine whether there is a successful match;

[0118] If there is a final new disease-drug combination obtained;

[0119] If it does not exist, execute 14);

[0120] 14) Obtain a synonym list of value1 based on the synonym library, assign the synonyms in the synonym list to value1, repeat 12) until a match is successful, obtain a new disease-drug combination, and end; if no match is successful, abandon the disease-drug combination;

[0121] Thesaurus such as SNOMED CT, UMLS, etc.

[0122] 2) Targeting the disease-symptom combination; the specific process is:

[0123] 21) When the type is a disease-symptom combination, extract the content of the "knowledge" field of the disease;

[0124] Get the value value1 of the key "central word" in the "knowledge" field;

[0125] The content of the "knowledge" field of the disease is stored in dictionary form;

[0126] 22) Match each key value in the "knowledge" field of the symptom with value1;

[0127] If the match is successful, the key value in the "knowledge" field corresponding to the symptom is added to the content of the "knowledge" field of the disease to obtain a new disease-symptom combination;

[0128] If the match fails, no processing will be done;

[0129] 23) Repeat 22) until all key values ​​in the "knowledge" field of the symptom are traversed to determine whether there is a successful match;

[0130] If there is a final new disease-symptom combination obtained;

[0131] If it does not exist, execute 24);

[0132] 24) Obtain a synonym list of value1 based on the synonym library, assign the synonyms in the synonym list to value1, repeat 22) until a match is successful, obtain a new disease-symptom combination, and end; if no match is successful, abandon the disease-symptom combination;

[0133] Thesaurus such as SNOMED CT, UMLS, etc.

[0134] 343. Targeting the disease-drug-symptom combination; the specific process is:

[0135] 1) The new disease-drug combination obtained in 14) is regarded as type 1, and the symptoms are regarded as type 2;

[0136] 2) Extract the content of the "knowledge" field of type 1; the content of the "knowledge" field of type 1 is stored in the form of a dictionary list; select the value value1 of the key "central word" in the first element of the list;

[0137] The element is a dictionary;

[0138] A disease is a dictionary, a symptom is also a dictionary, then disease-symptom is a list of dictionaries; facing a knowledge instance of disease-symptom, only the content of the first dictionary is needed to identify the central word, that is, the first dictionary in the dictionary list; an element is a dictionary, which is a long section, including many key-value pairs, just like the composition of a single type;

[0139] 3) Match each key value in the "knowledge" field of type 2 with value1;

[0140] If the match is successful, the key value in the "knowledge" field corresponding to type 2 is added to the content of the "knowledge" field of type 1 to obtain a new type 1-type 2 combination;

[0141] If the match fails, no processing will be done;

[0142] 4) Repeat 3) until all key values ​​in the "knowledge" field of type 2 are traversed to determine whether there is a successful match;

[0143] If it exists, obtain the final new type one-type two combination;

[0144] If it does not exist, execute 5);

[0145] 5) Obtain a synonym list of value1 based on the synonym library, assign the synonyms in the synonym list to value1, repeat 3) until a match is successful, obtain a new type 1-type 2 combination, and end; if no match is successful, abandon the type 1-type 2 combination;

[0146] Thesaurus such as SNOMED CT, UMLS, etc.

[0147] 344. Write a question template for disease-symptom, disease-drug, or disease-drug-symptom. The specific process is:

[0148] An example of a question template for the disease-symptom type is:

[0149] Question template 1: What are the symptoms that are not easily noticed in the early stage of the disease? ;

[0150] Question template 2: I have recently developed some symptoms. What disease might it be? What tests do I need to do? ;

[0151] An example of a question template for the disease-drug type is:

[0152] Question template 1: What medication should be used to treat the disease? Will there be side effects?

[0153] Question template 2: I have a certain disease, what drugs should I avoid?

[0154] For the disease-drug-symptom type, the following example question templates are written:

[0155] Question template 1: What should patients who have symptoms and are diagnosed with a disease pay attention to when using medication? ;

[0156] Question template 2: Generate a question-answer pair that includes the patient's detailed background and clear medical questions. The treatment methods of the disease should be explained in detail, such as the details of drug treatment and non-drug treatment. Please ensure that different knowledge points are covered in the answer and provide suggestions;

[0157] 3.4.5. Write prompts according to each question template;

[0158] An example of a prompt for a disease-symptom type question template is as follows:

[0159] prompt: Based on the given disease and corresponding symptom information, please generate questions and answers between doctors and patients about disease-symptoms. The types of questions include but are not limited to: 1. What are the symptoms that are not easy to detect in the early stage of the disease? ; 2. Some symptoms have appeared recently. What disease might it be and what tests are needed? : [{"class":"100 common diseases","central word":"liver cancer","drug treatment":["regorafenib","antiviral drugs effective for hepatitis B or C","sorafenib"],"high-incidence areas":["sub-Saharan Africa"],"high-risk factors":["obesity","HBV DNA is too high","Chronic alcoholism","Male","Chronic hepatitis B infection","Family history of liver cancer","Chronic hepatitis C cirrhosis","Core promoter mutation","Cirrhosis","HCV superinfection","Elderly valvular heart disease","Hepatitis B e antigen","Diabetes"],"Site of disease":["Liver"],"Auxiliary examination":["Liver function test"],"Medical history":["Long-term history of chronic hepatitis B"];},{"Keyword":"Fatigue","Examination":["Non-essential trace elements","Asparagine","Glutamine test","4-tibia proline"],"Related diseases":["Tuberculosis","Liver cancer","Pine caterpillar disease and osteoarthritis"],"Site of disease":["Whole body"],"Department":["Nutrition department","Internal medicine department","Physical examination, health care department"],"Related symptoms":["Congestive heart failure","Arm pain","Abdominal pain during menstruation due to uterine blood stasis","Hard skin"]}]

[0160] An example of a prompt for a disease-drug type question template is as follows:

[0161] prompt: Based on the given disease and corresponding drug information, please generate questions and answers about the disease and drug between doctors and patients. The types of questions include but are not limited to: 1. What drug should be used to treat the disease, and will there be side effects? ; 2. I have a certain disease, which drugs should I avoid? : [{"class":"100 common diseases","central word":"liver cancer","drug treatment":["regorafenib","antiviral drugs effective for hepatitis B or C","sorafenib"],"high-incidence areas":["sub-Saharan Africa"],"high-risk factors":["obesity","HBV DNA is too high","Chronic alcoholism","Male","Chronic hepatitis B infection","Family history of liver cancer","Chronic hepatitis C cirrhosis","Core promoter mutation","Cirrhosis","HCV superinfection","Elderly valvular heart disease","Hepatitis B e antigen","Diabetes"],"Site of disease":["Liver"],"Auxiliary examination":["Liver function test"],"Medical history":["Long-term history of chronic hepatitis B"];},{"class":"Western medicine","Central word":"bp element","Classification":["Cholelytic agent / Cholelitholytic agent / Hepatoprotective drug","Other immunosuppressants / Other respiratory system drugs","Immunoenhancer"],"OTC type":["Class A OTC","Class B OTC"],"Specification":["50mg per vial","10mg / bottle"],"Contraindications":["Prohibited for those who are allergic to this product"],"Drug type":["Prescription drug"],"Indications":["Adjuvant treatment of other cancers","Bronchopulmonary ","Urticaria","Tracheitis","Pancreatic cancer","Other diseases caused by low immune function","Adjuvant treatment of various malignant tumors","Soft tissue sarcoma","Psoriasis","Cancer","Gastric cancer","Hepatitis","Breast cancer","Treatment of urticaria, etc.","Liver cancer","Measles infection","Bronchopulmonary carcinoma","Asthma","Soft tissue type rhabdomyosarcoma"],"Storage":["Seal or seal tightly, store in a cool (not exceeding 20℃) dry place","Seal tightly or seal tightly, store in a cool and dry place"],"Adverse reactions":["Pain","Dizziness","Urticaria","Palpitations and urticaria","Fever","Redness and swelling","Nausea","Measles infection","Palpitations","Chest tightness"],"Precautions":["Stop using if the diameter of the injection site is more than 10cm","Use with caution in patients with allergic constitution"],"Validity period":["Tentatively 24 months"],"Drug supervision classification":["National basic medicine;The injection is a national medical insurance drug","External use drugs","Anesthetic drugs"],"Ingredients":["Sodium chloride","Ribonucleic acid extracted from the spleen of healthy pigs","The name is mannitol","Sugar","Ribonucleic acid III","Ribonucleic acid","Mannitol"],"Properties":["White or off-white freeze-dried blocks or powder, easily soluble in water"]},{"class":"Western medicine","Central words":"Triphasic contraceptive tablets","Third-level classification":["Short-acting oral contraceptives"],"First-level classification":["Gynecological drugs"],"Second-level classification":["Contraceptives"],"Adverse reactions":["Vaginal bleeding","Headache","Stomach pain","Pregnancy reaction","Nausea","Bleeding","Depression"],"Precautions":["This product should be properly stored to prevent children from taking it by mistake.","Store in a dark, airtight, cool place. ","Please read the instructions carefully before using the medicine."],"Dosage form":["Tablets: 21 tablets per pack, divided into yellow, white, and brown tablets. "],"Contraindications":["Stroke","Liver cancer","Endometrial cancer","Diabetes","Anemia","Sclerosis","Pruritus","Jaundice"],"Subjects":["Obstetrics and gynecology drugs"]}];

[0162] An example of a prompt for a disease-drug-symptom question template is as follows:

[0163] prompt: Based on the given disease and corresponding drug and symptom information, please generate questions and answers about disease-drug-symptoms between doctors and patients. The question types include but are not limited to: 1. What should patients who have symptoms and are diagnosed with the disease pay attention to when using drugs? ; 2. Generate a question-answer pair containing the patient's detailed background and clear medical questions, and explain in detail the treatment methods of the disease, such as drug treatment and non-drug treatment. Please make sure that the answer covers different knowledge points and provides suggestions: [{"class":"One hundred common diseases","Central word":"Liver cancer","Drug treatment":["Regorafenib","Antiviral drugs effective for hepatitis B or C","Sorafenib"],"High-incidence areas":["Sub-Saharan Africa"],"High-risk factors":["Obesity","HBV DNA is too high","Chronic alcoholism","Male","Chronic hepatitis B infection","Family history of liver cancer","Chronic hepatitis C cirrhosis","Core promoter mutation","Cirrhosis","HCV superinfection","Elderly valvular heart disease","Hepatitis B e antigen","Diabetes"],"Site of disease":["Liver"],"Auxiliary examination":["Liver function test"],"Medical history":["Long-term history of chronic hepatitis B"];},{"class":"Western medicine","Central word":"bp element","Classification":["Cholelytic agent / Cholelitholytic agent / Hepatoprotective drug","Other immunosuppressants / Other respiratory system drugs","Immunoenhancer"],"OTC type":["Class A OTC","Class B OTC"],"Specification":["50mg per vial","10mg / bottle"],"Contraindications":["Prohibited for those who are allergic to this product"],"Drug type":["Prescription drug"],"Indications":["Adjuvant treatment of other cancers","Bronchopulmonary ","Urticaria","Tracheitis","Pancreatic cancer","Other diseases caused by low immune function","Adjuvant treatment of various malignant tumors","Soft tissue sarcoma","Psoriasis","Cancer","Gastric cancer","Hepatitis","Breast cancer","Treatment of urticaria, etc.","Liver cancer","Measles infection","Bronchopulmonary carcinoma","Asthma","Soft tissue type rhabdomyosarcoma"],"Storage":["Seal or seal tightly, store in a cool (not exceeding 20℃) dry place","Seal tightly or seal tightly, store in a cool and dry place"],"Adverse reactions":["Pain","Dizziness","Urticaria","Palpitations and urticaria","Fever","Redness and swelling","Nausea","Measles infection","Palpitations","Chest tightness"],"Precautions":["Stop using if the diameter of the injection site is more than 10cm","Use with caution in patients with allergic constitution"],"Validity period":["Tentatively 24 months"],"Drug supervision classification":["National basic medicine;The injection is a national medical insurance drug","External use drugs","Anesthetic drugs"],"Ingredients":["Sodium chloride","Ribonucleic acid extracted from the spleen of healthy pigs","The name is mannitol","Sugar","Ribonucleic acid III","Ribonucleic acid","Mannitol"],"Properties":["White or off-white freeze-dried blocks or powder, easily soluble in water"]},{"class":"Western medicine","Central words":"Triphasic contraceptive tablets","Third-level classification":["Short-acting oral contraceptives"],"First-level classification":["Gynecological drugs"],"Second-level classification":["Contraceptives"],"Adverse reactions":["Vaginal bleeding","Headache","Stomach pain","Pregnancy reaction","Nausea","Bleeding","Depression"],"Precautions":["This product should be properly stored to prevent children from taking it by mistake.","Store in a dark, airtight, cool place. ","Please read the instructions carefully before using the medicine."],"Dosage form":["Tablets: 21 tablets per pack, divided into yellow, white, and brown tablets. "],"Contraindications":["Stroke","Liver cancer","Endometrial cancer","Diabetes","Anemia","Sclerosis","Pruritus","Jaundice"],"Subjects":["Gynecological and obstetrical drugs"]},{"Keywords":"Fatigue","Examinations":["Non-essential trace elements","Asparagine","Glutamine test","4-tibia proline"],"Related diseases":["Tuberculosis","Liver cancer","Pine caterpillar disease and osteoarthritis"],"Part of the disease":["Whole body"],"Departments":["Nutrition department","Internal medicine","Physical examination and health care department"],"Related symptoms":["Congestive heart failure","Arm pain","Uterine blood stasis and abdominal pain during menstruation","Hard skin"]}];

[0164] 3.4.6. Use the ChatGPTAPI in OpenAIAPI, input each prompt into the ChatGPTAPI, and obtain the corresponding question and answer data from the return value of the ChatGPTAPI.

[0165] The other steps and parameters are the same as those in Specific Embodiments 1 to 7.

[0166] Specific implementation method 9: This implementation method is different from specific implementation methods 1 to 81 in that the ChatGPTAPI in OpenAIAPI is used in the above 3, 4 and 6, each prompt is input into the ChatGPTAPI, and the corresponding question and answer data is obtained from the ChatGPTAPI return value;

[0167] The specific process is:

[0168] Extract the choices[0].message.content field from the chat_completion object returned by the ChatGPTAPI;

[0169] The format of the content (question and answer data) recorded in the choices[0].message.content field is:

[0170] question:

[0171] answer:

[0172] 3 are independent;

[0173] Disease-symptom specific:

[0174] Question: I have been feeling weak lately, and even feel tired after a little activity. What diseases might this be a manifestation of?

[0175] answer: Fatigue is a common symptom of many diseases and may be related to systemic diseases. For example, liver cancer is a disease that may cause fatigue, especially in the advanced stage of the disease. In addition, tuberculosis and other chronic diseases such as pine caterpillar disease and osteoarthritis can also cause similar symptoms. It is recommended that you do further examinations, such as non-essential trace element testing or asparagine testing, to clarify the cause.

[0176] Disease-specific drugs:

[0177] Question: What drugs can be used to treat liver cancer patients?

[0178] Answer: Drug treatment for liver cancer usually includes regorafenib and sorafenib, which slow down the progression of the disease by inhibiting the growth and angiogenesis of tumor cells. Doctors will develop a personalized treatment plan based on the patient's specific situation.

[0179] Disease-Drug-Symptom Targeted:

[0180] Question: The patient was diagnosed with liver cancer due to a long history of chronic hepatitis B and liver function tests. What are the treatments for liver cancer? ;

[0181] answer: Treatments for liver cancer include drug therapy, surgery, radiotherapy, and targeted therapy. Among drug treatments, regorafenib and sorafenib are commonly used targeted drugs that prevent tumor growth by inhibiting tumor angiogenesis. Surgical treatment is suitable for patients with early liver cancer, and the goal is to control the disease by removing the tumor. Radiotherapy is used to locally control tumor growth. For patients with chronic hepatitis B, the use of antiviral drugs is also key to reduce further damage to the liver by the virus. Each patient's treatment plan needs to be formulated according to the specific situation, with a comprehensive assessment of the condition, liver function, and the patient's physical condition.

[0182] The specific process of using the ChatGPTAPI in OpenAIAPI is as follows:

[0183] Register an OpenAI account, create a ChatGPTAPI key, load the openai library through the Python client, configure the obtained ChatGPTAPI key to the openai.api_key parameter of the code, and configure the version name of GPT 3.5 to the model parameter.

[0184] For example, "gpt-3.5-turbo"; at the same time, fill in the prompt configuration obtained in the code as the content key value of the messages list, and the role key value of the messages list is uniformly set to "user"; call the openai.ChatCompletion.create() method to send a request to the OpenAI server, and wait for the server to generate text based on the request and return it; the generated response will return a chat_completion object, and the answer content is extracted from the choices[0].message.content field of the object.

[0185] The other steps and parameters are the same as those in Specific Embodiments 1 to 8.

[0186] Specific implementation ten: This implementation is different from specific implementations one to nine-one in that the large language base model fine-tuning module is used to fine-tune the large language base model using the generated question-answering data to obtain a fine-tuned large language base model;

[0187] The specific process is:

[0188] The training set is the question-answering data generated by the question-answering data generation module;

[0189] Fine-tune the large language base model using supervised learning methods;

[0190] The optimization method adopts the LoRA method;

[0191] Initialize the large language base model parameters and load the pre-trained weights;

[0192] The large language base model is trained until convergence to obtain a fine-tuned large language base model.

[0193] The optimization method adopts the LoRA (Low-Rank Adaptation) method, which uses low-rank matrix decomposition to decompose the original fully connected layer weight matrix in the model into the product of two low-rank matrices. It only needs to update the low-rank matrix instead of the weight of the entire model, thus reducing the number of updated parameters to reduce training costs, while ensuring that the effect is equivalent to full fine-tuning.

[0194] Initialize the parameters of the large language base model and load the pre-trained weights (the large language base model is pre-trained and used directly) to ensure that the model has basic natural language understanding and generation capabilities when fine-tuning begins;

[0195] Focus on optimizing the model's performance in handling medical dialogue scenarios to ensure that it can give accurate answers in tasks such as medical consultation, disease diagnosis, and drug recommendations;

[0196] Define the task structure as medical question-answering, and use the medical knowledge question-answer pairs obtained in step 3 as training data;

[0197] Set training parameters and LoRA parameters;

[0198] The training data is input into the model in batches. Each batch is trained through supervised learning to adapt the model from the pre-trained general knowledge to the specific tasks in the medical field. The loss function is used to quantify the difference between the answers generated by the model and the standard answers, and the model parameters are adjusted through the back-propagation algorithm.

[0199] After each round of the training phase, the model’s performance is evaluated using the medical validation set. The accuracy and coherence of the medical answers generated by the model are evaluated to determine its generalization ability on unseen data.

[0200] The other steps and parameters are the same as those in Specific Embodiments 1 to 9.

[0201] Construct the evaluation index SUS for medical question answering tasks; the specific process is:

[0202] The SUS evaluation indicators are divided into three dimensions: Safety, Usability, and Smoothness:

[0203] Safety: The safety dimension focuses on ensuring that the generated responses do not pose potential health risks to users. For example, it checks whether the generated answers contain wrong or harmful medical advice, such as recommending inappropriate drugs or treatments. In the medical field, misinformation can lead to serious health consequences, and ensuring the safety of information is crucial. This dimension measures whether the medical information provided by the model is risk-free and whether the results obtained will not mislead users;

[0204] Usability: The usability dimension evaluates the extent to which the generated responses reflect medical expertise. It examines the accuracy, professionalism, and consistency of the answers with clinical practice. A highly usable response should be accurate, in line with medical standards, and have practical application value for medical practitioners or patients. This dimension measures the depth of the model's understanding of medical concepts and whether the generated answers can provide useful suggestions to users;

[0205] Fluency: The fluency dimension evaluates the language quality and coherence of the generated answers. Even if the content is medically correct, the way it is expressed will directly affect the user experience. Fluency focuses on the naturalness, fluency, and ease of understanding of the language, ensuring that the answers can clearly and accurately convey complex medical information. This dimension measures the model's ability as a language model and its level of effective communication of medical information.

[0206] For each dimension described, a three-point rating scale was used:

[0207] 1 point: indicates failure, the answer does not meet expectations;

[0208] 2 points: acceptable, the answer meets the minimum requirement;

[0209] 3 points: Good, the answer meets the ideal standard.

[0210] The SUS indicator is used to evaluate the ability of the fine-tuned model in the medical field; the specific process is as follows:

[0211] SUS is used to evaluate the performance of the model in medical scenarios, including testing medical question-answering tasks to evaluate the model's comprehensive performance in different medical scenarios, with a focus on the model's ability to respond to complex medical problems, its effectiveness in treating symptoms, and user acceptance and understanding of the model's answers.

[0212] Base weights: derived from the pre-trained weights of a general large language model on massive data, providing the model with general capabilities for multiple types of tasks;

[0213] Fine-tuning data: Fine-tuning data using medical question-answering instructions constructed in the knowledge graph;

[0214] Fine-tuning module: used to fine-tune the base weights through the LoRA fine-tuning method;

[0215] Fine-tuning weights: weights after fine-tuning on fine-tuning data. These weights are trained using medical datasets to optimize specific tasks, enabling the model to handle complex tasks in the medical field.

[0216] Input module: used to receive and process user text input, such as medical questions, symptom descriptions, and disease information, and send the received text to the large language model for processing;

[0217] Output module: used to generate and output the model's answers. Based on the fine-tuned large language model weights and the base weights, it generates accurate and detailed medical advice or answers for user input, ensuring that the content is accurate, in line with medical standards, and with fluent language.

[0218] The following examples are used to verify the beneficial effects of the present invention:

[0219] Embodiment 1:

[0220] In order to solve the problem that the general large language model performs poorly in the professional field of medicine, the present invention provides a method for fine-tuning a large medical language model, comprising the following steps:

[0221] The large language base module is used to select an open source large language model in a general field as the base model. The selected base model is LLaMA-7B. LLaMA is a series of multi-language open source large language models that are powerful in processing multiple tasks. The 7 billion (7B) parameter version is more lightweight, easy to train and reason under limited resources, and suitable for medical tasks;

[0222] The knowledge instance sampling module is used to sample knowledge instances from the knowledge graph. The medical knowledge graph CMeKG is selected to randomly sample instances whose “Type” fields are “Disease”, “Drug” and “Symptom”.

[0223] The question-answering data generation module is used to generate instruction data based on knowledge instances using a large language model;

[0224] 3.1. The knowledge sampled from the CMeKG knowledge graph is classified according to the Type field;

[0225] 32. Write question templates based on different single-type knowledge. For example, for a knowledge instance of disease type, the question templates are: Question Template 1: What disease is this?; Question Template 2: What are the symptoms of the disease?; Question Template 3: How to treat the disease?;

[0226] 3. Write prompt according to the question template and generate instruction data through OpenAI API; the specific process is:

[0227] 331. Merge the "Knowledge" field in the corresponding knowledge instance with the question template to construct a prompt. For example, a knowledge instance has a "Type" of "Disease" and a "Knowledge" of "{"class":"One hundred common diseases","Central word":"Liver cancer","Drug treatment":["Regorafenib","Antiviral drugs effective for hepatitis B or C","Sorafenib"],"High-incidence areas":["Sub-Saharan Africa"],"High-risk factors":["Obesity","HBV DNA is too high", "Chronic alcoholism", "Male", "Chronic hepatitis B infection", "Family history of liver cancer", "Chronic hepatitis C cirrhosis", "Core promoter mutation", "Cirrhosis", "HCV superinfection", "Elderly valvular heart disease", "Hepatitis B e antigen", "Diabetes"],"Site of disease": ["liver"],"Auxiliary examination": ["liver function test"],"Medical history": ["Long-term history of chronic hepatitis B"];}", then the type corresponding to this knowledge instance is disease, and it should be combined with the disease type question template "Question template 1: What disease is this?; Question template 2: What are the symptoms of the disease?; Question template 3: How to treat the disease?";

[0228] After combination, the prompt is: "Please generate questions and answers about the disease between doctors and patients based on the given information. The types of questions include but are not limited to: 1. What is this disease? 2. What are the symptoms of the disease? 3. How to treat the disease?: "class": "100 common diseases", "central word": "liver cancer", "drug treatment": ["regorafenib", "antiviral drugs effective for hepatitis B or C", "sorafenib"], "high-incidence areas": ["sub-Saharan Africa"], "high-risk factors": ["obesity", "HBV DNA is too high", "chronic alcoholism", "male", "chronic hepatitis B infection", "family history of liver cancer", "chronic hepatitis C cirrhosis", "core promoter mutation", "cirrhosis", "HCV superinfection", "senile heart valve disease", "hepatitis B e antigen", "diabetes"], "site of onset": ["liver"], "auxiliary examinations": ["liver function test"], "medical history": ["long-term history of chronic hepatitis B"];";

[0229] 332. Use ChatGPTAPI in OpenAI to input the prompt into the API and get the corresponding question-answer pair from the API return value. Example: "question: What is liver cancer?; answer: Liver cancer is a malignant tumor originating from the liver, usually caused by factors such as chronic hepatitis or cirrhosis. The early symptoms are not obvious, but may cause symptoms such as pain in the liver area, indigestion, and fatigue.";

[0230] 3.4. Integrate single-type knowledge instances to obtain cascade type instances, write prompts according to the cascade type, and generate cascade type question and answer data through OpenAI API; the specific process is:

[0231] 341. Combine the related knowledge instances sampled from the medical knowledge graph CMeKG, and the combined types are disease-symptom, disease-drug, or disease-drug-symptom;

[0232] For example, in the CMeKG knowledge graph, if the "Type" field is "Disease", that is, a disease type knowledge instance, the value of the "central word" key record in its dictionary is the disease name, which can be unified with the content type of the dictionary's "indications", "contraindications" and other key records in the drug type knowledge instance whose "Type" field is "Drug", then the disease and drug type knowledge instances can be cascaded into disease-drug type knowledge;

[0233] 342. Taking the disease-drug cascade as an example,

[0234] 11) First, extract the content of the "Knowledge" field of a knowledge instance entity1 in the disease to obtain the "knowledge" field; the example is: "{"class":"One hundred common diseases","Central word":"Liver cancer","Drug treatment":["Regorafenib","Antiviral drugs effective for hepatitis B or C","Sorafenib"],"High-incidence areas":["Sub-Saharan Africa"],"High-risk factors":["Obesity","HBV DNA is too high","Chronic alcoholism","Male","Chronic hepatitis B infection","Family history of liver cancer","Chronic hepatitis C cirrhosis","Core promoter mutation","Cirrhosis","HCV superinfection","Elderly valvular heart disease","Hepatitis B e antigen","Diabetes"],"Site of onset":["Liver"],"Auxiliary examination":["Liver function test"],"Medical history":["Long-term history of chronic hepatitis B"];}";

[0235] The content is stored in the form of a dictionary. Get the value value1 in the "central word" key of entity1, then value1 = liver cancer;

[0236] 12) Match each key value in the "knowledge" field of the drug with value1;

[0237] An example of knowledge in medicine is: "{"class":"Western medicine","Central word":"bp element","Classification":["Cholelytic agent / Cholelitholytic agent / Hepatoprotective drug","Other immunosuppressants / Other respiratory system drugs","Immunoenhancer"],"OTC type":["Class A OTC","Class B OTC"],"Specification":["50mg per vial","10mg / bottle"],"Contraindications":["Prohibited for those who are allergic to this product"],"Drug type":["Prescription drug"],"Indications":["Adjuvant treatment of other cancers","Bronchitis","Urticaria","Tracheitis","Pancreatic cancer","Various diseases caused by other immune dysfunction","Adjuvant treatment of various malignant tumors","Soft tissue sarcoma","Psoriasis","Cancer","Gastric cancer","Hepatitis","Breast cancer","Treatment of urticaria, etc.","Liver cancer","Measles infection","Bronchopulmonary carcinoma","Asthma","Soft tissue type striated muscle Sarcoma"],"Storage":["Seal or seal tightly, store in a cool (not exceeding 20℃) dry place","Seal tightly or seal tightly, store in a cool and dry place"],"Adverse reactions":["Pain","Dizziness","Urticaria","Palpitations and urticaria","Fever","Redness and swelling","Nausea","Measles infection","Palpitations","Chest tightness"],"Precautions":["Stop using if the diameter of the injection site is more than 10cm","Use with caution in patients with allergic constitution"],"Validity period":["Tentatively set at 24 months"],"Drug supervision classification":["National basic medicine; injection is a national medical insurance drug","External medicine","Anesthetic medicine"],"Ingredients":["Sodium chloride","Ribonucleic acid extracted from the spleen of healthy pigs","The name is mannitol","Sugar","Ribonucleic acid III","Ribonucleic acid","Mannitol"],"Properties":["White or off-white freeze-dried block or powder, easily soluble in water"]}",

[0238] In this instance, the value corresponding to the "indications" key is a list, one element of which is "liver cancer", indicating a successful match. Let the knowledge instance be entity2, and concatenate the key value of the "knowledge" field corresponding to entity2 with the content of entity1 to construct a new disease-drug type knowledge instance;

[0239] 13) Since all the contents in the drug have not been traversed, step 12) needs to be repeated until the list traversal is completed to ensure that all the knowledge about the central word of entity1 is spliced. Finally, two pieces of relevant knowledge are found. The spliced ​​disease-drug type knowledge example is:

[0240] [{"class":"Hundreds of common diseases","Central word":"liver cancer",……},{"class":"Western medicine","Central word":"bp element",……,"Indications":["Adjuvant treatment of other cancers","Bronchitis","Urticaria","Tracheitis","Pancreatic cancer","Various diseases caused by other immune dysfunction","Adjuvant treatment of various malignant tumors","Soft tissue sarcoma","Psoriasis","Cancer","Gastric cancer","Hepatitis","Breast cancer","Treatment of urticaria, etc.","Liver cancer","Measles infection","Bronchopulmonary carcinoma","Asthma","Soft tissue type rhabdomyosarcoma"],……},{"class":"Western medicine","Central word":"Triphasic contraceptive tablets",……,"Contraindications":["Stroke","Liver cancer","Endometrial cancer","Diabetes","Anemia","Sclerosis","Pruritus","Jaundice"],……}];

[0241] 14) If the query fails, for example, when value1 = "hypertension" but there is no matching value in the drug knowledge list, use SNOMED CT, a synonym database in the medical field, to perform a synonym query;

[0242] The method of using SNOMED CT query is to connect to the REST API service of SNOMED CT through the requests library, configure the query parameter "term" as "hypertension", use the requests.get() method to initiate an HTTP GET request, if the returned status code is 200, the request is successful, use response.json() to parse the returned JSON data, and extract the term value by traversing the results['items'] list. The term is the standard synonym found in the query. In the example, the first term value is "primary hypertension";

[0243] Assign the obtained synonym "primary hypertension" to value1, and repeat step 12) to perform the query;

[0244] 343. Targeting disease-drug-symptom combinations,

[0245] 1) The disease-drug cascade type is type 1 and the symptoms are type 2:

[0246] 2) First, extract the "knowledge" field content of a knowledge instance entity1 in the drug; the example is: "[{"class":"One hundred common diseases","Central word":"liver cancer",...},{"class":"Western medicine","Central word":"bp element",...,"Indications":["Adjuvant treatment of other cancers","Bronchitis","Urticaria","Tracheitis","Pancreatic cancer","Various diseases caused by other immune dysfunction","Adjuvant treatment of various malignant tumors" ,"Soft tissue sarcoma","psoriasis","cancer","gastric cancer","hepatitis","breast cancer","treatment of urticaria, etc.","liver cancer","measles infection","bronchial lung cancer","asthma","soft tissue type rhabdomyosarcoma"],……},{"class":"Western medicine","central word":"triphasic contraceptive tablets",……,"contraindications":["stroke","liver cancer","endometrial cancer","diabetes","anemia","sclerosis","pruritus","jaundice"],……}]";

[0247] The content is stored in the form of a dictionary list. Get the value value1 in the "central word" key of entity1, then value1 = liver cancer;

[0248] 3) Match each key value in the "knowledge" field of the symptom with value1;

[0249] An example of knowledge in symptoms is: "{"Central word":"fatigue","Examination":["non-essential trace elements","asparagine","glutamine test","4-tibia proline"],"Related diseases":["tuberculosis","liver cancer","osteoarthritis of pine caterpillar disease"],"Part of the disease":["whole body"],"Department":["Nutrition Department","Internal Medicine Department","Physical Examination and Health Department"],"Related symptoms":["congestive heart failure","arm pain","abdominal pain during menstruation due to uterine blood stasis","hardened skin"]}",

[0250] In this instance, the value corresponding to the "related diseases" key is a list, and the second element of the list is "liver cancer", indicating a successful match. Let the knowledge instance be entity2, and concatenate the key value of the "knowledge" field corresponding to entity2 with the content of entity1 to construct a new disease-drug-symptom type knowledge instance;

[0251] 4) Since all the contents in the drug have not been traversed, step 3) needs to be repeated until the list traversal is completed to ensure that all the knowledge about the central word of entity1 is spliced. Finally, a total of one piece of relevant knowledge is found. The spliced ​​disease-drug-symptom type knowledge example is:

[0252] [{"class":"100 common diseases","keyword":"liver cancer",……},{"class":"Western medicine","keyword":"bp element",……,"indications":["adjuvant treatment of other cancers","bronchitis","urticaria","tracheitis","pancreatic cancer","other diseases caused by low immune function","adjuvant treatment of various malignant tumors","soft tissue sarcoma","psoriasis","cancer","gastric cancer","hepatitis","breast cancer","treatment of urticaria, etc.", "liver cancer", "measles infection", "bronchial lung cancer", "asthma", "soft tissue type rhabdomyosarcoma"],……},{"class":"Western medicine","keyword":"triphasic contraceptive tablets",……,"contraindications":["stroke","liver cancer","endometrial cancer","diabetes","anemia","sclerosis","pruritus","jaundice"],……},{"keyword":"fatigue",……,"related diseases":["tuberculosis","liver cancer","osteoarthritis of pine caterpillar disease"],……}];

[0253] 344. Write question templates for disease-symptom, disease-drug, or disease-drug-symptom. An example of a disease-symptom-drug question template is; "Generate a question-answer pair that contains detailed patient background and clear medical questions. The treatment of the disease should be explained in detail, such as drug treatment and non-drug treatment details. Please make sure to cover different knowledge points in the answer and provide suggestions.";

[0254] 3.4.5. Write a prompt based on each question template. An example of a prompt for a disease-drug-symptom type question template is:

[0255] prompt: Based on the given disease, corresponding drug, and symptom information, please generate questions and answers about disease-drug-symptoms between doctors and patients. The question types include but are not limited to: 1. What should patients with symptoms and diagnosed with the disease pay attention to when using drugs? ; 2.: Introduce the symptoms, causes, and treatment plans of the disease: [{"class":"100 common diseases","central word":"liver cancer","drug treatment":["regorafenib","antiviral drugs effective for hepatitis B or C","sorafenib"],"high-incidence areas":["sub-Saharan Africa"],"high-risk factors":["obesity","HBV DNA is too high","Chronic alcoholism","Male","Chronic hepatitis B infection","Family history of liver cancer","Chronic hepatitis C cirrhosis","Core promoter mutation","Cirrhosis","HCV superinfection","Elderly valvular heart disease","Hepatitis B e antigen","Diabetes"],"Site of disease":["Liver"],"Auxiliary examination":["Liver function test"],"Medical history":["Long-term history of chronic hepatitis B"];},{"class":"Western medicine","Central word":"bp element","Classification":["Cholelytic agent / Cholelitholytic agent / Hepatoprotective drug","Other immunosuppressants / Other respiratory system drugs","Immunoenhancer"],"OTC type":["Class A OTC","Class B OTC"],"Specification":["50mg per vial","10mg / bottle"],"Contraindications":["Prohibited for those who are allergic to this product"],"Drug type":["Prescription drug"],"Indications":["Adjuvant treatment of other cancers","Bronchopulmonary ","Urticaria","Tracheitis","Pancreatic cancer","Other diseases caused by low immune function","Adjuvant treatment of various malignant tumors","Soft tissue sarcoma","Psoriasis","Cancer","Gastric cancer","Hepatitis","Breast cancer","Treatment of urticaria, etc.","Liver cancer","Measles infection","Bronchopulmonary carcinoma","Asthma","Soft tissue type rhabdomyosarcoma"],"Storage":["Seal or seal tightly, store in a cool (not exceeding 20℃) dry place","Seal tightly or seal tightly, store in a cool and dry place"],"Adverse reactions":["Pain","Dizziness","Urticaria","Palpitations and urticaria","Fever","Redness and swelling","Nausea","Measles infection","Palpitations","Chest tightness"],"Precautions":["Stop using if the diameter of the injection site is more than 10cm","Use with caution in patients with allergic constitution"],"Validity period":["Tentatively 24 months"],"Drug supervision classification":["National basic medicine;The injection is a national medical insurance drug","External use drugs","Anesthetic drugs"],"Ingredients":["Sodium chloride","Ribonucleic acid extracted from the spleen of healthy pigs","The name is mannitol","Sugar","Ribonucleic acid III","Ribonucleic acid","Mannitol"],"Properties":["White or off-white freeze-dried blocks or powder, easily soluble in water"]},{"class":"Western medicine","Central words":"Triphasic contraceptive tablets","Third-level classification":["Short-acting oral contraceptives"],"First-level classification":["Gynecological drugs"],"Second-level classification":["Contraceptives"],"Adverse reactions":["Vaginal bleeding","Headache","Stomach pain","Pregnancy reaction","Nausea","Bleeding","Depression"],"Precautions":["This product should be properly stored to prevent children from taking it by mistake.","Store in a dark, airtight, cool place. ","Please read the instructions carefully before using the medicine."],"Dosage form":["Tablets: 21 tablets per pack, divided into yellow, white, and brown tablets. "],"Contraindications":["Stroke","Liver cancer","Endometrial cancer","Diabetes","Anemia","Sclerosis","Pruritus","Jaundice"],"Subjects":["Gynecological and obstetrical drugs"]},{"Keywords":"Fatigue","Examinations":["Non-essential trace elements","Asparagine","Glutamine test","4-tibia proline"],"Related diseases":["Tuberculosis","Liver cancer","Pine caterpillar disease and osteoarthritis"],"Part of the disease":["Whole body"],"Departments":["Nutrition department","Internal medicine","Physical examination and health care department"],"Related symptoms":["Congestive heart failure","Arm pain","Uterine blood stasis and abdominal pain during menstruation","Hard skin"]}];

[0256] 346. Use the ChatGPTAPI in OpenAI, input the prompt into the API, and obtain the corresponding question-answer pair from the API return value. The obtained question-answer pair example is: "Question: The patient was diagnosed with liver cancer due to a long history of chronic hepatitis B and liver function tests. What are the treatments for liver cancer? Answer: Treatments for liver cancer include drug therapy, surgical treatment, radiotherapy, and targeted therapy. In drug therapy, regorafenib and sorafenib are commonly used targeted drugs that prevent tumor growth by inhibiting tumor angiogenesis. Surgical treatment is suitable for patients with early liver cancer, and the goal is to control the disease by removing the tumor. Radiotherapy is used to locally control tumor growth. For patients with chronic hepatitis B, the use of antiviral drugs is also key to reduce further damage to the liver by the virus. The treatment plan for each patient needs to be formulated according to the specific situation, comprehensively evaluating the condition, liver function, and the patient's physical condition.";

[0257] The large language base model fine-tuning module is used to fine-tune the large language base model using the generated question-answering data to obtain a fine-tuned large language base model;

[0258] The specific process is:

[0259] Download pre-trained weights from LLaMA-7B to ensure that the model has basic natural language understanding and generation capabilities when fine-tuning begins;

[0260] Define the task structure as medical question-answering, and use the medical knowledge question-answer pairs obtained in step 3 as the training set;

[0261] Define model training parameters epoch = 10, batch_size = 16, define LoRA parameters r = 8;

[0262] The training data is input into the model in batches. Each batch is trained through supervised learning to adapt the model from pre-trained general knowledge to specific tasks in the medical field. The cross-entropy loss function is used to quantify the difference between the answers generated by the model and the standard answers, and the model parameters are adjusted through the back-propagation algorithm.

[0263] The training set is the question-answering data generated by the question-answering data generation module;

[0264] Fine-tune the large language base model using supervised learning methods;

[0265] The optimization method adopts the LoRA method;

[0266] Initialize the large language base model parameters and load the pre-trained weights;

[0267] The large language base model is trained until convergence to obtain a fine-tuned large language base model.

[0268] The response generation module is used to answer the pending medical knowledge questions based on the fine-tuned large language base model.

[0269] Construct the evaluation index SUS for medical question answering tasks;

[0270] The SUS indicator was used to evaluate the ability of the fine-tuned model in the medical field. A manual evaluation method was adopted. Five evaluators with medical backgrounds were recruited to use a three-point scale to evaluate the SUS indicator performance of LLaMA, Alpaca, ChatGLM and the fine-tuned herbal model in the medical field. The average scores are shown in Table 1.

[0271] Table 1. Experimental results of model SUS scoring

[0272]

[0273]

[0274] From the experimental results in Table 1, we can see that in terms of usability, the fine-tuned Bencao model scores 2.12, which is higher than LLaMA's 1.21, Alpaca's 2.05, and ChatGLM's 1.93; in terms of fluency, the fine-tuned Bencao model scores 2.47, which is higher than LLaMA's 1.58, Alpaca's 2.30, and ChatGLM's 2.41; in terms of security, the Bencao model scores 2.88, which is only slightly lower than LLaMA's 2.93, and higher than Alpaca's 2.64 and ChatGLM's 2.59. The usability of the medical task has been significantly improved, the fluency of the response has also increased, and the damage to security is relatively low.

[0275] The present invention may also have other various embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. A medical question-answering system based on a large language model, characterized by: The system comprises: a large language base module, a knowledge instance sampling module, a question and answer data generation module, a large language base model fine-tuning module and a reply generation module; The large language base module is used to select an open source large language model in a general field as a base model; The knowledge instance sampling module is used to sample knowledge instances from the knowledge graph; The question-answer data generation module is used to generate question-answer data according to knowledge instances using a large language base model; The large language base model fine-tuning module is used to fine-tune the large language base model using the generated question-answering data to obtain a fine-tuned large language base model; The response generation module is used to answer the pending medical knowledge questions based on the fine-tuned large language base model.

2. A medical question-answering system based on a large language model according to claim 1, characterized in that: The large language base module is used to select an open source large language model in a general field as a base model; the specific process is: The large language base model is an open source large language model, and the open source large language model is LLaMA, BLOOM or ChatGLM.

3. A medical question-answering system based on a large language model according to claim 2, characterized in that: The question-answering data generation module is used to generate question-answering data according to knowledge instances using a large language base model; the specific process is as follows: Randomly sample knowledge instances from the medical knowledge graph CMeKG; Each knowledge instance is a group of a disease and its corresponding knowledge, a medicine and its corresponding knowledge, a symptom and its corresponding knowledge; The format of each knowledge instance is as follows: Type: Knowledge: The Type field records the type of knowledge instance; The Knowledge field records the content of the knowledge instance.

4. A medical question-answering system based on a large language model according to claim 3, characterized in that: The knowledge instance sampling module is used to sample knowledge instances from the knowledge graph; the specific process is as follows: 3.

1. Classify the knowledge instances randomly sampled from the medical knowledge graph CMeKG to obtain the types of knowledge instances; the specific process is as follows: Determine the type of the randomly sampled knowledge instance according to the Type field in the randomly sampled knowledge instance; 32. Write question templates for different types of knowledge instances; 3. Write prompts according to the question template and generate single-type question-answering data through OpenAI API; 3.

4. Integrate single-type knowledge instances to obtain cascade type instances, write prompts according to the cascade type, and generate cascade type question and answer data through OpenAI API.

5. A medical question-answering system based on a large language model according to claim 4, characterized in that: In the above three or two, question templates are written for different types of knowledge instances; the specific process is: Question templates written for the type "Type" is "disease"; Question templates written for the type "Type" is "drug"; A question template written for the type "Type" is "Symptom".

6. A medical question-answering system based on a large language model according to claim 5, characterized in that: In the above three steps, a prompt is written according to the question template, and single-type question-answering data is generated through OpenAI API; the specific process is:

331. Write prompts according to each question template; 332. Use ChatGPTAPI in OpenAI API, input each prompt into ChatGPTAPI, and obtain the corresponding question and answer data from the return value of ChatGPTAPI; the specific process is: Extract the choices[0].message.content field from the chat_completion object returned by the ChatGPTAPI; The format of the content recorded in the choices[0].message.content field is: question: answer:.

7. A medical question answering system based on a large language model according to claim 6, characterized in that: The specific process of using ChatGPTAPI in OpenAIAPI in the 332 is: Register an OpenAI account, create a ChatGPTAPI key, load the openai library through the Python client, configure the obtained ChatGPTAPI key to the openai.api_key parameter of the code, and configure the version name of GPT 3.5 to the model parameter.

8. A medical question-answering system based on a large language model according to claim 7, characterized in that: In the third and fourth steps, single-type knowledge instances are integrated to obtain cascade type instances, prompts are written according to the cascade type, and cascade type question-answering data is generated through OpenAI API; the specific process is as follows:

341. Combine the related knowledge instances sampled from the medical knowledge graph CMeKG, and the combined types are disease-drug, disease-symptom or disease-drug-symptom; 342 1) Targeting disease-drug combination; the specific process is: 11) When the type is a disease-drug combination, extract the content of the "knowledge" field of the disease; Get the value value1 of the key "central word" in the "knowledge" field; The content of the "knowledge" field of the disease is stored in dictionary form; 12) Match each key value in the "knowledge" field of the drug with value1; If the match is successful, the key value in the "knowledge" field corresponding to the drug is added to the content of the "knowledge" field of the disease to obtain a new disease-drug combination; If the match fails, no processing will be done; 13) Repeat 12) until all key values ​​in the "knowledge" field of the drug are traversed to determine whether there is a successful match; If there is a final new disease-drug combination obtained; If it does not exist, execute 14); 14) Obtain a synonym list of value1 based on the synonym library, assign the synonyms in the synonym list to value1, repeat 12) until a match is successful, obtain a new disease-drug combination, and end; if no match is successful, abandon the disease-drug combination; 2) Targeting the disease-symptom combination; the specific process is: 21) When the type is a disease-symptom combination, extract the content of the "knowledge" field of the disease; Get the value value1 of the key "central word" in the "knowledge" field; The content of the "knowledge" field of the disease is stored in dictionary form; 22) Match each key value in the "knowledge" field of the symptom with value1; If the match is successful, the key value in the "knowledge" field corresponding to the symptom is added to the content of the "knowledge" field of the disease to obtain a new disease-symptom combination; If the match fails, no processing will be done; 23) Repeat 22) until all key values ​​in the "knowledge" field of the symptom are traversed to determine whether there is a successful match; If there is a final new disease-symptom combination obtained; If it does not exist, execute 24); 24) Obtain a synonym list of value1 based on the synonym library, assign the synonyms in the synonym list to value1, repeat 22) until a match is successful, obtain a new disease-symptom combination, and end; if no match is successful, abandon the disease-symptom combination; 343. Targeting the disease-drug-symptom combination; the specific process is: 1) The new disease-drug combination obtained in 14) is regarded as type 1, and the symptoms are regarded as type 2; 2) Extract the content of the "knowledge" field of type 1; the content of the "knowledge" field of type 1 is stored in the form of a dictionary list; select the value value1 of the key "central word" in the first element of the list; The element is a dictionary; 3) Match each key value in the "knowledge" field of type 2 with value1; If the match is successful, the key value in the "knowledge" field corresponding to type 2 is added to the content of the "knowledge" field of type 1 to obtain a new type 1-type 2 combination; If the match fails, no processing will be done; 4) Repeat step 3) until all key values ​​in the "knowledge" field of type 2 are traversed to determine whether there is a successful match; If it exists, obtain the final new type one-type two combination; If it does not exist, execute 5); 5) Obtain a synonym list of value1 based on the synonym library, assign the synonyms in the synonym list to value1, repeat 3) until a match is successful, obtain a new type 1-type 2 combination, and end; if no match is successful, abandon the type 1-type 2 combination; 344. Write question templates for disease-symptom, disease-drug, or disease-drug-symptom; 3.4.

5. Write prompts according to each question template; 3.4.

6. Use ChatGPTAPI in OpenAI API, input each prompt into ChatGPTAPI, and obtain the corresponding question and answer data from the return value of ChatGPTAPI.

9. A medical question answering system based on a large language model according to claim 8, characterized in that: In the above steps, ChatGPTAPI in OpenAI API is used to input each prompt into ChatGPTAPI and obtain the corresponding question and answer data from the ChatGPTAPI return value; The specific process is: Extract the choices[0].message.content field from the chat_completion object returned by the ChatGPTAPI; The format of the content recorded in the choices[0].message.content field is: question: answer: The specific process of using the ChatGPTAPI in OpenAIAPI is as follows: Register an OpenAI account, create a ChatGPTAPI key, load the openai library through the Python client, configure the obtained ChatGPTAPI key to the openai.api_key parameter of the code, and configure the version name of GPT 3.5 to the model parameter.

10. A medical question answering system based on a large language model according to claim 9, characterized in that: The large language base model fine-tuning module is used to fine-tune the large language base model using the generated question-answering data to obtain a fine-tuned large language base model; The specific process is: The training set is the question-answering data generated by the question-answering data generation module; Fine-tune the large language base model using supervised learning methods; The optimization method adopts the LoRA method; Initialize the large language base model parameters and load the pre-trained weights; The large language base model is trained until convergence to obtain a fine-tuned large language base model.

Images