A medical question and answer system based on a large language model

Abstract

The application relates to a medical question and answer system based on a large language model, belongs to the field of large language models in computer field artificial intelligence, and particularly relates to a medical question and answer system based on a large language model. The application aims to solve the problems of low accuracy of a general large language model in the medical field and poor medical problem solving capability. The system comprises the following modules: a large language base module for selecting an open source large language model in a general field as a base model; a knowledge instance sampling module for sampling knowledge instances from a knowledge graph; a question and answer data generation module for generating question and answer data according to the knowledge instances by using the large language base model; a large language base model fine tuning module for fine tuning the large language base model by using the generated question and answer data to obtain a fine tuned large language base model; and a reply generation module for answering a medical knowledge problem to be processed based on the fine tuned large language base model.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of computer, specifically to the field of large language model in artificial intelligence, and particularly relates to a medical question and answer system based on a large language model. BACKGROUND

[0002] Since November 2022, large language models represented by ChatGPT have attracted great interest due to their excellent performance in understanding instructions and generating human-like responses. Compared with smaller models, large language models, with their large-scale data training and complex deep learning architecture, have shown strong generalization ability in various natural language processing tasks and have unique emergent ability in solving zero-shot tasks or complex tasks.

[0003] Although large language models have many advantages, they are not specifically designed for the medical field. When dealing with such professional problems, the general domain knowledge in large language models is often insufficient to answer them, and relevant knowledge in the specific medical field is crucial. The lack of such knowledge may lead to suboptimal diagnostic accuracy, drug recommendations, and medical advice, which may endanger patients.

[0004] In the medical field, it is difficult to obtain high-quality training data, and data sets related to medical question and answer tasks are relatively rare. The data provided on existing medical question and answer platforms often lack strict industry standard audits, and their professionalism is not up to the authoritative standards of the medical field. Although data can be collected from these platforms through crawling technology, due to the high noise and unstable quality of these data, fine-tuning using such data often cannot meet the accuracy and professionalism requirements of the output answers.

[0005] Therefore, how to construct a high-quality medical question and answer data set and effectively improve the performance of large language models in the medical field, which has a high degree of professionalism and relies heavily on the accuracy of generated knowledge, has become a problem that needs to be solved in the current technical field, and is also of great significance for the further expansion of large language models in the application fields of intelligent medical care and clinical decision support. SUMMARY

[0006] The purpose of the present application is to solve the problem of low accuracy of general large language models in the medical field and poor ability to solve medical problems, and to propose a medical question and answer system based on a large language model.

[0007] A medical question and answer system based on a large language model includes a large language base module, a knowledge instance sampling module, a question and answer data generation module, a large language base model fine-tuning module, and a reply generation module.

[0008] The large language base module is used to select a general-domain open-source large language model as the base model;

[0009] The knowledge instance sampling module is used to sample knowledge instances from the knowledge graph;

[0010] The question-and-answer data generation module is used to generate question-and-answer data based on knowledge instances using a large language foundation model.

[0011] The large language base model fine-tuning module is used to fine-tune the large language base model using the generated question-and-answer data to obtain a fine-tuned large language base model.

[0012] The response generation module is used to answer medical knowledge questions based on a finely tuned large language base model.

[0013] The beneficial effects of this invention are as follows:

[0014] This invention utilizes the CMeKG knowledge graph to construct medical knowledge question-answer pairs, which are obtained through supervised fine-tuning of the LLaMA-7B pedestal model. The ability to process tasks and expressive capabilities in medical scenarios is significantly improved, especially in terms of usability and fluency, without significantly reducing the security of the answers.

[0015] This invention includes a large language base module for selecting a general-domain open-source large language model as the base model; a knowledge instance sampling module for sampling knowledge instances from a knowledge graph; a question-and-answer data generation module for generating question-and-answer data based on the knowledge instances using the large language base model; a large language base model fine-tuning module for fine-tuning the large language base model using the generated question-and-answer data to obtain a fine-tuned large language base model; and a response generation module for answering medical knowledge questions based on the fine-tuned large language base model. This improves the accuracy of the general-domain large language model in answering questions in the medical field and enhances its ability to solve medical problems. Attached Figure Description

[0016] Figure 1 Flowchart for a medical fine-tuning method for large language models;

[0017] Figure 2 A schematic diagram for constructing multi-type medical question-and-answer data. Detailed Implementation

[0018] Specific Implementation Method 1: This implementation method provides a medical question-answering system based on a large language model, which includes:

[0019] The module consists of a large language foundation module, a knowledge instance sampling module, a question-and-answer data generation module, a large language foundation model fine-tuning module, and a response generation module.

[0020] The large language base module is used to select a general-domain open-source large language model as the base model;

[0021] The knowledge instance sampling module is used to sample knowledge instances from the knowledge graph;

[0022] The question-and-answer data generation module is used to generate question-and-answer data based on knowledge instances using a large language foundation model.

[0023] The large language base model fine-tuning module is used to fine-tune the large language base model using the generated question-and-answer data to obtain a fine-tuned large language base model.

[0024] The response generation module is used to answer medical knowledge questions based on a finely tuned large language base model.

[0025] Constructing an evaluation metric, SUS, for medical question-answering tasks;

[0026] The SUS index was used to evaluate the capabilities of the fine-tuned model in the medical field.

[0027] Specific Implementation Method Two: This implementation method differs from Specific Implementation Method One in that the large language base module selects a general-domain open-source large language model as the base model; the specific process is as follows:

[0028] The large language foundation model is an open-source large language model, which can be LLaMA, BLOOM, or ChatGLM.

[0029] The open-source large language model has scalability and multi-task processing capabilities, and supports fine-tuning the model to adapt to the complex needs of the medical field.

[0030] Based on existing large language models, the selected model needs to possess powerful natural language processing capabilities, capable of handling multiple language environments. Simultaneously, considering the specific needs of the medical field, a model with strong scalability and multi-task processing capabilities is required to ensure its adaptability to the complex demands of medicine. Open-source large language models include LLaMA, BLOOM, and ChatGLM.

[0031] The other steps and parameters are the same as in Specific Implementation Method 1.

[0032] Specific Implementation Method 3: This implementation method differs from Specific Implementation Method 1 or 2 in that the question-and-answer data generation module is used to generate question-and-answer data based on knowledge instances using the large language foundation model;

[0033] The specific process is as follows:

[0034] Randomly sample knowledge instances from the medical knowledge graph CMeKG;

[0035] Each knowledge instance is a set of one of the following: a disease and its corresponding knowledge, a drug and its corresponding knowledge, or a symptom and its corresponding knowledge (each knowledge instance is a disease and its corresponding knowledge, a drug and its corresponding knowledge, or a symptom and its corresponding knowledge).

[0036] The format of each knowledge instance is as follows:

[0037] Type:

[0038] Knowledge:

[0039] The Type field records the type of knowledge instance (disease, symptom, or medication).

[0040] The Knowledge field records the content of knowledge instances (knowledge corresponding to diseases, knowledge corresponding to symptoms, or knowledge corresponding to drugs);

[0041] The knowledge instances are sampled from the medical knowledge graph CMeKG, providing medical knowledge about diseases, drugs, symptoms, etc.

[0042] Examples of disease knowledge in the knowledge base are as follows:

[0043] Type: Disease;

[0044] Knowledge: {"class":"100 Common Diseases","Keyword":"Liver Cancer","Drug Treatment":["Regorafenib","Antiviral Drugs Effective Against Hepatitis B or C","Sorafenib"],"High-Incidence Areas":["Sub-Saharan Africa"],"High-Risk Factors":["Obesity","High HBV DNA Levels","Chronic Alcoholism","Male","Chronic Hepatitis B Infection","Family History of Liver Cancer","Chronic Hepatitis C Cirrhosis","Core Promoter Mutation","Cirrhosis","HCV Co-infection","Valvular Heart Disease in the Elderly","Hepatitis B e Antigen","Diabetes"],"Site of Onset":["Liver"],"Auxiliary Examinations":["Liver Function Tests"],"Medical History":["Long-Term Chronic Hepatitis B History"]}

[0045] Examples of drug knowledge in the knowledge base are as follows:

[0046] Type: Drug;

[0047] Knowledge: {"class":"Western Medicine","Keyword":"Metformin","Appearance":["Sugar-coated or film-coated tablets, white after removing the coating"],"English Name":["Isoflavone Tablets","Glucophage"],"Classification":["Biguanides","Anti-tuberculosis Drugs"],"Specification":["0.25g"],"OTC Type":["Class B OTC","Class A OTC"],"Indications":["Diabetes","Obesity"],"Generic Name":["Isoflavone Tablets"],"Ingredients":["Rifampin and Isoniazid","Isoniazid","Isoniazid 0.1g","Isoniazid 150mg","This product is a compound preparation","Rifampin","Rifampin 300mg","Rifampin 0.15g","Metformin Hydrochloride","Hydrogen Hydrochloride"]}

[0048] Examples of symptom knowledge in the knowledge base are as follows:

[0049] Type: Symptom;

[0050] Knowledge: {"Keywords":"Hair loss","Examination":["Hair mineral analysis"],"Related Diseases":["Alopecia areata","Chronic fatigue syndrome"],"Related Symptoms":["Light brown hair","Dry and brittle hair","Hardened skin"],"Department":["Internal Medicine","Dermatology and Venereology","Radiotherapy and Chemotherapy"],"Affected Area":["Scalp"]}

[0051] The "Type" field in the sampling knowledge includes "Disease", "Drug" and "Symptom".

[0052] The sampling step combines random sampling with targeted sampling based on medical disease, drug, and symptom classifications to cover a variety of medical scenarios.

[0053] The other steps and parameters are the same as in specific implementation method one or two.

[0054] Specific Implementation Method Four: This implementation method differs from one of Specific Implementation Methods One to Three in that the knowledge instance sampling module is used to sample knowledge instances from the knowledge graph;

[0055] The specific process is as follows:

[0056] 31. Classify the knowledge instances randomly sampled from the Medical Knowledge Graph (CMeKG) (disease, symptom, or drug) to obtain the type of knowledge instance; the specific process is as follows:

[0057] The type (disease, symptom, or drug) of the randomly sampled knowledge instance is determined based on the Type field in the randomly sampled knowledge instance;

[0058] 32. Develop question templates for different types of knowledge instances;

[0059] 33. Write a prompt based on the question template and generate single-type question-and-answer data using the OpenAI API;

[0060] Third and fourth, integrate single-type knowledge instances to obtain cascaded type instances, write a prompt based on the cascaded types, and generate cascaded type question-and-answer data through OpenAI API.

[0061] The other steps and parameters are the same as in any of the specific implementation methods one to three.

[0062] Specific Implementation Method Five: This implementation method differs from Specific Implementation Methods One to Four in that, in Specific Implementation Method Three Two, question templates are written for different types of knowledge instances;

[0063] The specific process is as follows:

[0064] A question template was created for the "Type" field, which is "Disease".

[0065] Example:

[0066] Question Template 1: What disease is this?

[0067] Question Template 2: What are the symptoms of the disease?

[0068] Question Template 3: How to treat diseases?

[0069] A question template was created for the "Type" field being "Drug".

[0070] Example:

[0071] Question Template 1: What diseases is this medication suitable for treating?

[0072] Question Template 2: What are the common side effects of medications?

[0073] A question template was created for the "Type" category, which is "Symptom".

[0074] Example:

[0075] Question Template 1: What disease might these symptoms be indicative of?

[0076] Question Template 2: What should be done when symptoms appear?

[0077] The other steps and parameters are the same as in any of the specific implementation methods one to four.

[0078] Specific Implementation Method Six: This implementation method differs from Specific Implementation Methods One through Five in that, in step three, a prompt is written based on a question template, and single-type question-and-answer data is generated using the OpenAI API; the specific process is as follows:

[0079] 331. Write a prompt based on the template for each question;

[0080] Here is an example of a prompt for a disease type question template:

[0081] Prompt: Based on the given disease information, please generate a doctor-patient dialogue about the disease, including but not limited to: 1. What is this disease? 2. What are the symptoms of the disease? 3. How to treat the disease? : "class":"100 common diseases","keyword":"liver cancer","drug treatment":["regorafenib","antiviral drugs effective against hepatitis B or C","sorafenib"],"prevalence areas":["sub-Saharan Africa"],"risk factors":["obesity","high HBV DNA","chronic alcoholism","male","chronic hepatitis B infection","family history of liver cancer","chronic hepatitis C cirrhosis","core promoter mutation","cirrhosis","HCV co-infection","senile valvular heart disease","hepatitis B e antigen","diabetes"],affected site":["liver"],"auxiliary examinations":["liver function tests"],"medical history":["long-term chronic hepatitis B history"];

[0082] Here is an example of a prompt for a question about drug type:

[0083] Prompt: Based on the given drug information, please generate a doctor-patient dialogue about the drug, including but not limited to: 1. What diseases is the drug suitable for treating?; 2. What are the common side effects of the drug? "class":"Western medicine","keyword":"metformin","appearance":["sugar-coated or film-coated tablets, white after removing the coating"],"English name":["Isoflavone tablets","Glucophage"],"classification":["biguanides","anti-tuberculosis drugs"],"specification":["0.25g"],"OTC type":["Class B OTC","Class A OTC"],"indications":["diabetes","obesity"],"generic name":["Isoflavone tablets"],"ingredients":["rifampin and isoniazid","isoniazid","isoniazid 0.1g","isoniazid 150mg","this product is a compound preparation","rifampin","rifampin 300mg","rifampin 0.15g","metformin hydrochloride","hydrochloride"]

[0084] Here is an example of a prompt for a symptom type question template:

[0085] Prompt: Based on the given symptom information, please generate a doctor-patient dialogue about the symptoms, including but not limited to: 1. What disease might these symptoms be indicative of? 2. How should these symptoms be managed? Keywords: "Hair loss", "Examination": ["Hair mineral analysis"], "Related Diseases": ["Alopecia areata", "Chronic fatigue syndrome"], "Related Symptoms": ["Pale brown hair", "Dry and brittle hair", "Hardened skin"], "Department": ["Internal medicine", "Dermatology and venereology", "Radiotherapy and chemotherapy"], "Affected area": ​​["Scalp"]

[0086] 332. Using the ChatGPTAPI in the OpenAI API, input each prompt into the ChatGPTAPI and retrieve the corresponding question-and-answer data from the ChatGPTAPI return value; the specific process is as follows:

[0087] Extract the choices[0].message.content field from the chat_completion object returned by ChatGPTAPI;

[0088] The format of the content (question and answer data) recorded in the choices[0].message.content field is as follows:

[0089] question:

[0090] answer:

[0091] The three are independent;

[0092] For diseases:

[0093] Question: What is liver cancer?

[0094] Answer: Liver cancer is a malignant tumor that originates in the liver. It is usually caused by factors such as chronic hepatitis or cirrhosis. Early symptoms are not obvious, but may cause symptoms such as pain in the liver area, indigestion, and fatigue.

[0095] For drugs:

[0096] Question: What diseases is metformin suitable for treating?

[0097] Answer: Metformin is primarily used to treat diabetes, especially type 2 diabetes. Additionally, it is used to treat obesity and can help control weight.

[0098] For symptoms:

[0099] Question: I've recently noticed a lot of hair loss. What kind of illness could this be a sign of?

[0100] Answer: Hair loss can be related to a variety of conditions, with alopecia areata and chronic fatigue syndrome being common causes. Alopecia areata typically causes sudden, localized hair loss, while chronic fatigue syndrome may be accompanied by symptoms such as dry, brittle hair. If you experience persistent fatigue or other discomfort, further examination is recommended.

[0101] The other steps and parameters are the same as those in any of the specific implementation methods one to five.

[0102] Specific Implementation Method Seven: This implementation method differs from Specific Implementation Methods One through Six in that the specific process of using the ChatGPTAPI in the OpenAI API in step 332 is as follows:

[0103] Register an OpenAI account, create a ChatGPTAPI key, load the openai library through a Python client, configure the obtained ChatGPTAPI key in the openai.api_key parameter of the code, and configure the GPT 3.5 version name in the model parameter.

[0104] For example, “gpt-3.5-turbo”; at the same time, fill in the content key-value of each prompt obtained in step 332 as the message list in the code, and uniformly set the role key-value of the message list to “user”; call the openai.ChatCompletion.create() method to send a request to the OpenAI server, wait for the server to generate text based on the request and return it; the generated response will return a chat_completion object, and the answer content will be extracted from the choices[0].message.content field of the object.

[0105] The other steps and parameters are the same as those in any of the specific implementation methods one to six.

[0106] Specific Implementation Method Eight: This implementation method differs from Specific Implementation Methods One to Seven in that, in steps Three and Four, single-type knowledge instances are integrated to obtain cascaded type instances, a prompt is written based on the cascaded types, and cascaded type question-and-answer data is generated through OpenAIAPI.

[0107] The specific process is as follows:

[0108] 341. Combine the related knowledge instances sampled from the medical knowledge graph CMeKG, and the combined types are disease-drug, disease-symptom, or disease-drug-symptom. 342,

[0110] 1) Disease-drug combination; the specific process is as follows:

[0111] 11) When the type is disease-drug combination, extract the content of the "knowledge" field of the disease;

[0112] Retrieve the value value1 of the key "center word" in the "knowledge" field;

[0113] The contents of the "knowledge" field for the disease are stored in dictionary form;

[0114] 12) Match each key value in the drug's "knowledge" field with value1;

[0115] If a match is found, the key value in the "knowledge" field corresponding to the drug will be added to the content of the "knowledge" field of the disease to obtain a new disease-drug combination.

[0116] If the match fails, no action is taken;

[0117] 13) Repeat step 12) until all key-value pairs in the "knowledge" field of the drug have been traversed, and check if a match exists.

[0118] If a new disease-drug combination can be obtained;

[0119] If it does not exist, proceed to step 14);

[0120] 14) Obtain a list of synonyms for value1 based on the thesaurus, assign the synonyms in the thesaurus to value1, repeat 12) until a match is successful, obtain a new disease-drug combination, and end; if no match is successful, discard the disease-drug combination.

[0121] Thesaurus such as SNOMED CT, UMLS, etc.;

[0122] 2) Targeting disease-symptom combinations; the specific process is as follows:

[0123] 21) When the type is a combination of disease and symptoms, extract the content of the "knowledge" field of the disease;

[0124] Retrieve the value value1 of the key "center word" in the "knowledge" field;

[0125] The contents of the "knowledge" field for the disease are stored in dictionary form;

[0126] 22) Match each key value in the "knowledge" field of the symptoms with value1;

[0127] If a match is found, the key value in the "knowledge" field corresponding to the symptom will be added to the content of the "knowledge" field of the disease to obtain a new disease-symptom combination;

[0128] If the match fails, no action is taken;

[0129] 23) Repeat step 22) until all key-value pairs in the “knowledge” field of the symptoms have been traversed, and determine if a match exists.

[0130] If a new combination of disease and symptoms is obtained;

[0131] If it does not exist, proceed to step 24);

[0132] 24) Obtain a list of synonyms for value1 based on the thesaurus, assign the synonyms in the thesaurus to value1, repeat 22) until a match is successful, obtain a new disease-symptom combination, and end; if no match is successful, discard the disease-symptom combination;

[0133] Thesaurus such as SNOMED CT, UMLS, etc.;

[0134] 343. Targeting the combination of disease, medication, and symptoms; the specific process is as follows:

[0135] 1) The new disease-drug combination obtained in 14) is classified as type one, and the symptoms are classified as type two;

[0136] 2) Extract the content of the "knowledge" field of type 1; the content of the "knowledge" field of type 1 is stored in the form of a dictionary list; select the value1 of the key "center word" in the first element of the list;

[0137] The element is a dictionary;

[0138] A disease is a dictionary, a symptom is a dictionary, and a disease-symptom is a list of dictionaries. When faced with a knowledge instance of a disease-symptom, the core word can be identified by the contents of the first dictionary, which is the first dictionary in the list of dictionaries. An element is a dictionary, which is a very long segment containing many key-value pairs, just like the structure of a single type.

[0139] 3) Match each key value in the "knowledge" field of type two with value1;

[0140] If a match is successful, the key value in the "knowledge" field corresponding to type 2 will be added to the content of the "knowledge" field of type 1 to obtain a new type 1-type 2 combination;

[0141] If the match fails, no action is taken;

[0142] 4) Repeat step 3) until all key-value pairs in the "knowledge" field of type 2 have been traversed, and check if a match exists.

[0143] If a new type 1-type 2 combination exists;

[0144] If it does not exist, proceed to step 5);

[0145] 5) Obtain a list of synonyms for value1 based on the thesaurus, assign the synonyms in the thesaurus to value1, repeat 3) until a match is successful, obtain a new type 1-type 2 combination, and end; if no match is successful, discard the type 1-type 2 combination.

[0146] Thesaurus such as SNOMED CT, UMLS, etc.;

[0147] 344. Create question templates for disease-symptom, disease-medication, or disease-medication-symptom; the specific process is as follows:

[0148] Here is an example of a question template written for each disease-symptom type:

[0149] Question Template 1: What are some easily overlooked symptoms in the early stages of a disease?

[0150] Question Template 2: I've recently developed some symptoms. What could this be, and what tests should I have done?

[0151] Here is an example of a question template for the disease-drug type:

[0152] Question Template 1: What medications should be used to treat this disease, and will there be any side effects?

[0153] Question Template 2: I have a certain disease, what medications are contraindicated for me?

[0154] The example question template corresponding to the disease-medication-symptom type is as follows:

[0155] Question Template 1: What precautions should be taken when using medication for patients who have developed symptoms and been diagnosed with a disease?

[0156] Template 2: Generate a question-and-answer pair containing the patient's detailed background and a clearly stated medical question. The answer should explain the treatment options for the disease in detail, including both pharmacological and non-pharmacological treatments. Please ensure that the answer covers different knowledge points and provides suggestions.

[0157] 3.4.5. Write a prompt based on the template for each question;

[0158] Here is an example of a prompt for the disease-symptom type question template:

[0159] Prompt: Based on the given disease and corresponding symptom information, please generate a doctor-patient question and answer regarding the disease and its symptoms. Question types include, but are not limited to: 1. What are some easily overlooked early symptoms of a disease?; 2. I've recently developed some symptoms; what disease might it be, and what tests are needed? :[{"class":"100 Common Diseases","Keywords":"Liver Cancer","Drug Treatment":["Regorafenib","Antiviral Drugs Effective Against Hepatitis B or C","Sorafenib"],"High-Incidence Areas":["Sub-Saharan Africa"],"High-Risk Factors":["Obesity","HBV"] "High DNA levels", "Chronic alcoholism", "Male", "Chronic hepatitis B infection", "Family history of liver cancer", "Chronic hepatitis C cirrhosis", "Core promoter mutation", "Cirrhosis", "HCV co-infection", "Senile valvular heart disease", "Hepatitis B e antigen", "Diabetes"], "Affected site": ["Liver"], "Auxiliary examinations": ["Liver function test"], "Medical history": ["Long-term chronic hepatitis B history"];}, {"Keywords": "Fatigue", "Examinations": ["Non-essential trace elements", "Aspartic acid", "Glutamate detection", "4-Tibeproline"], "Related diseases": ["Tuberculosis", "Liver cancer", "Pine caterpillar disease osteoarthritis"], "Affected site": ["Whole body"], "Department": ["Nutrition", "Internal medicine", "Physical examination and health care department"], "Related symptoms": ["Congestive heart failure", "Arm pain", "Uterine blood stasis and menstrual abdominal pain", "Skin hardening"]}

[0160] Here is an example of a prompt for the disease-drug type question template:

[0161] Prompt: Based on the given disease and corresponding drug information, please generate a doctor-patient question and answer regarding the disease and medication. Question types include, but are not limited to: 1. What medication should be used to treat this disease, and will it have side effects?; 2. I have a certain disease, and which medications are contraindicated for me? :[{"class":"100 Common Diseases","Keywords":"Liver Cancer","Drug Treatment":["Regorafenib","Antiviral Drugs Effective Against Hepatitis B or C","Sorafenib"],"High-Incidence Areas":["Sub-Saharan Africa"],"High-Risk Factors":["Obesity","HBV"] "High DNA levels", "Chronic alcoholism", "Male", "Chronic hepatitis B infection", "Family history of liver cancer", "Chronic hepatitis C cirrhosis", "Core promoter mutation", "Cirrhosis", "HCV co-infection", "Senile valvular heart disease", "Hepatitis B e antigen", "Diabetes"], "Site of onset": ["Liver"], "Auxiliary examinations": ["Liver function tests"], "Medical history": ["Long-term chronic hepatitis B history"];}, {"class": "Western medicine", "Central keyword": "bp", "Classification": ["Chocorticoids / Cholelithic agents / Hepatoprotective drugs", "Other immunosuppressants / Other respiratory drugs", "Immune enhancers"], "OTC type": ["Class A OTC", "Class B OTC"], "Specifications": ["50mg per vial", "10mg / bottle"], "Contraindications": ["Contraindicated in patients with known hypersensitivity to this product"], "Drug type": ["Prescription drug"], "Indications": ["Adjuvant treatment for other cancers", "Bronchiectasis"], "Bronchiectasis"], "Onset site": ["Liver"], "Auxiliary examinations": ["Liver function tests"], "Medical history": ["Long-term chronic hepatitis B history"]; "Bronchiitis", "Urticaria", "Tracheitis", "Pancreatic Cancer", "Other Diseases Caused by Immunodeficiency", "Adjuvant Treatment of Various Malignant Tumors", "Soft Tissue Sarcoma", "Psoriasis", "Cancer", "Gastric Cancer", "Hepatitis", "Breast Cancer", "Treatment of Urticaria, etc.", "Liver Cancer", "Measles Infection", "Bronchopulmonary Lung Cancer", "Asthma", "Soft Tissue Type Rhabdomyosarcoma"], "Storage": ["Sealed or tightly sealed, store in a cool (not exceeding 20℃) dry place", "Tightly sealed or sealed, store in a cool, dry place"], "Adverse Reactions": ["Pain", "Dizziness", "Urticaria", "Palpitation and Urticaria", "Elevated Body Temperature", "Redness and Swelling", "Nausea", "Measles Infection", "Palpitation", "Chest Tightness"], "Precautions": ["Discontinue use if the diameter of redness and swelling at the injection site is greater than 10cm", "Use with caution in patients with allergic constitution"], "Shelf Life": ["Tentatively 24 months"], "Drug Regulatory Classification": ["National Essential Medicines"]The injection is a national medical insurance drug, an external medicine, and an anesthetic. Its components include sodium chloride, ribonucleic acid extracted from the spleen of healthy pigs, mannitol, sugar, ribonucleic acid III, ribonucleic acid, and mannitol. It is described as a white or off-white freeze-dried block or powder, easily soluble in water. The product is classified as a short-acting oral contraceptive. It is also classified as a gynecological and obstetric drug. Adverse reactions include vaginal bleeding, headache, stomachache, pregnancy reaction, nausea, bleeding, and depression. Precautions include keeping the product safe for children. It should be stored in a cool, dark, and tightly closed container. Please read the instructions carefully before use. Dosage form includes tablets (21 tablets per pack, available in yellow, white, and brown varieties). "],"Contraindications":["Stroke","Liver Cancer","Endometrial Cancer","Diabetes","Anemia","Sclerosis","Pruritus","Jaundice"],"Subject":["Obstetrics and Gynecology Drugs"]}];

[0162] Here is an example of a prompt for a disease-medication-symptom type question template:

[0163] Prompt: Based on the given disease, corresponding medication, and symptom information, please generate a doctor-patient question and answer regarding the disease, medication, and symptoms. Question types include, but are not limited to: 1. What precautions should a patient with symptoms and a diagnosis of the disease take when using medication?; 2. Generate a question-and-answer pair containing detailed patient background and a clear medical question. The answer should explain the treatment methods for the disease in detail, such as details of drug and non-drug treatments. Please ensure that the answer covers different knowledge points and provides suggestions: [{"class":"100 Common Diseases","Keywords":"Liver Cancer","Drug Treatment":["Regorafenib","Antiviral Drugs Effective Against Hepatitis B or C","Sorafenib"],"High-Incidence Areas":["Sub-Saharan Africa"],"High-Risk Factors":["Obesity","HBV"] "High DNA levels", "Chronic alcoholism", "Male", "Chronic hepatitis B infection", "Family history of liver cancer", "Chronic hepatitis C cirrhosis", "Core promoter mutation", "Cirrhosis", "HCV co-infection", "Senile valvular heart disease", "Hepatitis B e antigen", "Diabetes"], "Site of onset": ["Liver"], "Auxiliary examinations": ["Liver function tests"], "Medical history": ["Long-term chronic hepatitis B history"];}, {"class": "Western medicine", "Central keyword": "bp", "Classification": ["Chocorticoids / Cholelithic agents / Hepatoprotective drugs", "Other immunosuppressants / Other respiratory drugs", "Immune enhancers"], "OTC type": ["Class A OTC", "Class B OTC"], "Specifications": ["50mg per vial", "10mg / bottle"], "Contraindications": ["Contraindicated in patients with known hypersensitivity to this product"], "Drug type": ["Prescription drug"], "Indications": ["Adjuvant treatment for other cancers", "Bronchiectasis"], "Bronchiectasis"], "Onset site": ["Liver"], "Auxiliary examinations": ["Liver function tests"], "Medical history": ["Long-term chronic hepatitis B history"]; "Bronchiitis", "Urticaria", "Tracheitis", "Pancreatic Cancer", "Other Diseases Caused by Immunodeficiency", "Adjuvant Treatment of Various Malignant Tumors", "Soft Tissue Sarcoma", "Psoriasis", "Cancer", "Gastric Cancer", "Hepatitis", "Breast Cancer", "Treatment of Urticaria, etc.", "Liver Cancer", "Measles Infection", "Bronchopulmonary Lung Cancer", "Asthma", "Soft Tissue Type Rhabdomyosarcoma"], "Storage": ["Sealed or tightly sealed, store in a cool (not exceeding 20℃) dry place", "Tightly sealed or sealed, store in a cool, dry place"], "Adverse Reactions": ["Pain", "Dizziness", "Urticaria", "Palpitation and Urticaria", "Elevated Body Temperature", "Redness and Swelling", "Nausea", "Measles Infection", "Palpitation", "Chest Tightness"], "Precautions": ["Discontinue use if the diameter of redness and swelling at the injection site is greater than 10cm", "Use with caution in patients with allergic constitution"], "Shelf Life": ["Tentatively 24 months"], "Drug Regulatory Classification": ["National Essential Medicines"]The injection is a national medical insurance drug, an external medicine, and an anesthetic. Its components include sodium chloride, ribonucleic acid extracted from the spleen of healthy pigs, mannitol, sugar, ribonucleic acid III, ribonucleic acid, and mannitol. It is described as a white or off-white freeze-dried block or powder, easily soluble in water. The product is classified as a short-acting oral contraceptive. It is also classified as a gynecological and obstetric drug. Adverse reactions include vaginal bleeding, headache, stomachache, pregnancy reaction, nausea, bleeding, and depression. Precautions include keeping the product safe for children. It should be stored in a cool, dark, and tightly closed container. Please read the instructions carefully before use. Dosage form includes tablets (21 tablets per pack, available in yellow, white, and brown varieties). Contraindications: ["Stroke", "Liver Cancer", "Endometrial Cancer", "Diabetes", "Anemia", "Sclerosis", "Pruritus", "Jaundice"], Subject: ["Obstetrics and Gynecology Drugs"]}, Keywords: "Fatigue", Examination: ["Non-essential Trace Elements", "Aspartic Acid", "Glutamate Detection", "4-Tibeproline"], Related Diseases: ["Tuberculosis", "Liver Cancer", "Pine Caterpillar Disease Osteoarthritis"], Affected Site: ["Whole Body"], Department: ["Nutrition Department", "Internal Medicine", "Physical Examination and Health Care Department"], Related Symptoms: ["Congestive Heart Failure", "Arm Pain", "Uterine Blood Stasis and Menstrual Pain", "Skin Hardening"]}]

[0164] 346. Use the ChatGPTAPI in OpenAIAPI to input each prompt into ChatGPTAPI and retrieve the corresponding question and answer data from the ChatGPTAPI return value.

[0165] The other steps and parameters are the same as those in any of the specific implementation methods one to seven.

[0166] Specific Implementation Method Nine: This implementation method differs from Specific Implementation Methods One through Eight in that, in steps Three, Four, and Six, ChatGPTAPI from OpenAIAPI is used. Each prompt is input into ChatGPTAPI, and the corresponding question and answer data is obtained from the return value of ChatGPTAPI.

[0167] The specific process is as follows:

[0168] Extract the choices[0].message.content field from the chat_completion object returned by ChatGPTAPI;

[0169] The format of the content (question and answer data) recorded in the choices[0].message.content field is as follows:

[0170] question:

[0171] answer:

[0172] The three are independent;

[0173] For disease-symptoms:

[0174] Question: I've been feeling constantly fatigued lately, even with minimal activity. What medical conditions might this be a symptom of?

[0175] Answer: Fatigue is a common symptom of many diseases and may be related to systemic illnesses. For example, liver cancer is a disease that can cause fatigue, especially in its advanced stages. In addition, tuberculosis and other chronic diseases such as pine caterpillar infestation and osteoarthritis can also present with similar symptoms. It is recommended that you undergo further examination, such as non-essential trace element testing or aspartic acid testing, to determine the cause.

[0176] For diseases - medications:

[0177] Question: What medications can be used to treat liver cancer?

[0178] Answer: Drug treatment for liver cancer typically includes regorafenib and sorafenib, which slow disease progression by inhibiting tumor cell growth and angiogenesis. Doctors will develop a personalized treatment plan based on each patient's specific situation.

[0179] For diseases, medications, and symptoms:

[0180] Question: The patient has a long history of chronic hepatitis B and has been diagnosed with liver cancer based on liver function tests. What are the treatment options for liver cancer?

[0181] Answer: Treatment options for liver cancer include medication, surgery, radiotherapy, and targeted therapy. In medication, regorafenib and sorafenib are commonly used targeted drugs that inhibit tumor angiogenesis to stop its growth. Surgery is suitable for patients with early-stage liver cancer, aiming to control the disease by removing the tumor. Radiotherapy is used to locally control tumor growth. For patients with chronic hepatitis B, the use of antiviral drugs is also crucial to reduce further viral damage to the liver. Each patient's treatment plan needs to be tailored to their specific situation, comprehensively assessing their condition, liver function, and overall health.

[0182] The specific process of using the ChatGPTAPI in the OpenAIAPI is as follows:

[0183] Register an OpenAI account, create a ChatGPTAPI key, load the openai library through a Python client, configure the obtained ChatGPTAPI key in the openai.api_key parameter of the code, and configure the GPT 3.5 version name in the model parameter.

[0184] For example, “gpt-3.5-turbo”; at the same time, fill in the obtained prompt configuration as the content key value of the messages list in the code, and uniformly set the role key value of the messages list to “user”; call the openai.ChatCompletion.create() method to send a request to the OpenAI server, wait for the server to generate text based on the request and return it; the generated response will return a chat_completion object, and the answer content will be extracted from the choices[0].message.content field of the object.

[0185] The other steps and parameters are the same as those in one of the specific implementation methods one to eight.

[0186] Specific Implementation Method 10: This implementation method differs from Specific Implementation Methods 1 to 91 in that the large language base model fine-tuning module is used to fine-tune the large language base model using the generated question-and-answer data to obtain a fine-tuned large language base model.

[0187] The specific process is as follows:

[0188] The training set consists of question-and-answer data generated by the question-and-answer data generation module.

[0189] Supervised learning methods were used to fine-tune the large language base model;

[0190] The optimization method used is the LoRA method;

[0191] Initialize the parameters of the large language base model and load the pre-trained weights;

[0192] Train the large language base model until it converges to obtain a finely tuned large language base model.

[0193] The optimization method adopts the LoRA (Low-Rank Adaptation) method, which uses low-rank matrix factorization to decompose the original fully connected layer weight matrix in the model into the product of two low-rank matrices. Only the low-rank matrix is ​​updated, rather than the weights of the entire model, which reduces the number of parameters to be updated and thus reduces the training cost, while ensuring that the effect is comparable to full fine-tuning.

[0194] Initialize the parameters of the large language base model and load the pre-trained weights (the large language base model is pre-trained and can be used directly) to ensure that the model has basic natural language understanding and generation capabilities when fine-tuning begins.

[0195] The focus is on optimizing the model's performance in medical dialogue scenarios to ensure it can provide accurate answers in tasks such as medical consultation, disease diagnosis, and drug recommendation.

[0196] The task structure is defined as a medical question-and-answer format, and the medical knowledge question-and-answer pairs obtained in step three are used as training data.

[0197] Set the training parameters and LoRA parameters;

[0198] Training data is input into the model in batches. Each batch trains the model through supervised learning, enabling it to adapt from pre-trained general knowledge to specific tasks in the medical field. A loss function is used to quantify the difference between the model-generated answers and the standard answers, and the model parameters are adjusted through the backpropagation algorithm.

[0199] After each round of training, the model's performance is evaluated using a medical validation set. The accuracy and coherence of the medical answers generated by the model are assessed to determine its ability to generalize on unseen data.

[0200] The other steps and parameters are the same as those in any of the specific implementation methods one to nine.

[0201] The evaluation metrics SUS for medical question-answering tasks were constructed; the specific process is as follows:

[0202] The SUS evaluation metrics are divided into three dimensions: Safety, Usability, and Smoothness.

[0203] Safety: The safety dimension focuses on ensuring that the generated responses do not pose potential health risks to users. For example, it checks whether the generated answers contain incorrect or harmful medical advice, such as recommending inappropriate medications or treatments. In the medical field, misinformation can lead to serious health consequences, making the safety of information paramount. This dimension measures whether the medical information provided by the model is risk-free and assesses whether the results will not mislead users.

[0204] Usability: The usability dimension assesses the extent to which the generated responses reflect medical expertise. It examines the accuracy, professionalism, and consistency with clinical practice of the answers. A highly usable response should be accurate, conform to medical standards, and have practical application value for healthcare professionals or patients. This dimension measures the model's depth of understanding of medical concepts and whether the generated answers provide useful advice to users.

[0205] Fluency: The fluency dimension assesses the language quality and coherence of the generated responses. Even if the content is medically accurate, the way it is expressed directly impacts the user experience. Fluency primarily focuses on the naturalness, fluency, and ease of understanding of the language, ensuring that the answers clearly and accurately convey complex medical information. This dimension measures the model's ability as a language model and its level of effectiveness in communicating medical information.

[0206] For each dimension, a three-point rating scale is used:

[0207] 1 point: Indicates unsatisfactory, the answer did not meet expectations;

[0208] 2 points: indicates that the answer is acceptable and meets the minimum requirements;

[0209] 3 points: indicates good, the answer meets the ideal standard.

[0210] The SUS index was used to evaluate the capabilities of the fine-tuned model in the medical field; the specific process was as follows:

[0211] The SUS (Supplemental Medical Services) model was used to evaluate its performance in medical scenarios, including testing through medical question-and-answer tasks. This evaluated the model's overall capabilities across different medical scenarios, with a focus on its ability to handle complex medical problems, its effectiveness in treating symptoms, and the user acceptance and comprehension of its answers.

[0212] Base weights: derived from the pre-trained weights of a general-purpose large language model on massive amounts of data, providing the model with general capabilities across multiple tasks;

[0213] Fine-tuning the data: Using medical question-and-answer instructions constructed in the knowledge graph to fine-tune the data;

[0214] Fine-tuning module: Used to fine-tune the base weights using the LoRA fine-tuning method;

[0215] Fine-tuned weights: Weights that have been fine-tuned on fine-tuned data. These weights are trained using datasets in the medical field to optimize for specific tasks, enabling the model to handle complex tasks in the medical field.

[0216] Input module: Used to receive and process user text input, such as medical questions, symptom descriptions, and disease information, and to send the received text into the large language model for processing;

[0217] Output module: Used to generate and output the model's answers. Based on the fine-tuned large language model weights and the base weights, it generates accurate and detailed medical advice or answers for user input, ensuring that the content is accurate and conforms to medical standards and uses fluent language.

[0218] The beneficial effects of the present invention are verified using the following embodiments:

[0219] Example 1:

[0220] To address the issue of suboptimal performance of general-purpose large language models in medical professional fields, this invention provides a method for fine-tuning a medical large language model, comprising the following steps:

[0221] The Large Language Base Module is used to select a general-domain open-source large language model as the base model. The selected base model is LLaMA-7B. LLaMA is a series of multilingual open-source large language models that perform powerfully in various task processing capabilities. The 7 billion (7B) parameter version is more lightweight, making it easier to train and infer with limited resources, and is suitable for medical tasks.

[0222] The knowledge instance sampling module is used to sample knowledge instances from the knowledge graph. Taking the medical knowledge graph CMeKG as an example, instances with the "Type" field being "Disease", "Drug" and "Symptom" are randomly sampled.

[0223] The question-and-answer data generation module is used to generate instruction data based on knowledge instances using a large language model;

[0224] 31. Knowledge sampled from the CMeKG knowledge graph is categorized according to the Type field;

[0225] 32. Develop question templates based on different types of knowledge. For example, for knowledge examples related to diseases, the question templates would be: Question Template 1: What disease is this?; Question Template 2: What are the symptoms of this disease?; Question Template 3: How is this disease treated?

[0226] 33. Write a prompt based on the question template and generate instruction data using the OpenAI API; the specific process is as follows:

[0227] 331. Merge the "Knowledge" field in the corresponding knowledge instance with the question template to construct a prompt. For example, a knowledge instance with "Type" as "Disease" and "Knowledge" as "{"class":"100 common diseases","keyword":"liver cancer","drug treatment":["regorafenib","antiviral drugs effective against hepatitis B or C","sorafenib"],"high-incidence areas":["sub-Saharan Africa"],"high-risk factors":["obesity","HBV"] The following keywords appear to be search terms or tags: "High DNA levels", "Chronic alcoholism", "Male", "Chronic hepatitis B infection", "Family history of liver cancer", "Chronic hepatitis C cirrhosis", "Core promoter mutation", "Cirrhosis", "HCV co-infection", "Senile valvular heart disease", "Hepatitis B e antigen", "Diabetes"; "Site of onset": ["Liver"]; "Auxiliary examinations": ["Liver function test"]; "Medical history": ["Long-term chronic hepatitis B history"]; Therefore, the knowledge instance corresponds to a disease type and should be combined with the disease type question templates: "Question template 1: What is this disease?; Question template 2: What are the symptoms of the disease?; Question template 3: How to treat the disease?".

[0228] The combined prompt is: "Based on the given information, please generate questions and answers between doctors and patients regarding this disease. Question types include, but are not limited to: 1. What is this disease? 2. What are the symptoms of the disease? 3. How to treat the disease?:"class":"100 common diseases","keyword":"liver cancer","drug treatment":["regorafenib","antiviral drugs effective against hepatitis B or C","sorafenib"]","prevalence areas":["sub-Saharan Africa"]","risk factors":["obesity","high HBV DNA","chronic alcoholism","male","chronic hepatitis B infection","family history of liver cancer","chronic hepatitis C cirrhosis","core promoter mutation","cirrhosis","HCV co-infection","senile valvular heart disease","hepatitis B e antigen","diabetes"]","site of onset":["liver"]","auxiliary examinations":["liver function tests"]","medical history":["long-term chronic hepatitis B history"];"

[0229] 332. Using the ChatGPTAPI in OpenAI, input the prompt into the API and retrieve the corresponding question-and-answer pair from the API's return value. Example: "question: What is liver cancer?; answer: Liver cancer is a malignant tumor that originates in the liver. It is usually caused by factors such as chronic hepatitis or cirrhosis. Early symptoms are not obvious, but it may cause symptoms such as pain in the liver area, indigestion, and fatigue."

[0230] 3.4. Integrate single-type knowledge instances to obtain cascaded-type instances, write a prompt based on the cascaded types, and generate cascaded-type question-and-answer data using the OpenAI API; the specific process is as follows:

[0231] 341. Combine the related knowledge instances sampled from the medical knowledge graph CMeKG, and the combined types are disease-symptom, disease-drug or disease-drug-symptom.

[0232] For example, in the CMeKG knowledge graph, if the "Type" field is "Disease", which is a disease type knowledge instance, the value recorded by the "Center Key" key in its dictionary is the disease name. In the drug type knowledge instance where the "Type" field is "Drug", the content type recorded by the "Indications", "Contraindications", etc. keys in the dictionary is consistent. Then, the disease and drug type knowledge instances can be cascaded into disease-drug type knowledge.

[0233] 342. Taking the disease-drug cascade as an example,

[0234] 11) First, extract the content of the "Knowledge" field of a knowledge instance entity1 in the disease, and obtain the "knowledge" field; for example: "{"class":"100 common diseases","keyword":"liver cancer","drug treatment":["regorafenib","antiviral drugs effective against hepatitis B or C","sorafenib"],"high-incidence areas":["sub-Saharan Africa"],"high-risk factors":["obesity","high HBV DNA","chronic alcoholism","male","chronic hepatitis B infection","family history of liver cancer","chronic hepatitis C cirrhosis","core promoter mutation","cirrhosis","HCV co-infection","senile valvular heart disease","hepatitis B e antigen","diabetes"],"site of onset":["liver"],"auxiliary examination":["liver function test"],"medical history":["long-term chronic hepatitis B history"];}";

[0235] The content is stored in the form of a dictionary. If the value value1 is obtained from the "headword" key of entity1, then value1 = liver cancer.

[0236] 12) Match each key value in the drug's "knowledge" field with value1;

[0237] An example of drug knowledge is: "{"class":"Western Medicine","Central Keyword":"bp","Classification":["Chlorogenic Agents / Cholelitholytic Agents / Hepatoprotective Drugs","Other Immunosuppressants / Other Respiratory System Drugs","Immune Enhancers"],"OTC Type":["Class A OTC","Class B OTC"],"Specification":["50mg per vial","10mg / bottle"],"Contraindications":["Contraindicated in patients with known hypersensitivity to this product"],"Drug Type":["Prescription Drug"],"Indications":["Adjuvant treatment for other cancers","Bronchiitis","Urticaria","Tracheitis","Pancreatic Cancer","Other diseases caused by immunodeficiency","Adjuvant treatment for various malignant tumors","Soft tissue sarcoma","Psoriasis","Cancer","Gastric Cancer","Hepatitis","Breast Cancer","Treatment for Urticaria, etc.","Liver Cancer","Measles Infection","Bronchopulmonary Lung Cancer","Asthma","Soft tissue type striated muscle"] [Storage]: ["Sealed or tightly sealed, store in a cool (not exceeding 20℃) dry place", "Sealed or tightly sealed, store in a cool, dry place"], ["Pain", "Dizziness", "Urticaria", "Palpitation and urticaria", "Fever", "Redness and swelling", "Nausea", "Measles infection", "Palpitation", "Chest tightness"], ["Precautions": ["Discontinue use if the diameter of redness and swelling at the injection site is greater than 10cm", "Use with caution in patients with allergic constitution"], ["Shelf life": ["Tentatively 24 months"], ["Drug regulatory classification": ["National essential medicine; injection is a national medical insurance drug", "External medicine", "Anesthetic drug"], ["Sodium chloride", "Ribonucleic acid extracted from the spleen of healthy pigs", "Name is mannitol", "Sugar", "Ribonucleic acid III", "Ribonucleic acid", "Mannitol"], ["White or off-white freeze-dried lumps or powder, easily soluble in water"]

[0238] In this example, the value corresponding to the "indication" key is a list, and one element of the list is "liver cancer", indicating a successful match. Let the knowledge instance be entity2, and concatenate the key-value pairs in the "knowledge" field corresponding to entity2 with the content of entity1 to construct a new disease-drug type knowledge instance.

[0239] 13) Since not all the contents of the drug list have been traversed, step 12) needs to be repeated until the list traversal is complete, ensuring that all knowledge related to the core term of entity1 is concatenated. A total of two relevant knowledge entries were found. An example of the concatenated disease-drug type knowledge is as follows:

[0240] [{"class":"100 Common Diseases","Centerword":"Liver Cancer",……},{"class":"Western Medicine","Centerword":"B-toxin",……,"Indications":["Adjuvant Treatment for Other Cancers","Bronchiitis","Urticaria","Tracheitis","Pancreatic Cancer","Various Diseases Caused by Other Immunodeficiency","Adjuvant Treatment for Various Malignant Tumors","Soft Tissue Sarcoma","Psoriasis","Cancer","Gastric Cancer","Hepatitis","Breast Cancer","Treatment for Urticaria, etc.","Liver Cancer","Measles Infection","Bronchopulmonary Lung Cancer","Asthma","Rhabdomyosarcoma of Soft Tissue Type"],……},{"class":"Western Medicine","Centerword":"Triphasic Contraceptive Pills",……,"Contraindications":["Stroke","Liver Cancer","Endometrial Cancer","Diabetes","Anemia","Sclerosis","Pruritus","Jaundice"],……}];

[0241] 14) If the query fails, for example when value1 = "hypertension" but there is no matching value in the drug knowledge list, use the medical thesaurus SNOMED CT to perform a thesaurus search.

[0242] The method for querying using SNOMED CT is as follows: connect to the SNOMED CT REST API service through the requests library, configure the query parameter "term" as "hypertension", and use the requests.get() method to initiate an HTTP GET request. If the returned status code is 200, the request is successful. Use response.json() to parse the returned JSON data, and extract the term value by traversing the results['items'] list. This term is the standard synonym found in the query. In the example, the first term value is "primary hypertension".

[0243] Assign the obtained synonym "primary hypertension" to value1, and repeat step 12) to perform the query.

[0244] 343. Targeting the combination of disease, medication, and symptoms.

[0245] 1) Classify the disease-drug cascade as Type 1 and symptoms as Type 2:

[0246] 2) First, extract the content of the "knowledge" field from a knowledge instance entity1 in the drug information; an example is: "[{"class":"100 common diseases","keyword":"liver cancer",……},{"class":"Western medicine","keyword":"bp",……,"indications":["adjuvant treatment for other cancers","bronchitis","urticaria","tracheitis","pancreatic cancer","various diseases caused by other immune deficiency","adjuvant treatment for various malignant tumors"]. , "soft tissue sarcoma", "psoriasis", "cancer", "gastric cancer", "hepatitis", "breast cancer", "treatment of urticaria, etc.", "liver cancer", "measles infection", "bronchogenic carcinoma", "asthma", "soft tissue type rhabdomyosarcoma"],……},{"class":"Western medicine","keyword":"triphasic contraceptive",……,"contraindications":["stroke","liver cancer","endometrial cancer","diabetes","anemia","sclerosis","pruritus","jaundice"],……}]”;

[0247] The content is stored in the form of a dictionary list. If the value value1 is obtained from the "center word" key of entity1, then value1 = liver cancer.

[0248] 3) Match each key-value pair in the "knowledge" field of the symptoms with value1;

[0249] One example of knowledge in the symptoms is: "{"Central keywords":"fatigue","Examination":["Non-essential trace elements","Aspartic acid","Glutamate detection","4-Tibeproline"],"Related diseases":["Tuberculosis","Liver cancer","Pine caterpillar disease and osteoarthritis"],"Affected site":["Whole body"],"Department":["Nutrition","Internal medicine","Physical examination and health care department"],"Related symptoms":["Congestive heart failure","Arm pain","Uterine blood stasis and menstrual cramps","Hardened skin"]}",

[0250] In this example, the value corresponding to the "related diseases" key is a list, and the second element of the list is "liver cancer", indicating that the match is successful. Let the knowledge instance be entity2, and concatenate the key-value pairs in the "knowledge" field corresponding to entity2 with the content of entity1 to construct a new disease-drug-symptom type knowledge instance.

[0251] 4) Since not all the contents of the drug list have been traversed, step 3) needs to be repeated until the list traversal is complete. Ensure that all knowledge related to the central keyword of entity1 is concatenated. The final result should contain one relevant piece of knowledge. An example of the concatenated disease-drug-symptom type knowledge is as follows:

[0252] [{"class":"100 Common Diseases","Keyword":"Liver Cancer",……},{"class":"Western Medicine","Keyword":"Bpi-12",……,"Indications":["Adjuvant Treatment for Other Cancers","Bronchi","Urticaria","Tracheitis","Pancreatic Cancer","Various Diseases Caused by Immune Deficiency","Adjuvant Treatment for Various Malignant Tumors","Soft Tissue Sarcoma","Psoriasis","Cancer","Gas Cancer","Hepatitis","Breast Cancer","Treatment for Urticaria, etc."] "Liver cancer", "Measles infection", "Bronchopulmonary carcinoma", "Asthma", "Soft tissue type rhabdomyosarcoma"],……},{"class":"Western medicine","Central keyword":"Triphasic contraceptive",……,"Contraindications":["Stroke","Liver cancer","Endometrial cancer","Diabetes","Anemia","Sclerosis","Pruritus","Jaundice"],……},{"Central keyword":"Fatigue",……,"Related diseases":["Tuberculosis","Liver cancer","Pine caterpillar disease osteoarthritis"],……}];

[0253] 344. Create question templates for disease-symptom, disease-medication, or disease-medication-symptom relationships. An example of a disease-symptom-medication question template is: "Generate a question-and-answer pair that includes detailed patient background and a clear medical question. It should explain the treatment methods for the disease in detail, such as drug and non-drug treatments. Ensure the answer covers different knowledge points and provides recommendations."

[0254] 3.4.5. Write a prompt based on each question template. An example of a prompt for a question template related to disease-medication-symptom type is as follows:

[0255] Prompt: Based on the given disease, corresponding medication, and symptom information, please generate a doctor-patient question and answer regarding the disease, medication, and symptoms. Question types include, but are not limited to: 1. What precautions should a patient with symptoms and a confirmed diagnosis take when using medication? 2. Describe the symptoms, causes, and treatment options for the disease: [{"class":"100 Common Diseases","Keywords":"Liver Cancer","Drug Treatment":["Regorafenib","Antiviral Drugs Effective Against Hepatitis B or C","Sorafenib"],"High-Incidence Areas":["Sub-Saharan Africa"],"High-Risk Factors":["Obesity","HBV"] "High DNA levels", "Chronic alcoholism", "Male", "Chronic hepatitis B infection", "Family history of liver cancer", "Chronic hepatitis C cirrhosis", "Core promoter mutation", "Cirrhosis", "HCV co-infection", "Senile valvular heart disease", "Hepatitis B e antigen", "Diabetes"], "Site of onset": ["Liver"], "Auxiliary examinations": ["Liver function tests"], "Medical history": ["Long-term chronic hepatitis B history"];}, {"class": "Western medicine", "Central keyword": "bp", "Classification": ["Chocorticoids / Cholelithic agents / Hepatoprotective drugs", "Other immunosuppressants / Other respiratory drugs", "Immune enhancers"], "OTC type": ["Class A OTC", "Class B OTC"], "Specifications": ["50mg per vial", "10mg / bottle"], "Contraindications": ["Contraindicated in patients with known hypersensitivity to this product"], "Drug type": ["Prescription drug"], "Indications": ["Adjuvant treatment for other cancers", "Bronchiectasis"], "Bronchiectasis"], "Onset site": ["Liver"], "Auxiliary examinations": ["Liver function tests"], "Medical history": ["Long-term chronic hepatitis B history"]; "Bronchiitis", "Urticaria", "Tracheitis", "Pancreatic Cancer", "Other Diseases Caused by Immunodeficiency", "Adjuvant Treatment of Various Malignant Tumors", "Soft Tissue Sarcoma", "Psoriasis", "Cancer", "Gastric Cancer", "Hepatitis", "Breast Cancer", "Treatment of Urticaria, etc.", "Liver Cancer", "Measles Infection", "Bronchopulmonary Lung Cancer", "Asthma", "Soft Tissue Type Rhabdomyosarcoma"], "Storage": ["Sealed or tightly sealed, store in a cool (not exceeding 20℃) dry place", "Tightly sealed or sealed, store in a cool, dry place"], "Adverse Reactions": ["Pain", "Dizziness", "Urticaria", "Palpitation and Urticaria", "Elevated Body Temperature", "Redness and Swelling", "Nausea", "Measles Infection", "Palpitation", "Chest Tightness"], "Precautions": ["Discontinue use if the diameter of redness and swelling at the injection site is greater than 10cm", "Use with caution in patients with allergic constitution"], "Shelf Life": ["Tentatively 24 months"], "Drug Regulatory Classification": ["National Essential Medicines"]The injection is a national medical insurance drug, an external medicine, and an anesthetic. Its components include sodium chloride, ribonucleic acid extracted from the spleen of healthy pigs, mannitol, sugar, ribonucleic acid III, ribonucleic acid, and mannitol. It is described as a white or off-white freeze-dried block or powder, easily soluble in water. The product is classified as a short-acting oral contraceptive. It is also classified as a gynecological and obstetric drug. Adverse reactions include vaginal bleeding, headache, stomachache, pregnancy reaction, nausea, bleeding, and depression. Precautions include keeping the product safe for children. It should be stored in a cool, dark, and tightly closed container. Please read the instructions carefully before use. Dosage form includes tablets (21 tablets per pack, available in yellow, white, and brown varieties). Contraindications: ["Stroke", "Liver Cancer", "Endometrial Cancer", "Diabetes", "Anemia", "Sclerosis", "Pruritus", "Jaundice"], Subject: ["Obstetrics and Gynecology Drugs"]}, Keywords: "Fatigue", Examination: ["Non-essential Trace Elements", "Aspartic Acid", "Glutamate Detection", "4-Tibeproline"], Related Diseases: ["Tuberculosis", "Liver Cancer", "Pine Caterpillar Disease Osteoarthritis"], Affected Site: ["Whole Body"], Department: ["Nutrition Department", "Internal Medicine", "Physical Examination and Health Care Department"], Related Symptoms: ["Congestive Heart Failure", "Arm Pain", "Uterine Blood Stasis and Menstrual Pain", "Skin Hardening"]}]

[0256] 346. Using the ChatGPTAPI in OpenAI, input the prompt into the API and retrieve the corresponding question-and-answer pair from the API's return value. An example of a question-and-answer pair is: "Question: A patient with a long history of chronic hepatitis B and liver function tests has been diagnosed with liver cancer. What are the treatment options for liver cancer? Answer: Treatment options for liver cancer include drug therapy, surgery, radiotherapy, and targeted therapy. In drug therapy, regorafenib and sorafenib are commonly used targeted drugs that inhibit tumor angiogenesis to stop its growth. Surgery is suitable for early-stage liver cancer patients, aiming to control the disease by removing the tumor. Radiotherapy is used to locally control tumor growth. For patients with chronic hepatitis B, the use of antiviral drugs is also crucial to reduce further viral damage to the liver. Each patient's treatment plan needs to be tailored to their specific situation, comprehensively assessing the disease, liver function, and the patient's overall health."

[0257] The large language base model fine-tuning module is used to fine-tune the large language base model using the generated question-and-answer data to obtain a fine-tuned large language base model;

[0258] The specific process is as follows:

[0259] Download pre-trained weights from LLaMA-7B to ensure that the model has basic natural language understanding and generation capabilities when fine-tuning begins;

[0260] Define the task structure as a medical question-and-answer format, and use the medical knowledge question-and-answer pairs obtained in step three as the training set;

[0261] Define the model training parameters as epoch=10, batch_size=16, and LoRA parameter as r=8;

[0262] Training data is input into the model in batches. Each batch trains the model through supervised learning, enabling it to adapt from pre-trained general knowledge to specific tasks in the medical field. The cross-entropy loss function is used to quantify the difference between the model-generated answers and the standard answers. The model parameters are adjusted through the backpropagation algorithm.

[0263] The training set consists of question-and-answer data generated by the question-and-answer data generation module.

[0264] Supervised learning methods were used to fine-tune the large language base model;

[0265] The optimization method used is the LoRA method;

[0266] Initialize the parameters of the large language base model and load the pre-trained weights;

[0267] Train the large language base model until it converges to obtain a finely tuned large language base model.

[0268] The response generation module is used to answer medical knowledge questions based on a finely tuned large language base model.

[0269] Constructing an evaluation metric, SUS, for medical question-answering tasks;

[0270] The SUS index was used to evaluate the capabilities of the fine-tuned model in the medical field. Five evaluators with medical backgrounds were recruited to conduct a manual evaluation and used a three-point scale to evaluate the SUS index performance of LLaMA, Alpaca, ChatGLM and the fine-tuned herbal model in the medical field. The average scores are shown in Table 1.

[0271] Table 1. Experimental Results of SUS Scoring Model

[0272]

[0273]

[0274] As shown in Table 1, the experimental results indicate that, in terms of usability, the finely tuned herbal model scored 2.12, higher than LLaMA's 1.21, Alpaca's 2.05, and ChatGLM's 1.93; in terms of fluency, the finely tuned herbal model scored 2.47, higher than LLaMA's 1.58, Alpaca's 2.30, and ChatGLM's 2.41; and in terms of safety, the herbal model scored 2.88, only slightly lower than LLaMA's 2.93, but higher than Alpaca's 2.64 and ChatGLM's 2.59. Usability in medical tasks was significantly improved, and the fluency of responses was also increased, while the impact on safety was low.

[0275] This invention may have other embodiments. Without departing from the spirit and essence of this invention, those skilled in the art can make various corresponding changes and modifications according to this invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

Claims

1. A medical question-answering system based on a large language model, characterized in that: The system includes: a large language base module, a knowledge instance sampling module, a question-and-answer data generation module, a large language base model fine-tuning module, and a response generation module; The large language base module is used to select a general-domain open-source large language model as the base model; The knowledge instance sampling module is used to sample knowledge instances from the knowledge graph; The question-and-answer data generation module is used to generate question-and-answer data based on knowledge instances using a large language foundation model. The large language base model fine-tuning module is used to fine-tune the large language base model using the generated question-and-answer data to obtain a fine-tuned large language base model. The response generation module is used to answer the medical knowledge questions to be processed based on the finely tuned large language base model. The question-and-answer data generation module is used to generate question-and-answer data based on knowledge instances using a large language foundation model; the specific process is as follows: Randomly sample knowledge instances from the medical knowledge graph CMeKG; Each knowledge instance is a set of the following: a disease and its corresponding knowledge, a drug and its corresponding knowledge, and a symptom and its corresponding knowledge. The format of each knowledge instance is as follows: Type: Knowledge: The Type field records the type of knowledge instance; The Knowledge field records the content of knowledge instances; The knowledge instance sampling module is used to sample knowledge instances from the knowledge graph; the specific process is as follows:

31. Classify the knowledge instances randomly sampled from the Medical Knowledge Graph (CMeKG) to obtain the types of knowledge instances; the specific process is as follows: The type of the randomly sampled knowledge instance is determined based on the Type field in the randomly sampled knowledge instance; 32. Develop question templates for different types of knowledge instances; 33. Write a prompt based on the question template and generate single-type question-and-answer data using the OpenAI API; Third and fourth, integrate single-type knowledge instances to obtain cascaded type instances, write a prompt based on the cascaded types, and generate cascaded type question-and-answer data through OpenAIAPI; The process described in section 3.2 involves developing question templates for different types of knowledge instances; the specific process is as follows: A question template was created for the "Type" field being "Disease". A question template was created for the "Type" field being "Drug". A question template was created for the "Type" category, which is "Symptoms". In step 3, a prompt is written based on a question template, and single-type question-and-answer data is generated using the OpenAI API; the specific process is as follows:

331. Write a prompt based on the template for each question; 332. Using the ChatGPTAPI in the OpenAI API, input each prompt into the ChatGPTAPI and retrieve the corresponding question-and-answer data from the ChatGPTAPI return value; the specific process is as follows: Extract the choices[0].message.content field from the chat_completion object returned by ChatGPTAPI; The format of the content recorded in the choices[0].message.content field is as follows: question: answer: The specific process of using the ChatGPTAPI from the OpenAIAPI in the 332 is as follows: Register an OpenAI account, create a ChatGPTAPI key, load the openai library through a Python client, configure the obtained ChatGPTAPI key in the openai.api_key parameter of the code, and configure the GPT 3.5 version name in the model parameter; The third and fourth steps integrate single-type knowledge instances to obtain cascaded type instances. A prompt is then written based on the cascaded types, and cascaded type question-and-answer data is generated using the OpenAI API. The specific process is as follows:

341. Combine related knowledge instances sampled from the medical knowledge graph CMeKG, and the combined types are disease-drug, disease-symptom, or disease-drug-symptom. 342, 1) Disease-drug combination; the specific process is as follows: 11) When the type is disease-drug combination, extract the content of the "knowledge" field of the disease; Retrieve the value value1 of the key "center word" in the "knowledge" field; The contents of the "knowledge" field for the disease are stored in dictionary form; 12) Match each key value in the "knowledge" field of the drug with value1; If a match is found, the key value in the "knowledge" field corresponding to the drug will be added to the content of the "knowledge" field of the disease to obtain a new disease-drug combination. If the match fails, no action is taken; 13) Repeat step 12) until all key-value pairs in the "knowledge" field of the drug have been traversed, and check if a match exists. If a new disease-drug combination can be obtained; If it does not exist, proceed to step 14); 14) Obtain a list of synonyms for value1 based on the thesaurus, assign the synonyms in the thesaurus to value1, repeat 12) until a match is successful, obtain a new disease-drug combination, and end; if no match is successful, discard the disease-drug combination. 2) Targeting disease-symptom combinations; the specific process is as follows: 21) When the type is a combination of disease and symptoms, extract the content of the "knowledge" field of the disease; Retrieve the value value1 of the key "center word" in the "knowledge" field; The contents of the "knowledge" field for the disease are stored in dictionary form; 22) Match each key value in the "knowledge" field of the symptoms with value1; If a match is found, the key value in the "knowledge" field corresponding to the symptom will be added to the content of the "knowledge" field of the disease to obtain a new disease-symptom combination; If the match fails, no action is taken; 23) Repeat step 22) until all key-value pairs in the "knowledge" field of the symptoms have been traversed, and check if a match exists. If a new combination of disease and symptoms is obtained; If it does not exist, proceed to step 24); 24) Obtain a list of synonyms for value1 based on the thesaurus, assign the synonyms in the thesaurus to value1, repeat 22) until a match is successful, obtain a new disease-symptom combination, and end; if no match is successful, discard the disease-symptom combination; 343. Targeting the combination of disease, medication, and symptoms; the specific process is as follows: 1) The new disease-drug combination obtained in 14) is classified as type one, and the symptoms are classified as type two; 2) Extract the content of the "knowledge" field of type one; the content of the "knowledge" field of type one is stored in the form of a dictionary list; select the value1 of the key "center word" in the first element of the list; The element is a dictionary; 3) Match each key value in the "knowledge" field of type two with value1; If a match is successful, the key value in the "knowledge" field corresponding to type 2 will be added to the content of the "knowledge" field of type 1 to obtain a new type 1-type 2 combination; If the match fails, no action is taken; 4) Repeat step 3) until all key values ​​in the "knowledge" field of type two have been traversed, and check if a match exists. If a new type 1-type 2 combination exists; If it does not exist, proceed to step 5); 5) Obtain a list of synonyms for value1 based on the thesaurus, assign the synonyms in the thesaurus to value1, repeat 3) until a match is successful, obtain a new type 1-type 2 combination, and end; if no match is successful, discard the type 1-type 2 combination.

344. Create question templates for disease-symptom, disease-medication, or disease-medication-symptom; 3.4.

5. Write a prompt based on the template for each question; 346. Use the ChatGPTAPI in the OpenAI API to input each prompt into ChatGPTAPI and retrieve the corresponding question and answer data from the ChatGPTAPI return value. The third, fourth, and sixth statements use the ChatGPT API from the OpenAI API to input each prompt into the ChatGPT API and obtain the corresponding question and answer data from the ChatGPT API return value. The specific process is as follows: Extract the choices[0].message.content field from the chat_completion object returned by ChatGPTAPI; The format of the content recorded in the choices[0].message.content field is as follows: question: answer: The specific process of using the ChatGPTAPI in the OpenAIAPI is as follows: Register an OpenAI account, create a ChatGPTAPI key, load the openai library through a Python client, configure the obtained ChatGPTAPI key in the openai.api_key parameter of the code, and configure the GPT 3.5 version name in the model parameter.

2. The medical question-answering system based on a large language model according to claim 1, characterized in that: The large language base module is used to select a general-domain open-source large language model as the base model; the specific process is as follows: The large language foundation model is an open-source large language model, which can be LLaMA, BLOOM, or ChatGLM.

3. A medical question-answering system based on a large language model according to claim 2, characterized in that: The large language base model fine-tuning module is used to fine-tune the large language base model using the generated question-and-answer data to obtain a fine-tuned large language base model. The specific process is as follows: The training set consists of question-and-answer data generated by the question-and-answer data generation module. Supervised learning methods were used to fine-tune the large language base model; The optimization method used is the LoRA method; Initialize the parameters of the large language base model and load the pre-trained weights; Train the large language base model until it converges to obtain a finely tuned large language base model.

Images