Medical dialogue service device and system based on large language model

Abstract

The invention belongs to the technical field of computers, and particularly discloses a medical dialogue service device and system based on a large language model, and the device comprises a data set construction module, a model training module and a model application module. The data set construction module comprises a dialogue sample reconstruction unit, a knowledge graph question and answer pair generation unit and a dialogue sample rewriting unit; the model training module is used for training a large language model based on the reconstructed dialogue sample, the knowledge graph question and answer pair and the rewritten dialogue sample to obtain a trained large language model; and the model application module is used for inputting the question content carried by the dialogue request to the trained large language model to obtain the answer content. According to the method and the device, high-quality medical dialogue service can be provided for the user in an end-to-end dialogue mode aiming at an actual medical consultation scene.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and more specifically, to a medical dialogue service device and system based on a large language model. Background Art

[0002] With the development of artificial intelligence technology, large language models (LLMs) have made significant progress in the field of natural language processing. These models can understand and generate natural language, which makes them show great potential in medical dialogue services. Medical dialogue services aim to provide patients with health consultation, symptom analysis, medication guidance and other services in an intelligent way, improving the efficiency and accessibility of medical services.

[0003] Previous research on models has mainly focused on specific tasks or diseases with limited applicability, and there is a gap between experimental progress and practical applications. For actual medical consultation scenarios, how to implement medical dialogue services based on large language models is a technical problem that needs to be solved in this field. Summary of the invention

[0004] In view of the shortcomings of the existing technology, the purpose of this application is to implement medical dialogue services based on large language models for actual medical consultation scenarios.

[0005] To achieve the above-mentioned purpose, in a first aspect, the present application provides a medical dialogue service device based on a large language model, comprising: a data set construction module, a model training module and a model application module;

[0006] The dataset construction module includes a dialogue sample reconstruction unit, a knowledge graph question-answer pair generation unit, and a dialogue sample rewriting unit;

[0007] A dialogue sample reconstruction unit is used to select dialogue samples from the medical dialogue data set, reconstruct the selected dialogue samples using a large language model, and obtain reconstructed dialogue samples. The answer content in the reconstructed dialogue samples has a unified format;

[0008] The knowledge graph question-answer pair generation unit is used to sample disease knowledge samples from the medical knowledge graph, process the sampled disease knowledge samples through a large language model, and generate knowledge graph question-answer pairs corresponding to the disease knowledge samples;

[0009] A dialogue sample rewriting unit is used to screen dialogue samples that meet the preset quality indicators from the medical dialogue data set based on the preset quality indicators, rewrite the dialogue samples that meet the preset quality indicators through a large language model, and obtain the rewritten dialogue samples, and the rewritten dialogue samples meet the dialogue behavior preference indicators;

[0010] The model training module is used to train the large language model based on the reconstructed dialogue samples, the knowledge graph question-answer pairs, and the rewritten dialogue samples to obtain the trained large language model;

[0011] The model application module is used to receive a dialogue request carrying question content, input the question content carried by the dialogue request into the trained large language model, and obtain the answer content output by the trained large language model.

[0012] In a possible implementation, the above-mentioned dialogue sample reconstruction unit is specifically used to:

[0013] Perform annotation preprocessing on the screened dialogue samples to obtain preprocessed dialogue samples;

[0014] Based on the preprocessed dialogue sample and the pre-configured first prompt word (Prompts), determine the input content (for example, combine the prompt word and the preprocessed dialogue sample to construct the input content) and input the input content into a large language model (for example, a GPT-3.5 model) to obtain a reconstructed dialogue sample output by the large language model;

[0015] The first prompt word is used to instruct the large language model to reconstruct the preprocessed dialogue sample according to the specified format.

[0016] In a possible implementation, the knowledge graph question-answer pair generation unit is specifically used to:

[0017] Based on the sampled disease knowledge samples and the pre-configured second prompt words, determine the input content (for example, combine the prompt words and the disease knowledge samples to construct the input content) and input the input content into a large language model (for example, a GPT-3.5 model) to obtain the knowledge graph question-answer pair output by the large language model;

[0018] Among them, the second prompt word is used to instruct the large language model to generate a knowledge graph question-answer pair based on the sampled disease knowledge samples.

[0019] In a possible implementation, the above-mentioned dialogue sample rewriting unit is specifically used to:

[0020] Based on the dialogue sample that meets the preset quality index and the pre-configured third prompt word, determine the input content (for example, combine the prompt word and the dialogue sample that meets the preset quality index to construct the input content) and input the input content into a large language model (for example, a GPT-4 model), and obtain the rewritten dialogue sample output by the large language model;

[0021] The third prompt word is used to instruct the large language model to generate a rewritten dialogue sample based on the rewritten dialogue sample.

[0022] In a possible implementation, the above-mentioned dialogue sample rewriting unit is specifically used to:

[0023] Displaying preset quality indicators and conversation samples in the medical conversation dataset, and receiving a target input, where the target input is used to indicate a conversation sample that meets the preset quality indicators;

[0024] In response to the target input, a dialog sample meeting a preset quality indicator is determined.

[0025] In a possible implementation, the above model training module is specifically used to:

[0026] Based on the reconstructed dialogue samples and knowledge graph question-answer pairs, a large language model (such as Baichuan-13B) is trained in the first stage to obtain a large language model after the first stage of training;

[0027] Based on the rewritten dialogue samples, the large language model trained in the first phase is trained in the second phase to obtain the large language model trained in the second phase.

[0028] In a possible implementation, it also includes: a sample adjustment module, used to adjust one or more of the following samples: reconstructed dialogue samples, knowledge graph question-answer pairs, and rewritten dialogue samples.

[0029] In a possible implementation, it also includes: a historical dialogue storage module, which is used to store the dialogue content of the trained large language model in the process of providing medical dialogue services.

[0030] In a possible implementation, the historical conversation storage module is further used to adjust the stored conversation content. The adjusted conversation content can be used to further train the large language model to improve the conversation service effect of the large language model.

[0031] In a second aspect, the present application provides a medical dialogue system, comprising: a user terminal and a medical dialogue service device based on a large language model as described in the first aspect or any possible implementation of the first aspect, the user terminal and the medical dialogue service device being communicatively connected;

[0032] The user terminal is used to send a dialogue request carrying question content to the medical dialogue service device, and receive a dialogue request feedback message carrying answer content sent by the medical dialogue service device.

[0033] In general, the above technical solutions conceived by this application have the following beneficial effects compared with the prior art:

[0034] (1) By training on medical knowledge graphs and real doctor-patient conversations, a large amount of medical expertise has been internalized, ensuring the professionalism of the answers. The model can provide more accurate and reliable medical information and advice.

[0035] (2) It is capable of conducting effective multi-round interactions, gradually deepening the conversation process, handling a wide range of scenarios from simple consultations to complex medical situations, providing end-to-end medical dialogue services, better understanding the patient's condition and needs, and providing more targeted medical consultations.

[0036] (3) Through data reconstruction and behavioral preference training, the model’s responses are made more consistent with human communication habits and preferences. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a structural diagram of a medical dialogue service device based on a large language model provided in an embodiment of the present application;

[0038] Figure 2 It is a flow chart of the technical principle provided by the embodiment of the present application;

[0039] Figure 3 is a flowchart for constructing a data set provided in an embodiment of the present application;

[0040] Figure 4 It is a training flow chart of the model provided in the embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] The terms "first" and "second" in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of the objects. For example, the first prompt word and the second prompt word are used to distinguish different prompt words rather than to describe a specific order of the prompt words.

[0043] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0044] In the description of the embodiments of the present application, unless otherwise specified, "multiple" means two or more than two. For example, multiple processing units refer to two or more processing units, etc.; multiple elements refer to two or more elements, etc.

[0045] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application.

[0046] Figure 1 is a schematic diagram of the structure of a medical dialogue service device based on a large language model provided in an embodiment of the present application, such as Figure 1 As shown, the device includes: a data set construction module 10, a model training module 20 and a model application module 30. Among them:

[0047] The data set construction module 10 includes a dialogue sample reconstruction unit, a knowledge graph question-answer pair generation unit, and a dialogue sample rewriting unit;

[0048] A dialogue sample reconstruction unit is used to select dialogue samples from the medical dialogue data set, and reconstruct the selected dialogue samples using a large language model (such as the GPT-3.5 model) to obtain reconstructed dialogue samples. The answer content in the reconstructed dialogue samples has a unified format;

[0049] The knowledge graph question-answer pair generation unit is used to sample disease knowledge samples from the medical knowledge graph, process the sampled disease knowledge samples through a large language model (such as the GPT-3.5 model), and generate knowledge graph question-answer pairs corresponding to the disease knowledge samples;

[0050] A dialogue sample rewriting unit is used to screen dialogue samples that meet the preset quality indicators from the medical dialogue data set based on the preset quality indicators, rewrite the dialogue samples that meet the preset quality indicators through a large language model (such as a GPT-4 model), and obtain the rewritten dialogue samples, and the rewritten dialogue samples meet the dialogue behavior preference indicators;

[0051] A model training module 20 is used to train a large language model (such as Baichuan-13B) based on the reconstructed dialogue samples, the knowledge graph question-answer pairs, and the rewritten dialogue samples to obtain a trained large language model;

[0052] The model application module 30 is used to receive a dialogue request carrying a question content, input the question content carried by the dialogue request into the trained large language model, obtain the answer content output by the trained large language model, and send a dialogue request feedback message carrying the answer content.

[0053] Here, an example is given of the unified format of the answers in the above reconstructed dialogue sample. The unified format of the answers means that the answers follow a certain template, such as "diagnosis results + treatment recommendations + precautions."

[0054] Here, we will provide an example of sampling disease knowledge samples from the medical knowledge graph. The medical knowledge graph is a structured knowledge base that aims to integrate and represent various medical-related information, including diseases, symptoms, treatment methods, drugs, clinical trials, etc. By sampling disease knowledge samples from the medical knowledge graph, we can provide basic data for generating knowledge graph question-answer pairs. Disease knowledge samples refer to a set of structured information related to a specific disease, usually including the name of the disease, the department corresponding to the disease, symptoms, causes, diagnostic methods, treatment plans, prognosis, and related clinical data.

[0055] Here, the preset quality indicators are exemplified. The preset quality indicators can be relevance indicators, clarity indicators, practical applicability indicators or interactivity indicators, etc. Relevance: The samples must be directly related to the medical topic to ensure that the content is helpful for medical decision-making or patient consultation. Clarity: The expression of questions and answers should be concise and easy to understand so as to be suitable for different audiences. Practical applicability: It has practical guiding significance and can effectively support patients in managing their health or help doctors make clinical decisions. Interactivity: It reflects real doctor-patient communication and demonstrates effective question-answering and interactive processes.

[0056] Here, the above-mentioned dialogue behavior preference index is exemplified. The dialogue behavior preference index refers to a quantitative standard used to measure the specific behaviors and attitudes shown by dialogue participants (such as doctors, patients, etc.) in the communication process. These indicators usually include the style, tone, politeness, emotional expression, efficiency and effectiveness of information transmission, etc., and are intended to evaluate the quality, effectiveness and satisfaction of the interaction between the two parties. For example, the dialogue behavior preference index can be politeness, professionalism, interactivity, etc.

[0057] Here, the large language models used in the dataset construction module and the model training module are exemplified. The model structure of the large language model used in the dataset construction module is different from the model structure of the large language model used in the model training module. For example, the large language model used in the dataset construction module is the GPT-3.5 model or the GPT-4 model, while the large language model used in the model training module is Baichuan-13B. Among them, GPT-3.5 has hundreds of billions of parameters and has strong language understanding and generation capabilities. The model performs well in many fields, such as text generation, translation, question and answer, etc. GPT-4 further improves the model parameter scale and performance based on GPT-3.5, and can better understand the context and generate more coherent and natural text. Baichuan-13B has 13 billion parameters. The model has strong capabilities in Chinese language processing and is suitable for a variety of scenarios, such as text generation, text classification, question and answer, etc.

[0058] It can be understood that by screening out conversation samples from the medical conversation dataset and reconstructing the screened conversation samples using a large language model (referred to as the large model), the answer content in the reconstructed conversation samples has a unified format, and using the reconstructed conversation samples to train the large language model can make the content output by the large language model meet the unified response standards.

[0059] By sampling knowledge triples from the medical knowledge graph based on the correlation between diseases and departments, and processing the sampled knowledge triples through a large language model, it is possible to generate knowledge graph question-answer pairs corresponding to the knowledge triples. Using the knowledge graph question-answer pairs to train the large language model can enhance the medical knowledge and application capabilities of the large language model.

[0060] Based on preset quality indicators, dialogue samples that meet the preset quality standards are screened out from the medical dialogue dataset, and the dialogue samples that meet the preset quality indicators are rewritten through a large language model. The rewritten dialogue samples meet the dialogue behavior preference indicators, and the large language model is trained with the rewritten dialogue samples, which can make the content output by the large language model better meet human behavior preferences.

[0061] Therefore, the reconstructed dialogue samples, knowledge graph question-answer pairs and rewritten dialogue samples are used to train the large language model. After the large language model is trained, by receiving dialogue requests carrying question content, inputting the question content carried by the dialogue request into the trained large language model, and obtaining the answer content output by the trained large language model, the trained large language model can be applied to providing medical dialogue services, so as to provide users with high-quality medical dialogue services in an end-to-end dialogue manner for actual medical consultation scenarios.

[0062] The present application also provides a medical dialogue system, including: a user terminal and the above-mentioned medical dialogue service device based on a large language model, the user terminal and the medical dialogue service device are communicatively connected; the user terminal is used to send a dialogue request carrying question content to the medical dialogue service device, and receive a dialogue request feedback message carrying answer content sent by the medical dialogue service device.

[0063] The following describes the medical dialogue service device and system based on a large language model provided by the present application with reference to several examples.

[0064] Figure 2 It is a technical principle flow chart provided by the embodiment of the present application, such as Figure 2 As shown, it includes data set construction and model training. By combining medical knowledge graphs, real doctor-patient conversations, and human preferences, a large language model is built that can understand and participate in real medical consultations, providing patients with end-to-end conversational medical services, so that patients can obtain accurate, real, and demand-oriented medical information.

[0065] The construction of the dataset is illustrated below.

[0066] Figure 3 is a flow chart for constructing a data set provided in an embodiment of the present application, such as Figure 3 As shown, in the process of constructing the MC-SFT (Medical Conversation Supervised Fine-Tuning) medical dialogue supervised fine-tuning dataset of this application, an innovative method is used to ensure the quality and relevance of the data, using the reconstruction of real dialogues, medical knowledge graphs, and human-guided preference reconstruction to train an efficient medical large language model (LLM). The following is a detailed description of the dataset construction.

[0067] The MC-SFT dataset includes reconstructed AI doctor-patient dialogues (that is, the reconstructed dialogue samples mentioned above), knowledge graph question-answer pairs, and behavioral preference datasets (that is, the dataset constructed using the rewritten dialogue samples mentioned above).

[0068] The data source for reconstructing the AI ​​doctor-patient dialogue part selected two public medical dialogue datasets, MedDialog and cMedQA2, from which 400,000 and 20,000 samples were selected as the basis for training data respectively. The samples cover a wide range of medical consultation scenarios to ensure the diversity and comprehensiveness of the data. In order to adjust the real doctor's answers to the required high-quality unified format answers, the model uses the GPT-3.5 model to reconstruct the selected samples to meet the response standards of the AI ​​medical assistant. The reconstructed doctor's response maintains medical accuracy and conforms to the communication style of the AI ​​assistant, while ensuring the integrity of the key information provided by the original doctor and providing patients with richer and more comprehensive help.

[0069] Data reconstruction step: Remove irrelevant or low-quality dialogue samples from the original dialogue dataset. Annotate the screened dataset from the aspects of medical entities, dialogue roles, medical advice, intentions and needs, and design prompts to effectively guide the model's answers. Input the preprocessed dialogue and designed prompts into the GPT-3.5 model. Generate adjusted dialogue samples using the language generation capabilities of GPT-3.5. Fine-tune the generated samples to ensure that they meet the response standards and medical professionalism of the AI ​​medical assistant. By asking medical professionals to review the medical accuracy, information completeness, language expression and emotional support of the extracted samples, collect the review feedback and use the adjusted samples as new samples to train the model so that the model can improve the output quality. Integrate the reconstructed and optimized samples into the training dataset.

[0070] Here is an example of removing irrelevant or low-quality conversation samples from the original conversation dataset. First, quality standards are set, including medical accuracy, completeness, and language expression. Then, natural language processing (NLP) tools are used to analyze the text to identify complex terms, ambiguous expressions, or grammatical errors, and to check whether key medical terms and concepts are included to ensure relevance. A scoring system is set for the samples, and samples with scores below the score are directly eliminated.

[0071] Here is an example of the prompt designed to effectively guide the model's answer. The prompt may be "Please reconstruct the doctor's answer based on the following dialogue sample to make it meet the standards of an AI medical assistant. Please ensure that the answer maintains medical accuracy, uses easy-to-understand language, and reflects a caring and supportive tone. At the same time, please reconstruct according to the following requirements:

[0072] 1. Use concise, gentle and friendly language, avoiding professional terms or explanations of terms;

[0073] 2. Provide symptomatic advice and self-management methods, and appropriately remind patients of the situation of reexamination or medical treatment;

[0074] 3. If the patient has questions about lifestyle habits, please include management suggestions such as diet and rest;

[0075] 4. Add brief words of encouragement at the end to help patients build confidence.

[0076] The following is a sample conversation. Please reconstruct it according to the above requirements: [fill in the location of the sample conversation]".

[0077] The prompt may reserve a [position to fill in the dialogue sample], and the dialogue sample may be filled in the position, and the prompt and the dialogue sample may be combined to determine the input content, and the input content may be input into the GPT-3.5 model. The language generation capability of GPT-3.5 is used to generate an adjusted dialogue sample (i.e., a reconstructed dialogue sample).

[0078] For example, the original conversation sample (conversation sample before reconstruction) can be "Patient: Doctor, I have been having stomach pain recently, and sometimes I have acid reflux, especially after eating, I feel very uncomfortable, what should I do?

[0079] Doctor (original answer): Stomach pain and acid reflux may be caused by chronic gastritis or excessive gastric acid secretion. It is recommended to avoid spicy and greasy food, try to eat a light diet, eat small meals frequently, and avoid overeating. You can try taking some acid-suppressing drugs, such as omeprazole, to control gastric acid secretion, but you need to take it according to the doctor's instructions. If the symptoms persist, it is recommended to go to the hospital for a gastroscopy to further clarify the diagnosis. "

[0080] Accordingly, the (doctor's) answer in the reconstructed dialogue sample can be "Hello, stomach pain and acid reflux are common in patients with chronic gastritis or increased gastric acid secretion. You can try the following measures to help relieve the discomfort:

[0081] 1. Diet control: Try to choose light, easily digestible food, avoid spicy, greasy or irritating food. At the same time, eat small meals frequently, and don't eat too much at one time, so as to reduce the burden on the stomach.

[0082] 2. Medication recommendations: You can try to use acid-suppressing drugs (such as omeprazole) under the guidance of a doctor to help reduce gastric acid secretion. The use of drugs should be in accordance with the doctor's advice to ensure safety and effectiveness.

[0083] 3. Adjust your lifestyle habits: Avoid lying down immediately after a meal, and try to keep your upper body upright for 30 minutes, which will help alleviate acid reflux.

[0084] If you still feel discomfort after taking these measures, it is recommended that you go to the hospital for a gastroscopy so that the doctor can further evaluate your condition and provide a more accurate treatment plan. I hope these suggestions are helpful to you and I wish you a speedy recovery. "

[0085] Prompts to guide the model's answers can also be designed for different dialogue scenarios, which may be basic consultation scenarios, symptom and suggestion scenarios, medical knowledge popularization scenarios, treatment plan scenarios, emotional support scenarios, individualized advice scenarios, attention follow-up scenarios, or scenarios where it is unclear how to answer.

[0086] (1) Design prompts to guide the model’s answers based on basic consultation scenarios;

[0087] Question: Based on the symptoms provided by the patient (e.g., stomach pain, acid reflux), please provide specific advice and treatment plans as a doctor. Make sure to use easy-to-understand language and express concern.

[0088] Answer: Hello, according to your symptoms, it may be gastritis or excessive gastric acid secretion. It is recommended that you: 1. Avoid spicy and greasy food, choose a light diet; 2. Eat small meals frequently and avoid overeating; 3. If the symptoms persist, you can consider using acid-suppressing drugs, but it is best to consult a doctor; please note that if the symptoms do not improve, it is recommended that you seek medical attention.

[0089] (2) Design prompts to guide the model's answers based on symptoms and suggestion scenarios;

[0090] Question: The patient says he has been feeling tired and insomniac recently. As a doctor, please explain the possible reasons and give specific lifestyle advice and self-management techniques.

[0091] Answer: Fatigue and insomnia may be related to a variety of factors, such as stress, lifestyle habits or underlying health problems. It is recommended that you: 1. Keep a regular schedule and go to bed and get up at the same time every day; 2. Avoid using electronic devices before going to bed; 3. Exercise moderately to help relieve stress. If the situation does not improve, it is recommended that you consult a professional doctor for further evaluation.

[0092] (3) Design prompts to guide the model’s answers for medical knowledge popularization scenarios;

[0093] Question: As a doctor, how would you explain to your patients the dangers of high blood pressure and how to manage it? Make sure your answer is simple and includes lifestyle adjustments that your patients need to be aware of.

[0094] Answer: High blood pressure increases the risk of heart disease and stroke. The key to managing high blood pressure is: 1. Eat a healthy diet, reduce salt intake, and increase fruits and vegetables; 2. Exercise regularly, at least 150 minutes of moderate-intensity activity per week; 3. Monitor blood pressure regularly and follow your doctor's treatment plan. If you have any questions, please feel free to communicate with your doctor.

[0095] (4) Design prompts to guide the model's answers based on treatment plan scenarios;

[0096] Question: If a patient is diagnosed with diabetes, can you provide them with detailed dietary advice and exercise plans? Make sure your answer includes medical evidence and shows concern for the patient's health.

[0097] Answer: The diet of diabetic patients should focus on controlling blood sugar. It is recommended to: 1. Choose low GI (glycemic index) foods, such as whole grains, beans, and non-starchy vegetables; 2. Eat regularly to avoid large meals that cause drastic fluctuations in blood sugar; 3. Exercise moderately, at least 150 minutes of aerobic exercise per week. Please be sure to follow the doctor's advice and check your blood sugar regularly.

[0098] (5) Design prompts to guide the model’s answers for emotional support scenarios;

[0099] Question: My patients are experiencing anxiety due to their chronic illness. As a doctor, please provide professional advice, encouragement, and emotional support to help them cope with this challenge.

[0100] Answer: I understand that chronic illness may make you anxious. I suggest you: 1. Communicate your feelings with your doctor and seek professional advice; 2. Join a support group to share experiences with other patients; 3. Maintain a healthy lifestyle, do moderate exercise and relaxation exercises. Please believe that a positive attitude and good management can help you cope with your condition.

[0101] (6) Design prompts to guide the model’s responses for individualized suggestion scenarios;

[0102] Question: The patient is a 65-year-old woman with high blood pressure and high cholesterol. Can you provide personalized dietary and lifestyle advice based on her health conditions? Make sure your advice is practical and easy to understand.

[0103] Answer: According to your health condition, we recommend: 1. Choose more fiber-rich foods, such as whole grains, fruits and vegetables; 2. Reduce the intake of red meat and high-fat foods, and choose fish and poultry; 3. Maintain moderate physical activity every day, such as walking or light fitness. Please check your blood pressure and cholesterol regularly and follow the doctor's instructions.

[0104] (7) Design prompts to guide the model’s answers for the focus follow-up scenario;

[0105] Ask: After making treatment recommendations, remind patients of the importance of regular checkups and describe symptoms or changes they should watch for.

[0106] Answer: After receiving treatment, please be sure to have regular checkups to monitor changes in your health. Pay attention to the following symptoms: 1. Fatigue, weakness or dizziness; 2. Stomach pain, nausea or other digestive discomfort. 3. Changes in weight or abnormal eating habits. If the above symptoms occur, please contact your doctor immediately.

[0107] (8) Designing prompts to guide the model’s answers when it is unclear how to answer the scenario;

[0108] Answer: I'm sorry, I don't know much about this question. I suggest you consult a professional medical staff to get accurate information.

[0109] The knowledge graph question-answer pair is based on the CMeKG medical knowledge graph. GPT-3.5 is used to perform knowledge sampling based on the correlation between diseases and departments. The data set after sampling presents a structured format. The data set structure contains the following fields: disease name, department name, relevance score (score calculated based on statistical analysis or model), other information (including common symptoms of the disease, recommended examinations or treatments), and more than 50,000 medical scenario question-answer pairs are generated through the designed GPT-3.5 model prompts to enhance the model's medical knowledge and application capabilities. For example, scenario: chronic cough; Question: What are the common causes of chronic cough? Answer: Common causes of chronic cough include chronic bronchitis, asthma, gastroesophageal reflux disease, allergies, and lung infections. Another example: scenario: diabetes management; Question: How do diabetic patients control blood sugar levels? Answer: Diabetic patients can control blood sugar levels through healthy eating, regular exercise, blood sugar monitoring, taking medication on time, and maintaining a good mental state.

[0110] Based on the manual selection criteria of relevance, clarity, practical applicability, and interactivity, the behavioral preference dataset manually selected about 2,000 representative high-quality samples from MedDialog and cMedQA2. Combining the generation capabilities of GPT-4 and manual review, the selected samples were rewritten and optimized to better match human behavioral preferences. Finally, a small number of optimized samples were used for further training through GPT-3.5 to refine the model's behavioral patterns and response quality.

[0111] Table 1 is a description of the data sources.

[0112] Table 1 Data source table

[0113]

[0114]

[0115] The following is an exemplary explanation of model training.

[0116] Figure 4 is a training flow chart of the model provided in the embodiment of the present application, such as Figure 4 As shown, the model training is based on a general-domain Chinese large language model Baichuan-13B (LLM) with 13 billion parameters, aiming to optimize the model's application performance in the medical consultation field through a two-stage training process.

[0117] Phase 1: Basic domain knowledge and conversation ability training. Use large-scale multi-source datasets, including AI doctor-patient conversation datasets (i.e., datasets constructed using reconstructed conversation samples), to simulate interactive conversations in real medical scenarios. Knowledge graph question-answer pairs, structured medical information extracted from medical knowledge graphs. Use supervised learning techniques to conduct preliminary training on the model using the above datasets. Through training, the model learns and internalizes medical expertise and conversation communication patterns. Use a deep learning framework to optimize model parameters. Apply natural language processing technology to enhance the model's ability to understand and respond to medical contexts.

[0118] Phase II: Behavioral Preference and Human-Guided Training. Use behavioral preference datasets (i.e., datasets constructed using rewritten conversation samples) that are carefully designed and screened based on human behavioral preferences. Based on the training in the first phase, further fine-tuning training is performed using behavioral preference datasets. By refining the response mechanism of the model, the naturalness and accuracy of the model in simulating human doctor behavior are improved. Careful parameter adjustments are implemented to ensure that the model output is highly consistent with human preferences.

[0119] Through the above two stages of training, the model not only mastered a wealth of medical knowledge and conversation skills, but also was able to interact naturally and accurately based on human communication preferences. This training method significantly improved the application value and user satisfaction of the model in actual medical consultation scenarios.

[0120] The following is an example of a conversation that illustrates how to apply a trained large language model to provide medical conversation services. Based on their health conditions, users can ask the model (in this example, the model can be called an AI medical assistant) clear questions about their needs. The model will give possible reasons, recommended treatment plans, etc. as references, and actively ask questions when information is lacking to enhance the pertinence and accuracy of the response. The following is the conversation content of this example.

[0121] In the first round of conversation, the user asked: I don’t know why my stomach has been hurting lately.

[0122] In the first round of dialogue, the AI ​​medical assistant answered: Hello, I am sorry to hear about your discomfort. According to the symptoms you described, it may be caused by gastritis or indigestion. In order to more accurately determine the cause of the disease, I suggest that you provide more descriptions of the current symptoms. In addition, if you can do some routine tests, such as blood routine, liver function, and gastroscopy, it will help to better understand the condition. While waiting for medical treatment, you can try the following measures to relieve symptoms: 1. Maintain good eating habits and avoid greasy and spicy foods; 2. Drink more water and eat more fresh vegetables and fruits; 3. Exercise moderately to strengthen your physical fitness; 4. Keep a good mood and avoid overwork and mental stress. If the symptoms do not improve significantly or worsen, or if other discomfort occurs, please seek medical attention in time and tell the doctor the details. I hope you will get well soon. If you have any questions or need further help, please feel free to let me know.

[0123] In the second round of conversation, the user asked: My abdominal pain is persistent, and strenuous activities will make it worse. In addition to the abdominal pain, I also feel a little nauseous. My eating habits are moderate, and sometimes I eat spicy food, but after eating, the abdominal pain will be aggravated.

[0124] In the second round of dialogue, the AI ​​medical assistant answered: According to your description, your abdominal pain may be related to a variety of factors, including gastrointestinal problems (such as gastric ulcers) and gallbladder diseases (such as gallstones). However, the specific diagnosis cannot be determined based on this information alone. Therefore, I suggest that you go to the hospital as soon as possible so that the cause can be determined through professional medical examinations. Before seeing a doctor, you can pay attention to your lifestyle and dietary adjustments. For example, try to avoid eating spicy and irritating foods to avoid aggravating the symptoms. At the same time, it is also important to maintain a regular schedule and get enough rest. If you have any other questions or need further help, please feel free to let me know. I will do my best to provide you with support and guidance.

[0125] In the third round of dialogue, the user asked: I went to see a doctor, and the doctor prescribed some stomach-strengthening and digestion-promoting tablets for me. They have some effect, but I still have abdominal pain sometimes, which is usually more severe at night. Please help me analyze what disease I might have, the specific causes, and give me a more effective solution, preferably about seven.

[0126] In the third round of dialogue, the AI ​​medical assistant answered: According to your description, you may have chronic superficial gastritis. This disease usually causes symptoms such as abdominal pain and loss of appetite. In addition, some doctors have mentioned other possible causes, such as indigestion and gastric ulcers. For treatment, oral omeprazole enteric-coated capsules are a commonly used drug option, which can reduce gastric acid secretion and thus reduce inflammatory response. In addition, Daxi can also be used as one of the options for adjuvant treatment, which has the effect of protecting the gastric mucosa. It should be noted that the above is only a preliminary inference, and the specific diagnosis needs to be determined in combination with more clinical manifestations and laboratory test results. If your symptoms do not improve significantly or worsen, or if other discomfort occurs, it is recommended to seek medical attention in time and provide more information to the doctor to obtain a more accurate diagnosis and treatment plan. If you have any questions about the above answers or need further help, please feel free to let me know.

[0127] In the fourth round of dialogue, the user asked: I have tried Omeprazole Enteric-coated Capsules. The effect is OK, but not very effective. By the way, I also have symptoms such as oral ulcers. Sometimes I feel a little dizzy and want to sleep. Please help me determine the cause of my disease and the solution in more detail.

[0128] AI medical assistant answers in the fourth round of dialogue: Thank you very much for the additional information you provided. According to your description, in addition to chronic superficial gastritis, oral ulcers and mild dizziness may also be related to other potential health problems. For the problem of oral ulcers, this may be caused by a variety of reasons, such as abnormal immune system, vitamin deficiency, stress, etc. It is recommended that you maintain oral hygiene, avoid spicy and irritating foods, and appropriately supplement vitamin B group and other nutrients. If oral ulcers seriously affect your daily life, you can consult your doctor whether it is appropriate to use some local antibacterial drugs or mouthwash. As for the feeling of dizziness and wanting to sleep, this may be caused by a variety of reasons, such as lack of sleep, anemia, low blood pressure, etc. It is recommended that you ensure adequate sleep time and avoid standing for a long time or getting up too quickly when getting up suddenly. If the symptoms of dizziness persist and affect your daily life, it is recommended that you consult a doctor for more details and obtain corresponding treatment suggestions. In short, the possibility of other potential health problems cannot be completely ruled out at present. If your symptoms do not improve significantly or worsen, or if you experience other discomfort, it is recommended that you seek medical attention in time and provide your doctor with more information to obtain a more accurate diagnosis and treatment plan. If you have any questions or need further help, please feel free to let me know.

[0129] The model evaluation is described below.

[0130] In order to systematically evaluate the conversational capabilities of the model, samples were randomly selected from three public data sets - Chinese Medical Benchmark Evaluation (CMB-Clin), Chinese Medical Dialogue Dataset (CMD) and Chinese Medical Intent Dataset (CMID). The GPT-3.5 model was used to simulate the role of the patient and conduct multiple rounds of interaction with the model to be evaluated to simulate the real medical consultation process. The GPT-4 model was used as an evaluation tool to score based on these four evaluation indicators - Proactivity, Accuracy, Helpful and Linguistic. The scoring results are shown in Table 2, where MC-MedLLM is a trained large language model in the medical dialogue service device based on a large language model provided in this application.

[0131] Table 2 Scoring results

[0132]

[0133] Through the above evaluation methods, the performance of the model in the field of medical consultation has been comprehensively tested and verified. The evaluation results show that the model can accurately understand medical-related questions and provide reliable answers, while showing excellent interaction capabilities and user satisfaction in multiple rounds of dialogue.

[0134] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A medical dialogue service device based on a large language model, characterized in that: include: Dataset construction module, model training module and model application module; The dataset construction module includes a dialogue sample reconstruction unit, a knowledge graph question-answer pair generation unit, and a dialogue sample rewriting unit; A dialogue sample reconstruction unit is used to select dialogue samples from the medical dialogue data set, reconstruct the selected dialogue samples using a large language model, and obtain reconstructed dialogue samples. The answer content in the reconstructed dialogue samples has a unified format; The knowledge graph question-answer pair generation unit is used to sample disease knowledge samples from the medical knowledge graph, process the sampled disease knowledge samples through a large language model, and generate knowledge graph question-answer pairs corresponding to the disease knowledge samples; A dialogue sample rewriting unit is used to screen dialogue samples that meet the preset quality indicators from the medical dialogue data set based on the preset quality indicators, rewrite the dialogue samples that meet the preset quality indicators through a large language model, and obtain the rewritten dialogue samples, and the rewritten dialogue samples meet the dialogue behavior preference indicators; The model training module is used to train the large language model based on the reconstructed dialogue samples, the knowledge graph question-answer pairs, and the rewritten dialogue samples to obtain the trained large language model; The model application module is used to receive a dialogue request carrying question content, input the question content carried by the dialogue request into the trained large language model, and obtain the answer content output by the trained large language model.

2. The medical dialogue service device based on a large language model according to claim 1, characterized in that: The dialogue sample reconstruction unit is specifically used for: Perform annotation preprocessing on the screened dialogue samples to obtain preprocessed dialogue samples; Based on the preprocessed dialogue sample and the pre-configured first prompt word, determine the input content and input the input content into the large language model to obtain the reconstructed dialogue sample output by the large language model; The first prompt word is used to instruct the large language model to reconstruct the preprocessed dialogue sample according to the specified format.

3. According to claim 1, the medical dialogue service device based on a large language model is characterized in that: The knowledge graph question-answer pair generating unit is specifically used for: Based on the sampled disease knowledge samples and the pre-configured second prompt words, determine the input content and input the input content into the large language model to obtain the knowledge graph question-answer pair output by the large language model; Among them, the second prompt word is used to instruct the large language model to generate a knowledge graph question-answer pair based on the sampled disease knowledge samples.

4. The medical dialogue service device based on a large language model according to claim 1, characterized in that: The dialogue sample rewriting unit is specifically used for: Based on the dialogue sample meeting the preset quality index and the pre-configured third prompt word, determine the input content and input the input content into the large language model to obtain the rewritten dialogue sample output by the large language model; The third prompt word is used to instruct the large language model to generate a rewritten dialogue sample based on the rewritten dialogue sample.

5. The medical dialogue service device based on a large language model according to claim 1, characterized in that: The dialogue sample rewriting unit is specifically used for: Displaying preset quality indicators and conversation samples in the medical conversation dataset, and receiving a target input, where the target input is used to indicate a conversation sample that meets the preset quality indicators; In response to the target input, a dialog sample meeting a preset quality indicator is determined.

6. The medical dialogue service device based on a large language model according to claim 1, characterized in that: The model training module is specifically used for: Based on the reconstructed dialogue samples and knowledge graph question-answer pairs, the large language model is trained in the first stage to obtain the large language model after the first stage of training; Based on the rewritten dialogue samples, the large language model trained in the first phase is trained in the second phase to obtain the large language model trained in the second phase.

7. The medical dialogue service device based on a large language model according to claim 1, characterized in that: Also includes: The sample adjustment module is used to adjust one or more of the following samples: reconstructed dialogue samples, knowledge graph question-answer pairs, and rewritten dialogue samples.

8. The medical dialogue service device based on a large language model according to claim 1, characterized in that: Also includes: The historical conversation storage module is used to store the conversation content of the trained large language model in the process of providing medical conversation services.

9. The medical dialogue service device based on a large language model according to claim 8, characterized in that: The historical conversation storage module is also used to adjust the stored conversation content.

10. A medical dialogue system, characterized in that: include: A user terminal and a medical dialogue service device based on a large language model as described in any one of claims 1 to 9, wherein the user terminal and the medical dialogue service device are in communication connection; The user terminal is used to send a dialogue request carrying question content to the medical dialogue service device, and receive a dialogue request feedback message carrying answer content sent by the medical dialogue service device.

Images