Intelligent medical decision and intervention method and system based on multi-source data fusion large language model

Abstract

The invention relates to the technical field of intelligent hospital guidance, and discloses an intelligent medical decision and intervention method and system based on a multi-source data fusion large language model. The method specifically comprises the steps of collecting medical dialogue data from real dialogue data of a plurality of medical centers, establishing a database, and performing fine adjustment on a pre-trained large language model based on the medical dialogue data to obtain a large language model in a medical dialogue scene. The real-time medical dialogue data is analyzed according to the large language model in the medical dialogue scene, the doctor's advice content is generated, the mood and frequency of the doctor's advice content are adjusted according to the illness state and the emotional state of the patient, the capabilities of the large language model in the aspects of semantic comprehension, natural interaction and regionalization adaptation are remarkably improved, and the user experience is improved. A more efficient, accurate and personalized solution is provided for communication between the patient and the medical staff, communication obstacles between the patient and the medical staff can be relieved, and powerful technical support is provided for intelligence of medical services.

Description

Technical Field

[0001] The present application relates to the field of intelligent medical guidance technology, and specifically to an intelligent medical decision-making and intervention method and system based on multi-source data fusion and large language model. Background Art

[0002] In medical scenarios, the following problems are common in the communication process between patients and nurses: the needs expressed by patients are not clear enough, and there may even be missing information or vague descriptions; patients often lack understanding of the medical process, and the questions they raise may involve multiple departments or cross-departmental affairs; nurses need to quickly judge the complex needs of patients and provide solutions within a limited time.

[0003] Traditionally, patients receive guidance information through manual interactions with nurses, front desk staff, or the information desk. This model relies on human experience and judgment. While flexible enough to handle complex scenarios, it's prone to omissions and errors. Furthermore, due to high labor costs and limited service efficiency, this approach struggles to meet the needs of large numbers of patients, especially during peak medical periods. This often results in long wait times and a poor patient experience.

[0004] In recent years, with the rapid development of artificial intelligence and natural language processing technologies, intelligent guidance systems have gradually become a vital tool for medical institutions to optimize their services. Traditional guidance systems primarily rely on manual guidance, rule-based question-and-answer systems, or simple keyword searches. While these systems have improved guidance efficiency and alleviated the uneven distribution of medical resources to a certain extent, existing systems are limited in their intelligence. They lack the ability to accurately parse complex patient semantics, are unable to dynamically address multi-level semantic decomposition or in-depth interaction, and struggle to meet personalized needs. Furthermore, the needs of medical institutions in different regions vary significantly, but traditional guidance technologies are generally designed for general use and lack regional adaptability, making it difficult to optimize service strategies based on local medical resources.

[0005] With the emergence of large language models (such as GPT and BERT), the application potential of deep learning-based natural language processing technology in the medical field has gradually become apparent. Large language models have been proven to be capable of handling complex language interaction tasks due to their powerful semantic understanding and generation capabilities. However, in actual medical guidance, traditional large language models face the following challenges: First, most existing models are trained based on general corpora and lack support from specialized corpora for medical scenarios, resulting in insufficient analysis of medical terminology, situational semantics, and patient needs; second, medical guidance needs are highly regionalized and diversified, and a single model cannot take into account the specific needs of different medical institutions; third, communication between patients and nurses involves emotional expressions, incomplete descriptions, and contextual associations, which places higher demands on the guidance system's understanding capabilities.

[0006] In summary, in complex medical scenarios, traditional manual medical guidance is prone to omissions or errors. Traditional medical guidance systems have problems such as rigid interactions, low response accuracy, and lack of personalization. Large language models lack the ability to analyze the medical field, making it difficult to take into account the specific needs of different medical institutions, and their understanding ability is limited. Summary of the Invention

[0007] This application provides an intelligent medical decision-making and intervention method based on a large language model that integrates multi-source data to address the problems in the existing technology: traditional manual guidance is prone to omissions or errors in complex medical scenarios; traditional guidance systems have rigid interactions, low response accuracy, and insufficient personalization; and large language models have insufficient analytical capabilities in the medical field, difficulty in balancing the specific needs of different medical institutions, and limited understanding capabilities.

[0008] Correspondingly, the present application also provides an intelligent medical decision-making and intervention system based on multi-source data fusion and large language model, an electronic device, and a computer-readable storage medium to ensure the implementation and application of the above method.

[0009] In order to solve the above technical problems, the present application discloses an intelligent medical decision-making and intervention method based on multi-source data fusion and large language model, the method comprising:

[0010] Collect medical conversation data from real conversation data from multiple medical centers;

[0011] Fine-tune the pre-trained large language model based on medical conversation data to obtain a large language model for medical conversation scenarios;

[0012] Analyze real-time medical conversation data based on the large language model in medical conversation scenarios, generate medical advice content, and adjust the tone and frequency of the advice content according to the patient's condition and emotional state.

[0013] This application also discloses an intelligent medical decision-making and intervention system based on multi-source data fusion and large language model, the system comprising:

[0014] A data collection module, used to collect medical conversation data from real conversation data of multiple medical centers;

[0015] The model optimization module is used to fine-tune the pre-trained large language model based on medical conversation data to obtain a large language model for medical conversation scenarios;

[0016] The medical order generation module is used to analyze real-time medical conversation data based on the large language model in the medical conversation scenario, generate medical order content, and adjust the tone and frequency of the medical order content according to the patient's condition and emotional state.

[0017] The present application also discloses an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, one or more methods described in the present application are implemented.

[0018] The present application also discloses a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, one or more methods described in the present application are implemented.

[0019] In this application, medical conversation data is collected from real conversation data of multiple medical centers, and a large language model for medical conversation scenarios is obtained based on the medical conversation data and fine-tuning of the pre-trained large language model. Real-time medical conversation data is analyzed according to the large language model in the medical conversation scenario to generate medical advice content, and the tone and frequency of the medical advice content are adjusted according to the patient's condition and emotional state. The above method significantly improves the ability of the large language model in semantic understanding, natural interaction and regional adaptation, and has obvious technical advantages and application value in the field of intelligent medical guidance. It provides a more efficient, accurate and personalized solution for communication between patients and medical staff, and can also alleviate the communication barriers between patients and medical staff, providing strong technical support for the intelligentization of medical services.

[0020] Additional aspects and advantages of the present application will be given in the following description, which will become apparent from the following description, or will be understood through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0022] Figure 1 A flowchart of an intelligent medical decision-making and intervention method based on multi-source data fusion and large language model provided in an embodiment of the present application;

[0023] Figure 2 A flowchart of the application of a large language model in a medical conversation scenario provided by an embodiment of this application;

[0024] Figure 3 A detailed flowchart of generating medical advice content using a large language model in a medical conversation scenario provided in an embodiment of the present application;

[0025] Figure 4 A schematic diagram of the structure of an intelligent medical decision-making and intervention system based on a multi-source data fusion large language model provided in an embodiment of the present application;

[0026] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The following describes embodiments of the present application in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and are not to be construed as limiting the present application.

[0028] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of this application refers to the presence of features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.

[0029] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art in the art to which the present invention pertains. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and, unless specifically defined as herein, will not be interpreted in an idealized or overly formal sense.

[0030] The solution provided in the embodiments of the present application can be executed by any electronic device, such as a terminal device or a server, wherein the server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this. With respect to the technical problems existing in the prior art, the intelligent medical decision-making and intervention method and system based on multi-source data fusion and large language model provided in this application is intended to solve at least one of the technical problems of the prior art.

[0031] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0032] The present application embodiment provides a possible implementation method, such as Figure 1 As shown, a flowchart of an intelligent medical decision-making and intervention method based on multi-source data fusion and a large language model is provided. The solution can be executed by any electronic device, and optionally, can be executed on the server side or the terminal device.

[0033] like Figure 1 As shown in , the method may include the following steps:

[0034] Step 101: Collect medical conversation data from real conversation data of multiple medical centers.

[0035] For example, medical conversation data was collected from real conversation data of 6 medical centers. These medical conversation data cover multiple types of medical inquiry scenarios, including administrative affairs, department classifications, primary care issues, etc. After sorting, typical cases with classification labels were generated, providing a solid data foundation for the precise training and optimization of large language models.

[0036] Step 102: fine-tune the pre-trained large language model based on the medical conversation data to obtain a large language model for the medical conversation scenario;

[0037] In this embodiment, a model pre-trained on a large-scale corpus, such as the DeepSeek large language model, can be selected as the base model. By training a large language model based on real medical conversation data, the semantic understanding ability of the large language model can be significantly improved, enabling accurate understanding of patients' diverse expressions, including ambiguous descriptions, multi-task queries, and emotional language.

[0038] Step 103: Analyze the real-time medical conversation data based on the large language model in the medical conversation scenario, generate medical advice content, and adjust the tone and frequency of the medical advice content according to the patient's condition and emotional state.

[0039] In an embodiment of the present application, medical conversation data is collected from real conversation data of multiple medical centers, and a large language model for medical conversation scenarios is obtained based on the medical conversation data and fine-tuning of the pre-trained large language model. Real-time medical conversation data is analyzed according to the large language model in the medical conversation scenario to generate medical advice content, and the tone and frequency of the medical advice content are adjusted according to the patient's condition and emotional state. The above method significantly improves the ability of the large language model in semantic understanding, natural interaction and regional adaptation, and has obvious technical advantages and application value in the field of intelligent medical guidance. It provides a more efficient, accurate and personalized solution for communication between patients and medical staff, and can also alleviate the communication barriers between patients and medical staff, providing strong technical support for the intelligentization of medical services.

[0040] In an optional embodiment, the types of medical centers include general hospitals, specialized hospitals, and primary medical institutions;

[0041] The real conversation data is a three-party conversation data including the doctor, nurse and patient.

[0042] The medical conversation data includes audio conversation data, text conversation data converted from the audio conversation data, and corresponding classification labels.

[0043] For example, a total of 269,854 minutes of audio conversation data were collected from conversations between real patients and nurses and doctors in 6 medical centers, with at least 4,000 minutes of audio conversation data collected from each medical center. The corresponding text conversation data was obtained through speech-to-text conversion, and a comprehensive corpus containing 1,984,613 words was constructed through further data cleaning (such as privacy de-identification processing, compliance with HIPAA standards). Among them, the types of medical centers include general hospitals (including internal medicine, surgery, obstetrics and gynecology, etc.), specialized hospitals (such as ophthalmology, pediatrics, psychiatry, etc.) and primary medical institutions (such as community hospitals, rural health centers, etc.). Therefore, the constructed comprehensive corpus covers multiple types of medical inquiry scenarios, including administrative affairs, department classification, primary care issues, etc. Through conversation annotation, 26,582 medical conversation data with classification labels were organized as typical cases.

[0044] In an optional embodiment, fine-tuning the pre-trained large language model based on medical conversation data to obtain a large language model for medical conversation scenarios includes:

[0045] Incorporate scenario feature knowledge into the pre-trained large language model, input medical conversation data into the pre-trained large language model, and fine-tune the pre-trained large language model to obtain a large language model optimized for medical conversation scenarios;

[0046] Among them, scenario feature knowledge includes general medical knowledge, hospital-specific knowledge, and dynamically updated knowledge.

[0047] In this embodiment, a scenario-specific knowledge base is constructed based on scenario-specific knowledge, including medical knowledge, hospital-specific knowledge, and dynamically updated knowledge, providing real-time, callable background knowledge for the large language model. General medical knowledge includes basic health knowledge and common disease diagnosis and treatment guidelines; hospital-specific knowledge includes outpatient procedures, department locations, doctor schedules, and medical insurance policies; and dynamically updated knowledge includes database content updated based on user feedback and the latest medical policies.

[0048] During fine-tuning of the pre-trained large language model using medical conversation data, a specially designed scenario-specific knowledge base is incorporated to incorporate scenario-specific knowledge, including hospital processes, departmental distribution, medical terminology, and answers to common medical questions. This provides multi-round dialogue support and ensures contextual consistency in complex scenarios, enabling the large language model to provide professional and accurate responses in real-world medical settings. The scenario-specific knowledge base can be dynamically updated to ensure adaptability and continued effectiveness in various medical scenarios. Through intelligent knowledge management and self-learning capabilities, the large language model can quickly adapt to various medical scenarios and is continuously optimized based on feedback from doctors and patients, improving its adaptability and accuracy.

[0049] In an optional embodiment, when fine-tuning the pre-trained large language model, a distributed training method combined with a feedback mechanism is used to optimize the pre-trained large language model.

[0050] Distributed training involves parallel model training across multiple computers (or computing units). Due to the diversity of data, the model develops the ability to flexibly adapt to different expressions within different weight spaces. This data is ultimately uploaded and synchronized with the master model, allowing it to incorporate regional variations.

[0051] The process of optimizing the pre-trained large language model using the distributed training method and feedback mechanism in the embodiment of the present application is as follows:

[0052] Medical conversation data from different medical centers is distributed to multiple nodes (GPUs or servers) and trained simultaneously. Each node fine-tunes its sub-model to adapt to the data in its area.

[0053] The sub-model of each node synchronizes parameters through federated learning or model integration to form a large language model that has been initially fine-tuned for medical dialogue scenarios. This model can take into account the fine-grained differences among various medical centers and can answer most medical questions.

[0054] When doctors and patients, doctors, and nurses have online conversations, the embodiment of the present application can collect data such as patient feedback, doctor annotations, and system automatic scoring, and form a new fine-tuning data set by sorting;

[0055] Fine-tune the sub-model using the new fine-tuning dataset for local optimization, and then update the large language model for medical conversation scenarios;

[0056] After multiple rounds of iterations, the large language model for medical conversation scenarios becomes more and more familiar with the actual scenarios and can continuously improve the response quality.

[0057] In the embodiment of the present application, a distributed training method is used to conduct special training on medical conversation data from different medical centers, so that the large language model can be dynamically adjusted according to the resource allocation characteristics and service processes of medical institutions in different regions, thereby realizing guidance services tailored to local conditions and making up for the problem of insufficient regional adaptability.

[0058] In the embodiment of this application, Figure 2 As shown, after collecting medical conversation data from multiple medical centers and fine-tuning the pre-trained large language model to generate a large language model for the medical conversation scenario, the application process of the large language model in the medical conversation scenario includes: initialization and setting, real-time conversation and feedback, medical order generation and prompts, closed-loop feedback, and equipment monitoring and model optimization. The specific implementation method will be described in detail in the following embodiments.

[0059] In an optional embodiment, if Figure 3 As shown, before analyzing real-time medical conversation data based on a large language model in a medical conversation scenario, generating medical advice content, and adjusting the tone and frequency of the medical advice content based on the patient's condition and emotional state, the method also includes:

[0060] System preparation and installation: Connect the large language model for medical conversation scenarios with the hospital information management system (HIS), and use the HIS to enter patient information and authenticate identities to ensure the security of patient information.

[0061] System integration and docking: Integrate the large language model in the medical dialogue scenario with the medical data system of the medical center to ensure that information such as medical orders, patient health records, department processes, and system output content can be called and recorded in real time.

[0062] The embodiment of the present application supports multiple device interfaces: hospital information management system integration interface, medical staff dedicated terminal interface, and supports multimodal input (voice, text), ensuring that medical staff and patients can communicate and interact at any time, facilitating monitoring and control, and improving operability and humanization.

[0063] In an optional embodiment, real-time medical conversation data is analyzed based on a large language model in a medical conversation scenario to generate medical advice content, and the tone and frequency of the advice content are adjusted according to the patient's condition and emotional state, including:

[0064] Obtain the patient's basic information, current health status and historical medical history, and display them in the doctor's window;

[0065] If the doctor provides initial medical order content, the initial medical order content is adjusted according to the real-time medical conversation data to generate the medical order content; otherwise, the medical order content is generated according to the real-time medical conversation data.

[0066] In the embodiment of the present application, data is collected based on the patient's basic information, current health status and historical medical history to obtain the patient's health data. The collected health data is transmitted to the large language model through the hospital information management system for the large language model to generate personalized medical advice and health recommendations.

[0067] Specifically, if Figure 3 As described above, the collected health data is displayed in the doctor's window. If the doctor provides the initial medical order content based on these data, the customized initial medical order content is automatically pushed, providing key information, and the medical conversation data between the doctor, nurse and patient is obtained through the large language model for analysis, and the initial medical order content provided by the doctor is adjusted according to the analyzed medical conversation data and the patient's health data. If the doctor does not provide the initial medical order content based on the patient's health data, the large language model obtains the medical conversation data between the doctor, nurse and patient for analysis, and generates the medical order content based on the analyzed medical conversation data and the patient's health data. The medical order content automatically generated by the large language model includes drug dosage, treatment cycle, precautions, etc.

[0068] Among them, the large language model can achieve real-time support for three parties: supporting real-time collaborative dialogues between doctors, nurses and patients, being able to understand the medical dialogue data between doctors, nurses and patients in real time, converting speech into text and generating speech feedback through real-time speech recognition and synthesis sub-modules, and generating responses in medical scenarios, providing accurate and personalized medical order content.

[0069] The large language model can realize sentiment analysis and empathy regulation: In the process of three-party real-time support, with the support of deep learning algorithms, the large language model can parse and respond to the complex interactions between patients, doctors, and nurses in real time, providing patients with clearer, more accurate, and more empathetic responses. Specifically, the large language model can analyze the patient's emotional changes (such as anger, anxiety) during the conversation in real time, and make real-time emotional adjustments and feedback based on the patient's needs and emotional state. When the patient is depressed or anxious, the tone and emotional expression of the conversation content are adjusted in a timely manner to provide patients with clearer, more accurate, and more empathetic soothing sentences to help patients reduce anxiety and increase their trust and compliance. This not only ensures high-quality dialogue generation, but also can be adaptively adjusted according to different medical scenarios to improve the patient's overall experience.

[0070] After analyzing sentiment and adjusting empathy, the large language model can adjust the tone and frequency of medical instructions based on the patient's condition and emotional state, using different tones and words to emphasize the instructions to ensure that the patient understands and follows them. Based on patient feedback, the system continuously analyzes sentiment and adjusts empathy, adjusting the tone and frequency of medical instructions. Finally, the conversation content is recorded and entered into the patient's medical record.

[0071] The large language model can also achieve problem triage: when a conversation occurs between doctors, nurses, and patients, the large language model can automatically identify the type of question and assign it to the appropriate object (such as medical order questions are transferred to doctors, and administrative questions are transferred to nurses).

[0072] The large language model can automatically optimize based on feedback information: by collecting feedback from patients and medical staff, it automatically optimizes the output to ensure the accuracy and effectiveness of medical orders, especially by timely adjusting the content of medical orders during multiple rounds of conversations to ensure the integrity of the information.

[0073] The embodiment of the present application is based on the semantic generation capability of a large language model, achieving human-like natural language dialogue, accurately capturing patient needs in multiple rounds of dialogue, dynamically adjusting inquiry strategies, and providing patients with a more intimate interactive experience, which is far superior to the mechanized question-and-answer mode.

[0074] In the embodiment of the present application, when the large language model identifies an uncertain or potentially high-risk answer, manual intervention is triggered if necessary. In the manual review interface, medical staff are allowed to modify or confirm the output of the large language model in real time.

[0075] In an optional embodiment, after analyzing real-time medical conversation data based on a large language model in a medical conversation scenario to generate medical advice content, and adjusting the tone and frequency of the medical advice content based on the patient's condition and emotional state, the method further includes:

[0076] Push medical advice to patients and adjust the frequency and intensity of push notifications based on set rules.

[0077] The system automatically adjusts the frequency and intensity of medical advice push notifications based on pre-set rules, ensuring patients receive adequate guidance in their doctor's absence. For example, at night or when the on-call doctor is unable to respond immediately, the system can automatically reinforce medical advice reminders to remind patients to follow their treatment plan.

[0078] In an optional embodiment, after analyzing real-time medical conversation data based on a large language model in a medical conversation scenario to generate medical advice content, and adjusting the tone and frequency of the medical advice content based on the patient's condition and emotional state, the method further includes:

[0079] Optimize the large language model in medical dialogue scenarios based on patients' feedback on medical advice content, emotional changes, symptom changes, and treatment effects, and update the medical advice content.

[0080] In the embodiments of this application, patient feedback on the content of medical orders, emotional changes, symptom changes, and treatment effects are continuously collected to ensure the effective implementation of treatment plans and the accurate delivery of medical orders. Through feedback mechanisms and self-learning mechanisms, the large language model is continuously optimized to improve its adaptability to the personalized needs of patients. After each data feedback, the large language model can optimize the medical order content generation and dialogue management strategy based on the patient's changing situation, thereby improving the quality and efficiency of medical services.

[0081] Through feedback and self-learning mechanisms, the large language model's output can be adjusted in real time as patients raise questions, and conversation results can be updated and optimized in real time based on the medical staff's actions. For example, at night, the system can automatically increase the frequency and emphasis of medical advice reminders based on the patient's health status and the content of the instructions, ensuring that patients do not miss important treatment information due to the doctor's absence. Furthermore, the system can monitor patient feedback in real time and dynamically adjust communication strategies based on the patient's level of understanding and compliance, ensuring the efficient delivery of medical advice.

[0082] Through the feedback mechanism and self-learning mechanism, real-time data analysis and evaluation can be achieved: real-time analysis of patient feedback data can be performed to detect patients' understanding and execution of medical orders. If patients fail to take medication on time or have other abnormalities, the system will trigger a preset alarm mechanism to prompt doctors or nurses to intervene.

[0083] Feedback and self-learning mechanisms enable personalized adjustments and interventions: Based on real-time feedback data, the system intelligently adjusts the frequency and intensity of doctor's orders, ensuring that patients accurately understand and implement their treatment plans. For example, if a patient exhibits an allergic reaction to a medication or resists treatment, the system automatically adjusts the treatment plan and notifies the doctor for further diagnosis and treatment.

[0084] Feedback and self-learning mechanisms enable treatment effectiveness evaluation: by monitoring the patient's treatment progress and health status, the system automatically assesses efficacy and makes real-time adjustments. Based on the patient's mood, symptom changes, and treatment response, it optimizes medical advice and adjusts treatment plans to achieve personalized treatment results.

[0085] Through the above-mentioned feedback mechanism and self-learning mechanism, the large language model has the ability to continuously learn, and can optimize the performance of the large language model through real-time feedback of patient interaction data, thereby being able to adapt to rapidly changing medical needs and maintain long-term efficient and accurate service capabilities.

[0086] Based on the above, the efficiency, flexibility, and intelligence of the intelligent medical decision-making and intervention method based on the multi-source data fusion large language model in the embodiment of this application are highlighted, which greatly improves the quality and efficiency of medical services, provides patients with a more personalized medical experience, and optimizes the hospital's operational efficiency and the workflow of medical staff. It can effectively solve problems such as doctor duty hours and uneven distribution of medical resources, ensure that patients receive timely medical support at critical moments, and provide medical staff with a more convenient work platform, ultimately promoting the intelligent and refined management of medical services.

[0087] Based on the same principle as the method provided in the embodiment of the present application, the embodiment of the present application also provides an intelligent medical decision-making and intervention system based on multi-source data fusion and large language model, such as Figure 4 As shown, the system includes:

[0088] The data collection module 401 is used to collect medical conversation data from real conversation data of multiple medical centers.

[0089] For example, medical conversation data was collected from real conversation data of 6 medical centers. These medical conversation data cover multiple types of medical inquiry scenarios, including administrative affairs, department classifications, primary care issues, etc. After sorting, typical cases with classification labels were generated, providing a solid data foundation for the precise training and optimization of large language models.

[0090] The model optimization module 402 is used to fine-tune the pre-trained large language model based on the medical conversation data to obtain a large language model in the medical conversation scenario.

[0091] In this embodiment, a model pre-trained on a large-scale corpus, such as the DeepSeek large language model, can be selected as the base model. By training a large language model based on real medical conversation data, the semantic understanding ability of the large language model can be significantly improved, enabling accurate understanding of patients' diverse expressions, including ambiguous descriptions, multi-task queries, and emotional language.

[0092] The medical order generation module 403 is used to analyze real-time medical conversation data based on the large language model in the medical conversation scenario, generate medical order content, and adjust the tone and frequency of the medical order content according to the patient's condition and emotional state.

[0093] In the embodiment of the present application, medical conversation data is collected from real conversation data of multiple medical centers, and a large language model for medical conversation scenarios is obtained based on the medical conversation data and fine-tuning of the pre-trained large language model. The real-time medical conversation data is analyzed according to the large language model for medical conversation scenarios to generate medical advice content, and the tone and frequency of the medical advice content are adjusted according to the patient's condition and emotional state. This significantly improves the system's capabilities in semantic understanding, natural interaction, and regional adaptation. It has obvious technical advantages and application value in the field of intelligent medical guidance, provides a more efficient, accurate, and personalized solution for communication between patients and medical staff, and can also alleviate the communication barriers between patients and medical staff, providing strong technical support for the intelligentization of medical services.

[0094] In an optional embodiment, the types of medical centers include general hospitals, specialized hospitals, and primary medical institutions;

[0095] The real conversation data is a three-party conversation data including the doctor, nurse and patient.

[0096] The medical conversation data includes audio conversation data, text conversation data converted from the audio conversation data, and corresponding classification labels.

[0097] In an optional embodiment, the model optimization module includes:

[0098] The model optimization submodule is used to integrate scenario feature knowledge into the pre-trained large language model, input medical conversation data into the pre-trained large language model, and fine-tune the pre-trained large language model to obtain a large language model optimized for medical conversation scenarios;

[0099] Among them, scenario feature knowledge includes general medical knowledge, hospital-specific knowledge, and dynamically updated knowledge.

[0100] In an optional embodiment, when fine-tuning the pre-trained large language model, a distributed training method combined with a feedback mechanism is used to optimize the pre-trained large language model.

[0101] In an optional embodiment, the medical order generation module includes:

[0102] The first medical order generation submodule is used to obtain the patient's basic information, current health status and historical medical history, and display them in the doctor's window;

[0103] The second medical order generation submodule is used to adjust the initial medical order content according to the real-time medical conversation data to generate the medical order content if the doctor provides the initial medical order content; otherwise, the medical order content is generated according to the real-time medical conversation data.

[0104] In an optional embodiment, the system further includes:

[0105] The medical order push module is used to push medical order content to patients and adjust the frequency and intensity of push according to the set rules.

[0106] In an optional embodiment, the system further includes:

[0107] The medical order update module is used to optimize the large language model in medical dialogue scenarios and update the medical order content based on the patient's feedback on the medical order content, emotional changes, symptom changes and treatment effects.

[0108] In an optional embodiment, the system supports multiple device interfaces, including a hospital information system integration interface and a medical staff-specific terminal interface, and supports multimodal input (voice, text), ensuring that medical staff and patients can communicate and interact at any time, facilitating monitoring and control, and improving operability and humanization.

[0109] During system initialization, the following are included:

[0110] Equipment preparation and installation: In a hospital environment, the system is first pre-configured and configured. This includes entering patient information and performing identity authentication through the Hospital Information Management System (HIS) to ensure the security of patient information.

[0111] System integration and docking: Integrate the large language model with the hospital's internal medical data system to ensure that information such as medical orders, patient health records, departmental processes, and system output content can be called and recorded in real time.

[0112] In an optional embodiment, the system realizes all-weather real-time two-way wireless acquisition of neural signals through an in vitro wireless power supply and signal transmission system, greatly improving the efficiency and stability of signal acquisition.

[0113] The intelligent medical decision-making and intervention system based on multi-source data fusion and large language model provided in the embodiment of the present application can achieve Figures 1 to 3 To avoid repetition, the various processes implemented in the method embodiment will not be described again here.

[0114] The intelligent medical decision-making and intervention system based on multi-source data fusion and large language model in the embodiment of the present application can execute the intelligent medical decision-making and intervention method based on multi-source data fusion and large language model provided in the embodiment of the present application. The implementation principle is similar. The actions performed by each module and unit in the intelligent medical decision-making and intervention system based on multi-source data fusion and large language model in each embodiment of the present application correspond to the steps in the intelligent medical decision-making and intervention method based on multi-source data fusion and large language model in each embodiment of the present application. For the detailed functional description of each module of the intelligent medical decision-making and intervention system based on multi-source data fusion and large language model, please refer to the description of the corresponding intelligent medical decision-making and intervention method based on multi-source data fusion and large language model shown in the previous text, which will not be repeated here.

[0115] Based on the same principles as the methods described in the embodiments of this application, embodiments of this application also provide an electronic device, which may include, but is not limited to, a processor and a memory; the memory is configured to store a computer program; and the processor is configured to execute, by invoking the computer program, the intelligent medical decision-making and intervention method based on a multi-source data fusion large language model described in any optional embodiment of this application. Compared to the prior art, the intelligent medical decision-making and intervention method based on a multi-source data fusion large language model provided in this application collects medical conversation data from real conversation data from multiple medical centers, and fine-tunes a pre-trained large language model based on the medical conversation data to obtain a large language model for medical conversation scenarios. The large language model for medical conversation scenarios analyzes real-time medical conversation data, generates medical advice content, and adjusts the tone and frequency of the advice content based on the patient's condition and emotional state. This method significantly improves the large language model's capabilities in semantic understanding, natural interaction, and regional adaptation. It has significant technical advantages and application value in the field of intelligent medical guidance, providing a more efficient, accurate, and personalized solution for communication between patients and medical staff, alleviating communication barriers between patients and medical staff, and providing strong technical support for the intelligentization of medical services.

[0116] In an optional embodiment, an electronic device is also provided, such as Figure 5 As shown, Figure 5 The electronic device 500 shown may be a server, including a processor 501 and a memory 503. The processor 501 and the memory 503 are connected, for example, via a bus 502. Optionally, the electronic device 500 may further include a transceiver 504. It should be noted that in actual applications, the number of transceivers 504 is not limited to one, and the structure of the electronic device 500 does not constitute a limitation on the embodiments of the present application.

[0117] The processor 501 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor 501 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0118] The bus 502 may include a path for transmitting information between the above components. The bus 502 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 502 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0119] The memory 503 may be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0120] The memory 503 is used to store application code for executing the solution of the present application, and the execution is controlled by the processor 501. The processor 501 is used to execute the application code stored in the memory 503 to implement the content shown in the above method embodiment.

[0121] Among them, electronic devices include but are not limited to: mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0122] The server provided in this application can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to these. The terminal and the server can be directly or indirectly connected through wired or wireless communication, and this application does not limit this.

[0123] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer-readable storage medium is run on a computer, the computer can execute the corresponding contents of the aforementioned method embodiment.

[0124] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0125] It should be noted that the computer-readable storage medium mentioned above in this application may also be a computer-readable signal medium or a combination of a computer-readable storage medium and a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0126] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0127] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0128] According to one aspect of the present application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to implement the intelligent medical decision-making and intervention method and system based on a multi-source data fusion large language model provided in the various optional implementations described above.

[0129] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0130] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0131] The modules described in the embodiments of this application may be implemented in software or hardware. The name of a module does not necessarily limit the module itself. For example, a data acquisition module may be described as a "data acquisition module for collecting medical conversation data from real conversation data from multiple medical centers."

[0132] The above description is merely a preferred embodiment of the present application and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this application is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but also encompasses other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. An intelligent medical decision-making and intervention method based on multi-source data fusion and large language model, characterized by: The method comprises: Collect medical conversation data from real conversation data from multiple medical centers; Fine-tuning the pre-trained large language model based on the medical conversation data to obtain a large language model for the medical conversation scenario; Real-time medical conversation data is analyzed based on the large language model in the medical conversation scenario to generate medical advice content, and the tone and frequency of the medical advice content are adjusted according to the patient's condition and emotional state.

2. The intelligent medical decision-making and intervention method based on multi-source data fusion and large language model according to claim 1 is characterized in that: The types of medical centers include general hospitals, specialized hospitals, and primary medical institutions; The real conversation data is real three-party conversation data including the speech of the doctor, the speech of the nurse and the speech of the patient; The medical conversation data includes audio conversation data, text conversation data converted from the audio conversation data, and corresponding classification labels.

3. The intelligent medical decision-making and intervention method based on multi-source data fusion and large language model according to claim 1 is characterized in that: Fine-tuning the pre-trained large language model based on the medical conversation data to obtain a large language model for the medical conversation scenario includes: Incorporating scenario feature knowledge into a pre-trained large language model, inputting the medical conversation data into the pre-trained large language model, and fine-tuning the pre-trained large language model to obtain a large language model optimized for medical conversation scenarios; The scenario feature knowledge includes general medical knowledge, hospital-specific knowledge and dynamically updated knowledge.

4. The intelligent medical decision-making and intervention method based on multi-source data fusion and large language model according to claim 1 or 3, characterized in that: When fine-tuning the pre-trained large language model, a distributed training method combined with a feedback mechanism is used to optimize the pre-trained large language model.

5. The intelligent medical decision-making and intervention method based on multi-source data fusion and large language model according to claim 1 is characterized in that: The method of analyzing real-time medical conversation data based on a large language model in a medical conversation scenario, generating medical advice content, and adjusting the tone and frequency of the medical advice content based on the patient's condition and emotional state includes: Obtain the patient's basic information, current health status and historical medical history, and display them in the doctor's window; If the doctor provides initial medical order content, the initial medical order content is adjusted according to the real-time medical conversation data to generate the medical order content; otherwise, the medical order content is generated according to the real-time medical conversation data.

6. The intelligent medical decision-making and intervention method based on multi-source data fusion and large language model according to claim 1 is characterized in that: After analyzing the real-time medical conversation data based on the large language model in the medical conversation scenario to generate medical advice content, and adjusting the tone and frequency of the medical advice content according to the patient's condition and emotional state, the method further includes: Push the medical advice content to the patient and adjust the frequency and intensity of the push according to the set rules.

7. The intelligent medical decision-making and intervention method based on multi-source data fusion and large language model according to claim 1 is characterized in that: After analyzing the real-time medical conversation data based on the large language model in the medical conversation scenario to generate medical advice content, and adjusting the tone and frequency of the medical advice content according to the patient's condition and emotional state, the method further includes: Optimize the large language model in medical dialogue scenarios based on patients' feedback on medical advice content, emotional changes, symptom changes, and treatment effects, and update the medical advice content.

8. An intelligent medical decision-making and intervention system based on multi-source data fusion and large language model, characterized by: The system comprises: A data collection module, used to collect medical conversation data from real conversation data of multiple medical centers; A model optimization module, configured to fine-tune the pre-trained large language model based on the medical conversation data to obtain a large language model for the medical conversation scenario; The medical order generation module is used to analyze real-time medical conversation data based on the large language model in the medical conversation scenario, generate medical order content, and adjust the tone and frequency of the medical order content according to the patient's condition and emotional state.

9. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images