Intelligent medical information interaction system based on artificial intelligence

Abstract

The invention discloses an intelligent medical information interaction system based on artificial intelligence. The system comprises a multi-mode sensing unit, an AI interaction decision engine, an intelligent interaction terminal and a knowledge self-evolution unit. And the multi-modal sensing unit is used for performing multi-modal acquisition on various data of the body of the patient and storing the data into the intelligent medical information interaction system. The AI interactive decision engine can pre-process various data of the body of the patient acquired by the multi-modal sensing unit, and generate a preliminary diagnosis and treatment scheme based on fusion of the various data of the body of the patient and a medical knowledge graph. The intelligent interaction terminal is used for outputting data of the multi-mode sensing unit and the AI interaction decision engine and can be used for conducting remote interaction with a patient, and medical staff determine subsequent diagnosis and treatment decisions according to the output data of the multi-mode sensing unit and the AI interaction decision engine and remote interaction of the patient. The knowledge evolution unit can extract updated data in an existing medical knowledge base and integrate the data into the intelligent medical information interaction system. The intelligent medical information interaction system comprehensively obtains patient data through multi-modal perception, an AI interaction decision engine quickly generates a preliminary scheme, an intelligent interaction terminal realizes efficient data output and remote interaction, and a knowledge self-evolution unit continuously updates a knowledge base, so that the diagnosis and treatment efficiency and accuracy can be improved, remote effective communication between doctors and patients is guaranteed, and the diagnosis and treatment efficiency is improved. And the system knowledge is advanced with the times.

Description

Technical Field

[0001] The present invention relates to the field of medical systems, and more specifically, to an intelligent medical information interaction system based on artificial intelligence. Background Art

[0002] In the traditional field of medical information technology, medical data collection, analysis, and decision support have long faced technical bottlenecks. Existing medical systems typically rely on single-modality data acquisition methods (such as manual entry of vital sign parameters or static imaging examinations), making it difficult to achieve multi-dimensional integration of patients' physiological and pathological information. This leads to severe data silos and limits the comprehensiveness and accuracy of clinical diagnosis and treatment decisions.

[0003] In addition, traditional medical assistance systems mostly use fixed rule engines or simple statistical models, lack the ability to dynamically model complex medical knowledge, and are unable to cope with individualized diagnosis and treatment needs and the nonlinear characteristics of disease evolution.

[0004] Inefficient interaction between patients and medical staff is another technical pain point in telemedicine. Traditional telemedicine systems offer only one-way data transmission or basic video consultations, but lack the ability to visualize and dynamically feedback medical data. This results in diagnostic and treatment decisions being highly dependent on the medical staff's experience and judgment, making it difficult to establish an efficient and accurate closed-loop medical process.

[0005] At the same time, there is a contradiction between the rapid iteration of medical knowledge and the static knowledge base of existing systems. Traditional systems lack adaptive learning mechanisms and cannot automatically absorb the latest clinical guidelines, research literature or case data, resulting in diagnosis and treatment plans lagging behind medical frontier advances.

[0006] Therefore, there is an urgent need for a new type of intelligent medical system that integrates multimodal data perception, intelligent decision-making reasoning, remote real-time interaction and dynamic knowledge evolution capabilities. Summary of the Invention

[0007] The present invention aims to solve at least one of the technical problems in the related art to a certain extent. To this end, the present invention aims to provide an intelligent medical information interaction system based on artificial intelligence.

[0008] To achieve the above objectives, an artificial intelligence-based smart medical information interaction system according to an embodiment of the present invention includes a multimodal perception unit, an AI interactive decision engine, an intelligent interactive terminal and a knowledge self-evolution unit.

[0009] The multimodal sensing unit is used to acquire various body data of the patient in a multimodal manner and store them in the smart medical information interaction system.

[0010] The AI ​​interactive decision engine can pre-process the various patient body data acquired by the multimodal perception unit, and generate a preliminary diagnosis and treatment plan based on the integration of the various patient body data and the medical knowledge graph.

[0011] The intelligent interactive terminal is used to output data from the multimodal perception unit and the AI ​​interactive decision-making engine, and can be used for remote interaction with patients. Medical staff determine subsequent diagnosis and treatment decisions based on the output data from the multimodal perception unit, the AI ​​interactive decision-making engine and the patient's remote interaction.

[0012] The knowledge evolution unit can extract updated data from the existing medical knowledge base and integrate it into the smart medical information interaction system.

[0013] In addition, the artificial intelligence-based smart medical information interaction system according to the above embodiment of the present invention may also have the following additional technical features:

[0014] According to one embodiment of the present invention, the multimodal sensing unit includes a wearable sensor device, an environmental sensing module and a medical imaging direct connection module.

[0015] The wearable sensor device is used to be worn on the patient to monitor physiological indicators such as body temperature, heart rate, blood oxygen, blood sugar, blood pressure and respiratory rate in real time.

[0016] The environmental sensing module is used to monitor the temperature, humidity, air quality and ultraviolet index in the medical environment.

[0017] The medical imaging direct connection module is used to connect to the hospital medical system, obtain the patient's medical imaging data, and automatically perform denoising, image registration, and abnormal area marking on the acquired medical imaging data.

[0018] According to one embodiment of the present invention, the multimodal sensing unit further includes a radar sensing module, which is used to perform non-contact respiratory rate, heartbeat detection and fall monitoring on the patient.

[0019] According to one embodiment of the present invention, the AI ​​interactive decision engine includes a hybrid expert system architecture, a dynamic routing module, a treatment plan simulation module and an uncertainty quantification module.

[0020] The hybrid expert system architecture can accurately diagnose and treat different pathological types.

[0021] The dynamic routing module can assign the patient to the corresponding pathology type in the hybrid expert system architecture based on the patient's pathology feature frame to perform accurate diagnosis and treatment.

[0022] The treatment plan simulation module can generate a corresponding treatment plan based on the diagnosis and treatment results of the hybrid expert system architecture.

[0023] The uncertainty quantification module is used to output the confidence interval and basis of the treatment plan generated by the treatment plan simulation module.

[0024] According to one embodiment of the present invention, the hybrid expert system architecture includes a medical natural language processing module and multiple sub-models.

[0025] The medical natural language processing module is used to parse medical text to convert unstructured medical record text into structured data.

[0026] Each of the sub-models corresponds to a pathological type specialty and is used to accurately treat the corresponding pathological type.

[0027] According to one embodiment of the present invention, the AI ​​interactive decision engine also includes an emotion perception module, which optimizes the interactive experience by analyzing the patient's voice and tone and recognizing text emotions.

[0028] According to one embodiment of the present invention, the intelligent interactive terminal includes a multimodal input device and a multimodal output device.

[0029] The multimodal input device is used for multimodal human-computer interaction.

[0030] The multimodal output device is used for performing multimodal data output.

[0031] According to one embodiment of the present invention, the multimodal input device includes a voice acquisition module, a gesture recognition module, a touch control module, and a biometric feature acquisition module.

[0032] The voice collection module is used to record the voice of doctors or patients.

[0033] The gesture recognition module is used to recognize the gesture characteristics of doctors or patients.

[0034] The touch module is used by doctors or patients to touch and control the intelligent interactive terminal.

[0035] The biometric feature collection module is used to collect the doctor's fingerprint, face and pupil features for security verification during the use of the smart interactive terminal.

[0036] According to one embodiment of the present invention, the multimodal output device includes a high-resolution display screen, a holographic projection device, and a speech synthesis module.

[0037] The high-resolution display screen is used to display the display interface of the intelligent interactive terminal in high definition.

[0038] The holographic projection device is used to perform 3D projection display of human body structure and pathological characteristics.

[0039] The speech synthesis module is used to simulate human speech and convert text information into natural and fluent speech output.

[0040] According to one embodiment of the present invention, the knowledge self-evolution unit includes a federated learning enhancement architecture, a model watermark tracking module and a dynamic knowledge distillation module.

[0041] The federated learning enhanced architecture can continuously update the medical knowledge graph protected by differential privacy.

[0042] The model watermark tracking module can embed invisible watermark information into the AI ​​model, providing the system with the ability to trace the model, protect copyright, and track malicious behavior.

[0043] The dynamic knowledge distillation module is used to migrate large model knowledge to edge devices to lightweight the model.

[0044] According to the artificial intelligence-based smart medical information interaction system provided by the embodiment of the present invention, the multimodal sensing unit can obtain and store various patient body data in a multimodal manner, providing a rich and comprehensive information basis for subsequent diagnosis and treatment. The AI ​​interactive decision engine can pre-process the acquired patient data and generate a preliminary diagnosis and treatment plan in combination with the medical knowledge graph, assisting medical staff in formulating diagnosis and treatment plans more quickly and improving diagnosis and treatment efficiency. The interactive terminal can output data from the multimodal sensing unit and the AI ​​interactive decision engine, and can also realize remote interaction with patients, making it convenient for medical staff to obtain patient feedback, thereby more accurately determining subsequent diagnosis and treatment decisions and improving the convenience of patients seeking medical treatment. The knowledge self-evolution unit can extract updated data from the existing medical knowledge base and integrate it into the system, ensuring that the system knowledge keeps pace with the times and continuously improving the system's diagnosis and treatment capabilities and accuracy.

[0045] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned by practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0047] Figure 1 1 is a schematic diagram of the overall structure of an embodiment of the present invention;

[0048] Figure 21 is a schematic diagram of the overall structure of a multimodal sensing unit in an embodiment of the present invention;

[0049] Figure 3 This is a schematic diagram of the overall structure of the AI ​​interactive decision-making engine in an embodiment of the present invention;

[0050] Figure 4 This is a schematic diagram of the overall structure of the intelligent interactive terminal in an embodiment of the present invention;

[0051] Figure 5 Schematic diagram of the overall structure of the knowledge self-evolution unit in an embodiment of the present invention.

[0052] Figure Number:

[0053] Multimodal perception unit 10;

[0054] Wearable sensor devices11;

[0055] Environmental perception module 12;

[0056] Medical imaging direct connection module 13;

[0057] Radar perception module 14;

[0058] AI interactive decision engine 20;

[0059] Hybrid expert system architecture 21;

[0060] Medical natural language processing module 211;

[0061] submodel 212;

[0062] Dynamic routing module 22;

[0063] Treatment plan simulation module 23;

[0064] Uncertainty quantification module 24;

[0065] Emotion perception module 25;

[0066] Intelligent interactive terminal 30;

[0067] Multimodal input device 31;

[0068] Voice collection module 311;

[0069] Gesture recognition module 312;

[0070] Touch module 313;

[0071] Biometrics collection module 314;

[0072] Multimodal output device 32;

[0073] High-resolution display 321;

[0074] Holographic projection device 322;

[0075] Speech synthesis module 323;

[0076] Knowledge Self-Evolution Unit 40;

[0077] Federated Learning Enhanced Architecture 41;

[0078] Model watermark tracking module 42;

[0079] Dynamic knowledge distillation module 43.

[0080] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0081] The following describes in detail embodiments of the present invention, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0082] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "circumferential", "radial", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore cannot be understood as limiting the present invention.

[0083] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature identified as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0084] In the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," "connect," "fixed," etc. should be understood broadly. For example, they may refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediary; or internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0085] In the present invention, unless otherwise expressly specified or limited, a first feature being "above" or "below" a second feature may include the first and second features being in direct contact, or may include the first and second features being in contact not directly but through another feature between them. Furthermore, a first feature being "above," "above," and "above" a second feature may include the first feature being directly above or obliquely above the second feature, or may simply mean that the first feature is higher in level than the second feature. A first feature being "below," "below," and "below" a second feature may include the first feature being directly below or obliquely below the second feature, or may simply mean that the first feature is lower in level than the second feature.

[0086] The following describes in detail an artificial intelligence-based smart medical information interaction system according to an embodiment of the present invention with reference to the accompanying drawings.

[0087] Reference Figures 1 to 5 As shown, an artificial intelligence-based smart medical information interaction system is provided according to an embodiment of the present invention.

[0088] To achieve the above objectives, an artificial intelligence-based smart medical information interaction system according to an embodiment of the present invention includes a multimodal perception unit 10, an AI interaction decision engine 20, an intelligent interaction terminal 30 and a knowledge self-evolution unit 40.

[0089] The multimodal sensing unit 10 is used to acquire various body data of the patient in a multimodal manner and store them in the smart medical information interaction system.

[0090] The AI ​​interactive decision engine 20 can pre-process the various body data of the patient acquired by the multimodal perception unit 10, and generate a preliminary diagnosis and treatment plan based on the integration of the various body data of the patient and the medical knowledge graph.

[0091] The intelligent interactive terminal 30 is used to output data from the multimodal perception unit 10 and the AI ​​interactive decision engine 20, and can be used for remote interaction with patients. Medical staff determine subsequent diagnosis and treatment decisions based on the output data from the multimodal perception unit 10, the AI ​​interactive decision engine 20 and the patient's remote interaction.

[0092] The knowledge evolution unit can extract updated data from the existing medical knowledge base and integrate it into the smart medical information interaction system.

[0093] Based on the above, the multimodal sensing unit 10 can acquire and store various patient body data in a multimodal manner, providing a rich and comprehensive information basis for subsequent diagnosis and treatment. The AI ​​interactive decision engine 20 can pre-process the acquired patient data and generate a preliminary diagnosis and treatment plan in combination with the medical knowledge graph, assisting medical staff in formulating diagnosis and treatment plans more quickly and improving diagnosis and treatment efficiency. The interactive terminal can output data from the multimodal sensing unit 10 and the AI ​​interactive decision engine 20, and can also realize remote interaction with patients, making it convenient for medical staff to obtain patient feedback, and then more accurately determine subsequent diagnosis and treatment decisions, thereby improving the convenience of patients seeking medical treatment. The knowledge self-evolution unit 40 can extract updated data from the existing medical knowledge base and integrate it into the system to ensure that the system knowledge keeps pace with the times and continuously improves the system's diagnosis and treatment capabilities and accuracy.

[0094] Preferably, in one embodiment of the present invention, the multimodal sensing unit 10 includes a wearable sensor device 11, an environmental sensing module 12 and a medical imaging direct connection module 13.

[0095] The wearable sensor device 11 is worn on the patient to monitor physiological indicators such as body temperature, heart rate, blood oxygen, blood sugar, blood pressure and respiratory rate in real time.

[0096] The environmental sensing module 12 is used to monitor the temperature, humidity, air quality and UV index in the medical environment. Advantageously, the environmental sensing module 12 can be integrated into the wearable sensor device.

[0097] The medical image direct connection module 13 is used to connect to the hospital medical system, obtain the patient's medical image data, and automatically perform denoising, image registration, and abnormal area marking on the obtained medical image data.

[0098] In this way, the wearable sensor device 11 can be worn by the patient in real time, so as to continuously and accurately monitor key physiological indicators such as body temperature, heart rate, blood oxygen, blood sugar, blood pressure and respiratory rate, and provide medical staff with timely and accurate health data, which helps to detect health problems early and take intervention measures. The environmental perception module 12 can monitor the temperature, humidity, air quality and ultraviolet index in the medical environment, which helps to create a more suitable and safe medical environment, protect the health of patients and medical staff, and is also conducive to the normal operation of medical equipment.

[0099] The medical imaging direct connection module 13 can not only connect to the hospital medical system to obtain patient medical imaging data, but also automatically perform denoising, image registration and abnormal area marking on the imaging data, greatly improving the efficiency and quality of image processing, providing clearer and more accurate imaging basis for subsequent diagnosis, and helping to more accurately judge the condition.

[0100] Advantageously, the multimodal sensing unit 10 can be used in various medical scenarios:

[0101] First, it can facilitate the management of chronic diseases, such as the daily monitoring of diabetic patients. Patients can wear wearable sensor devices11, such as smart bracelets and patch sensors, to monitor their blood sugar levels in real time. The devices automatically collect data at set intervals and transmit the data to the smart medical information interaction system. The devices can also monitor the patient's heart rate, number of steps taken, and other data, as exercise and heart rate changes can affect blood sugar fluctuations.

[0102] The environmental sensing module 12 monitors indoor temperature and humidity. Excessive temperature or humidity can affect the patient's absorption and metabolism of insulin, which in turn affects blood sugar control. Based on this environmental data and the patient's blood sugar monitoring results, the system provides recommendations for adjusting the patient's living environment or medication regimen.

[0103] Secondly, it can also be beneficial for monitoring postoperative recovery. Patients wear wearable sensor devices 11 after surgery to monitor physiological indicators such as body temperature, heart rate, blood oxygen, blood pressure and respiratory rate in real time. For example, after abdominal surgery, fluctuations in the patient's body temperature may reflect wound infection, while changes in heart rate and blood oxygen can reflect the body's recovery state and cardiopulmonary function. Medical staff can view these data at any time through the system, detect abnormalities in time and take corresponding measures. The environmental perception module 12 monitors the temperature, humidity and air quality of the ward. Appropriate temperature and humidity help patients' wounds heal and their bodies recover, while good air quality can reduce the risk of infection in patients. If the temperature in the ward is too high or the humidity is too high, the system will remind medical staff to adjust equipment such as air conditioners or humidifiers; if the air quality is poor, it will recommend increasing ventilation or using air purification equipment.

[0104] Of course, the above examples are only some of the examples of the multimodal sensing unit 10, which can also be used in other medical scenarios in actual applications, so no further examples are given here.

[0105] Preferably, in one embodiment of the present invention, the multimodal sensing unit 10 further includes a radar sensing module 14, and the radar sensing module 14 is used to perform non-contact respiratory rate, heartbeat detection and fall monitoring on the patient.

[0106] In this way, the radar sensing module 14 can monitor patients' falls in real time. Once a fall is detected, it can immediately send an alert to medical staff or family members. This is particularly important for elderly people living alone and patients with mobility impairments, as they can receive quick assistance after a fall, reducing secondary injuries and adverse consequences caused by falls and improving patient safety. In rehabilitation institutions or nursing homes, the fall detection function of the radar sensing module 14 allows medical staff to keep abreast of patients' movements, rationally allocate nursing resources, and improve the efficiency and quality of care.

[0107] At the same time, by combining the radar sensing module 14 and the wearable sensor device 11, real-time and comprehensive transmission and analysis of data can be achieved, providing doctors with more timely and accurate decision-making basis, further improving medical efficiency and optimizing medical processes.

[0108] Preferably, in one embodiment of the present invention, the AI ​​interactive decision engine 20 includes a hybrid expert system architecture 21, a dynamic routing module 22, a treatment plan simulation module 23 and an uncertainty quantification module 24.

[0109] The hybrid expert system architecture can accurately diagnose and treat different pathological types.

[0110] The dynamic routing module 22 can assign the patient to the corresponding pathology type in the hybrid expert system architecture 21 according to the pathology feature frame for accurate diagnosis and treatment.

[0111] The treatment plan simulation module 23 can generate a corresponding treatment plan based on the diagnosis and treatment results of the hybrid expert system architecture 21.

[0112] The uncertainty quantification module 24 is used to output the confidence interval and basis of the treatment plan generated by the treatment plan simulation module 23.

[0113] It is important to understand that the Mixture of Experts (MoE) architecture21 is a deep learning model architecture that aims to improve model performance and efficiency by breaking down complex tasks into multiple subtasks and having different "expert" models (sub-networks) handle these subtasks separately. Its core concept is "divide and conquer," that is, each expert focuses on a specific domain or task, while a gating network dynamically distributes input data to the most appropriate expert.

[0114] In this way, this architecture has the ability to accurately diagnose and treat different pathological types, deeply analyze various pathological characteristics and patterns, and combine the medical knowledge base with a large amount of clinical data to provide patients with different pathological types with personalized diagnosis and treatment plans that conform to their pathological characteristics, thereby improving the accuracy of disease diagnosis and targeted treatment, and reducing the risk of misdiagnosis and mistreatment.

[0115] The dynamic routing module 22 accurately assigns patients to the corresponding pathology type module in the hybrid expert system architecture 21 based on their pathological characteristics. This intelligent allocation mechanism avoids the errors and delays that may occur with manual allocation, ensuring that patients receive the most accurate diagnosis and treatment for their pathology type in the shortest possible time, thereby improving the efficiency and quality of diagnosis and treatment.

[0116] Simultaneously, the treatment plan simulation module 23 generates treatment plans based on the diagnosis and treatment results of the hybrid expert system architecture 21. It comprehensively considers multiple factors, including the patient's pathology, physical condition, and treatment goals. Through simulation and optimization algorithms, it generates scientific, reasonable, and feasible treatment plans. The uncertainty quantification module 24 outputs confidence intervals and evidence for the treatment plans generated by the treatment plan simulation module 23, providing a quantitative reference for physicians' decision-making. Physicians can assess the reliability and effectiveness of the plans based on the confidence intervals and, combined with the evidence, gain a deeper understanding of the plans' strengths, weaknesses, and scope of application. This allows them to make more scientific and reasonable decisions based on a comprehensive consideration of various factors, thereby improving the safety and success rate of treatment.

[0117] Preferably, in one embodiment of the present invention, the hybrid expert system architecture 21 includes a medical natural language processing module 211 and multiple sub-models 212 .

[0118] The medical natural language processing module 211 is used to parse medical text to convert unstructured medical record text into structured data.

[0119] Each of the sub-models 212 corresponds to a pathological type specialty, and is used to accurately process the corresponding pathological type.

[0120] Thus, thanks to its powerful text parsing capabilities, the Medical Natural Language Processing Module 211 is capable of in-depth processing of massive amounts of unstructured medical record text. In actual medical scenarios, medical records are often recorded in natural language and contain a large amount of descriptive and subjective information. This module can convert this complex unstructured data into structured data, making medical record information easier to store, query, and analyze. This provides a clear and standardized data foundation for subsequent medical decision-making, improving data processing efficiency and quality.

[0121] At the same time, by converting unstructured medical record text into structured data, it achieves the integration and standardization of medical record information from different sources and formats. Medical records from different hospitals and doctors may have different formats and presentations. After structuring, the data can be presented in a unified format, facilitating data sharing and analysis across hospitals and departments. This helps build large-scale medical data sets, providing richer data support for medical research and clinical decision-making.

[0122] Furthermore, since each sub-model 212 corresponds to a specific pathology, this specialized design allows each sub-model 212 to focus on the research and treatment of a specific pathology. Different pathologies have distinct etiologies, pathological mechanisms, and clinical manifestations. Specialized sub-models 212 can perform deep learning and optimization based on the characteristics of a specific pathology, improving diagnostic accuracy and targeted treatment plans for that pathology, thereby providing patients with more precise medical services.

[0123] Preferably, in one embodiment of the present invention, the AI ​​interactive decision engine 20 further includes an emotion perception module 25, which optimizes the interactive experience by analyzing the patient's voice and tone and recognizing text emotions.

[0124] In this way, the emotion perception module 25 can accurately perceive the patient's emotional state during the communication process, such as anxiety, tension, frustration, or joy, by analyzing the patient's voice and tone and recognizing textual emotions. Based on this emotional information, the AI ​​interaction decision engine 20 can dynamically adjust the interaction method and content to provide more personalized and emotionally appropriate responses. For example, when it senses that the patient is anxious, the engine can adopt a gentler, more comforting tone to communicate and provide emotional support to the patient, thereby greatly improving the patient's satisfaction and comfort in interacting with the AI ​​system.

[0125] Preferably, in one embodiment of the present invention, the intelligent interactive terminal 30 includes a multimodal input device 31 and a multimodal output device 32 .

[0126] The multimodal input device 31 is used for multimodal human-computer interaction.

[0127] The multimodal output device 32 is used for multimodal data output.

[0128] In this way, the multimodal input device 31 supports multiple interaction methods, such as voice, gestures, touch, text input, etc. Users can freely choose the most convenient interaction method according to their own habits, environment and specific needs. For example, when both hands are busy or it is inconvenient to operate the screen, users can interact through voice commands; when precise information input is required, text input can be selected, which greatly improves the flexibility and convenience of human-computer interaction. Different interaction methods are suitable for different scenarios and tasks. The multimodal input device 31 allows users to combine multiple interaction methods according to specific circumstances, so as to complete interactive operations more efficiently. For example, when browsing medical information, users can first quickly locate the general content through voice, and then combine touch operations for detailed viewing and operation, which greatly shortens the time for information acquisition and processing and improves interaction efficiency.

[0129] The multimodal output device 32 can output data in a variety of forms, such as voice broadcast, image display, video presentation, and text presentation. This multimodal output method can convey information more comprehensively and accurately, meeting the information acquisition and understanding needs of different users. For example, when explaining a complex medical plan to a patient, in addition to textual explanations, images of the surgical procedure and videos demonstrating rehabilitation training methods can be used, accompanied by voice explanations, to enable the patient to more intuitively and clearly understand the plan content.

[0130] Preferably, in one embodiment of the present invention, the multimodal input device 31 includes a voice acquisition module 311 , a gesture recognition module 312 , a touch control module 313 and a biometrics acquisition module 314 .

[0131] The voice collection module 311 is used to record the voice of the doctor or the patient.

[0132] The gesture recognition module 312 is used to recognize gesture features of a doctor or a patient.

[0133] The touch module 313 is used by doctors or patients to control the intelligent interactive terminal 30 by touching.

[0134] The biometric feature collection module 314 is used to collect the doctor's fingerprint, face and pupil features for security verification during the use of the intelligent interactive terminal 30.

[0135] Thus, the voice acquisition module 311 allows doctors or patients to input information by voice, which is extremely convenient when both hands are busy (such as when doctors are preparing for surgery or patients are unable to operate with their hands) or when they need to express ideas quickly. The gesture recognition module 312 allows users to complete specific operations without touching the device. For example, in a sterile environment, doctors can use gestures to switch between different medical imaging data. The touch module 313 provides intuitive operation methods, allowing users to interact directly by clicking and sliding on the screen. This coexistence of multiple interaction methods allows users to freely choose according to specific scenarios and their own needs, greatly improving the convenience of interaction.

[0136] The biometric collection module 314 collects the doctor's fingerprint, facial, and pupil features for security verification during use of the intelligent interactive terminal 30. These biometric features are unique and non-replicable, offering greater security than traditional password authentication methods. Only authorized doctors can use the intelligent interactive terminal 30 through biometric verification, effectively preventing unauthorized access and data leakage, protecting patient privacy and the security of medical data.

[0137] Preferably, in one embodiment of the present invention, the multimodal output device 32 includes a high-resolution display screen 321 , a holographic projection device 322 and a speech synthesis module 323 .

[0138] The high-resolution display screen 321 is used to display the display interface of the intelligent interactive terminal 30 in high definition.

[0139] The holographic projection device 322 is used to perform 3D projection display of human body structures and pathological features.

[0140] The speech synthesis module 323 is used to simulate human speech and convert text information into natural and fluent speech output.

[0141] In this way, the high-resolution display screen 321 can present various display interfaces of the intelligent interactive terminal 30 in high-definition image quality, clearly and accurately displaying medical data charts, patient medical records, and operating instructions. This helps doctors view and analyze information more accurately, avoiding misjudgments caused by blurry or distorted displays, and also helps patients clearly understand their condition and treatment plan.

[0142] Holographic projection device 322 can project 3D images of human anatomy and pathological features, transforming previously abstract, two-dimensional medical knowledge into intuitive, three-dimensional images. By observing 3D projections, doctors can gain a more comprehensive and in-depth understanding of the human body's internal structure, the location and morphology of lesions, and their relationship to surrounding tissues, providing a more accurate basis for disease diagnosis and treatment. For example, during surgical planning, doctors can use holographic projections to more clearly visualize the lesion site and develop a more appropriate surgical plan.

[0143] Correspondingly, medical knowledge is often technical and complex, making it difficult for patients to understand. Multimodal output device 32 can present medical information to patients in an easily understandable manner. High-resolution display screen 321 can display illustrated health education materials, holographic projection device 322 allows patients to visually visualize their internal pathological conditions, and speech synthesis module 323 can explain the condition and treatment plan in a friendly and understandable language. In this way, patients can better understand their physical condition and treatment process, enhancing their disease awareness and self-management abilities.

[0144] Preferably, in one embodiment of the present invention, the knowledge self-evolution unit 40 includes a federated learning enhancement architecture 41 , a model watermark tracking module 42 and a dynamic knowledge distillation module 43 .

[0145] The federated learning enhancement architecture 41 can continuously update the medical knowledge graph protected by differential privacy.

[0146] The model watermark tracking module 42 can embed invisible watermark information into the AI ​​model, providing the system with the ability to trace the model, protect copyright, and track malicious behavior.

[0147] The dynamic knowledge distillation module 43 is used to migrate large model knowledge to edge devices to lightweight the model.

[0148] It is understandable that Federated Learning (FL) is a distributed machine learning paradigm that aims to train models collaboratively through multiple clients (such as devices and institutions) without the need to centralize the original data to a central server. In this way, the architecture can continuously update the medical knowledge graph protected by differential privacy. Knowledge in the medical field is constantly updated and iterated, and new disease discoveries, improved treatments, etc. need to be integrated into the knowledge system in a timely manner. The Federated Learning Enhanced Architecture41 continuously enriches and improves the medical knowledge graph by aggregating local data from multiple medical institutions (under the premise of protecting privacy), enabling AI systems to obtain more comprehensive and accurate medical knowledge, thereby improving the accuracy of diagnosis and treatment. For example, when a hospital discovers a new treatment for a rare disease, through the Federated Learning Enhanced Architecture41, this method can be quickly integrated into the overall medical knowledge graph for reference and learning by other hospitals.

[0149] Furthermore, by embedding invisible watermark information into AI models, when any model issues arise or when tracing its origins is necessary, the model's creator, usage scenario, and other information can be quickly and accurately determined. In the healthcare field, the reliability of AI models is directly related to patient health and well-being. Model traceability can help regulators and medical institutions better manage models and ensure their legal and compliant use. For example, if an AI diagnostic model misdiagnoses, the watermark information can quickly identify the model's development team for troubleshooting and resolution. If the model is maliciously tampered with, attacked, or used for illicit purposes, the watermark information can help track the malicious activity and the individuals responsible. This helps maintain the security and stability of medical AI systems and protects the interests of patients. For example, if someone is found to have maliciously modified the parameters of an AI diagnostic model, resulting in inaccurate diagnostic results, the watermark information can be used to track the specific operator and take appropriate action.

[0150] Furthermore, the knowledge of large models is migrated to edge devices to lightweight the models. Edge devices typically have limited computing resources and storage capacity, making it difficult for large models to run efficiently on edge devices. The dynamic knowledge distillation module 43 uses knowledge migration technology to extract important knowledge and features from large models and construct lightweight models. This enables edge devices to run AI models with limited resources and achieve real-time processing and analysis of medical data. For example, on wearable medical devices, lightweight models can monitor patients' vital signs in real time and issue timely warnings.

[0151] Advantageously, lightweight models run faster on edge devices, significantly improving system response speed. In scenarios like medical emergencies, time is of the essence, and rapid response and processing capabilities are crucial. The dynamic knowledge distillation module 43 enables edge devices to analyze and diagnose patients' conditions in a timely manner, buying valuable time for treatment.

[0152] Furthermore, because the model runs locally on the edge device, it reduces reliance on cloud servers and minimizes the impact of network latency and failures on the system. Even in the event of network instability or cloud server failure, the edge device can still rely on the local lightweight model for basic medical data processing and analysis, ensuring system stability and reliability.

[0153] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0154] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. All equivalent structural transformations made by using the contents of the present invention description and drawings under the inventive concept of the present invention, or direct / indirect application in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. An intelligent medical information interaction system based on artificial intelligence, characterized in that: include: A multimodal sensing unit, which is used to acquire various body data of the patient in a multimodal manner and store them in the smart medical information interaction system; An AI interactive decision-making engine, which can pre-process the various patient body data acquired by the multimodal sensing unit and generate a preliminary diagnosis and treatment plan based on the various patient body data and the integration of medical knowledge graphs; An intelligent interactive terminal, which is used to output data from the multimodal sensing unit and the AI ​​interactive decision-making engine and can be used to remotely interact with patients. Medical staff determine subsequent diagnosis and treatment decisions based on the output data from the multimodal sensing unit, the AI ​​interactive decision-making engine, and the patient's remote interaction; The knowledge self-evolution unit can extract updated data from the existing medical knowledge base and integrate it into the intelligent medical information interaction system.

2. The artificial intelligence-based intelligent medical information interaction system according to claim 1, characterized in that: The multimodal sensing unit includes: Wearable sensor devices, which are worn on patients to monitor physiological indicators such as body temperature, heart rate, blood oxygen, blood sugar, blood pressure, and respiratory rate in real time; An environmental sensing module, which is used to monitor the temperature, humidity, air quality, and UV index in the patient's medical environment; Medical imaging direct connection module, the medical imaging direct connection module is used to connect to the hospital medical system, obtain the patient's medical imaging data, and automatically denoise, image registration and abnormal area marking of the acquired medical imaging data.

3. The artificial intelligence-based intelligent medical information interaction system according to claim 2, characterized in that: The multimodal sensing unit also includes a radar sensing module, which is used to perform non-contact respiratory rate, heartbeat detection and fall monitoring on the patient.

4. The artificial intelligence-based intelligent medical information interaction system according to claim 1, characterized in that: The AI ​​interactive decision engine includes: A hybrid expert system architecture that enables accurate diagnosis and treatment of different pathological types; A dynamic routing module, which can assign patients to corresponding pathology types in the hybrid expert system architecture based on their pathological feature framework to enable accurate diagnosis and treatment; A treatment plan simulation module, which can generate a corresponding treatment plan based on the diagnosis and treatment results of the hybrid expert system architecture; An uncertainty quantification module is used to output the confidence interval and basis of the treatment plan generated by the treatment plan simulation module.

5. The artificial intelligence-based intelligent medical information interaction system according to claim 4 is characterized in that: The hybrid expert system architecture includes: A medical natural language processing module, which is used to parse medical text to convert unstructured medical record text into structured data; Multiple sub-models, each of which corresponds to a pathological type specialty, are used to accurately process the corresponding pathological type.

6. The artificial intelligence-based intelligent medical information interaction system according to claim 4 is characterized in that: The AI ​​interactive decision engine also includes an emotion perception module, which optimizes the interactive experience by analyzing the patient's voice and tone and recognizing text emotions.

7. The artificial intelligence-based intelligent medical information interaction system according to claim 1, characterized in that: The intelligent interactive terminal includes: A multimodal input device, wherein the multimodal input device is used for multimodal human-computer interaction; A multimodal output device is used for performing multimodal data output.

8. The artificial intelligence-based intelligent medical information interaction system according to claim 7, characterized in that: The multimodal input device comprises: Voice collection module, used to record the voice of doctors or patients; A gesture recognition module, used to identify the gesture characteristics of doctors or patients; A touch module, used by doctors or patients to control the intelligent interactive terminal by touching; The biometric feature collection module is used to collect the doctor's fingerprint, face and pupil features for security verification during the use of the smart interactive terminal.

9. The artificial intelligence-based intelligent medical information interaction system according to claim 7, characterized in that: The multimodal output device comprises: A high-resolution display screen for displaying the display interface of the intelligent interactive terminal in high definition; Holographic projection device, used to display the human body structure and pathological characteristics in 3D projection; The speech synthesis module is used to simulate human speech and convert text information into natural and fluent speech output.

10. The artificial intelligence-based intelligent medical information interaction system according to claim 1, characterized in that: The knowledge self-evolution unit includes: A federated learning-enhanced architecture that continuously updates a differentially private medical knowledge graph; The model watermark tracking module can embed invisible watermark information into the AI ​​model and provide the system with model traceability, copyright protection, and malicious behavior tracking capabilities; A dynamic knowledge distillation module is used to migrate large model knowledge to edge devices to lightweight the model.

Images