Multi-modal man-machine cooperation airway evaluation method and system based on AI

Abstract

The invention provides a multi-mode man-machine cooperation airway evaluation method and system based on AI, and relates to the technical field of medical data processing. The method comprises the following steps: collecting multi-modal medical record data of at least one patient; the multi-modal medical record data comprises medical text data, medical audio data, medical image data, patient facial expression data, patient physiological data and patient behavior data; a preset AI airway analysis model is used, airway threat information influencing airway safety of the patient is obtained through analysis on the basis of the multi-modal medical record data, and an airway diagnosis and treatment scheme matched with the patient is obtained through evaluation; wherein the airway threat information comprises airway physiological threat information and airway psychological threat information; the airway diagnosis and treatment scheme comprises an airway physiological diagnosis and treatment scheme and an airway psychological diagnosis and treatment scheme. The physiological and psychological states of the patient can be evaluated, and while the airway physiological diagnosis and treatment scheme is provided, the matched airway psychological diagnosis and treatment scheme can be provided, so that the patient is helped to overcome psychological obstacles in the airway diagnosis and treatment process.

Description

Technical Field

[0001] The present invention relates to the field of medical data processing technology, and in particular to an AI-based multimodal human-machine collaborative airway assessment method. Background Art

[0002] Along with the actual medical treatment process of patients, a large amount of multimodal medical record data will be generated. These multimodal medical record data are various types of recorded medical data from different sources, forms and time points, including structured data (such as laboratory test results), semi-structured data (such as text content in electronic health records EHR) and unstructured data (such as medical images, pathological sections, genome sequences). The multimodal medical record data can comprehensively reflect the health status of patients, but its complexity makes it difficult to evaluate and diagnose efficiently and accurately by manual processing alone. Therefore, data processing operations based on the human-computer collaborative mode of multimodal medical record data came into being.

[0003] In the Human-AI Collaboration model, artificial intelligence (AI) and medical personnel can use their unique advantages to jointly participate in diagnosis, treatment plan formulation, treatment process monitoring and patient management. Among them, AI is mainly used for data processing, pattern recognition and prediction, while doctors interpret and verify the analysis results provided by AI through their own clinical experience and expertise, thereby improving the quality and efficiency of overall medical decision-making.

[0004] In the process of data processing, data is often processed only from a professional perspective, while the psychological state of the patient reflected in these multimodal medical record data is ignored. As a result, the medical technology solutions provided and implemented later cannot bring good diagnosis and treatment results. Therefore, it is necessary to focus on the changes in the patient's psychological state during the analysis and processing of data, so as to improve the patient's psychological state during the diagnosis and treatment process.

[0005] Based on this, the present invention provides an AI-based multimodal human-machine collaborative airway assessment method and system to solve the problem of poor treatment effect caused by lack of consideration of the patient's psychological state, which is a technical problem that needs to be solved urgently. Summary of the invention

[0006] The purpose of the present invention is to overcome the shortcomings of the prior art and provide an AI-based multimodal human-machine collaborative airway assessment method and system. The present invention can assess the patient's physiological and psychological state, and while providing airway physiological diagnosis and treatment plans, it can also provide a matching airway psychological diagnosis and treatment plan to help patients overcome psychological barriers during the airway diagnosis and treatment process.

[0007] In order to solve the existing technical problems, the present invention provides the following technical solutions: An AI-based multimodal human-machine collaborative airway assessment method, including: Collecting multimodal medical record data of at least one patient; the multimodal medical record data includes medical text data, medical audio data, medical image data, patient facial expression data, patient physiological data and patient behavior data; Using a preset AI airway analysis model, airway threat information that affects the patient's airway safety is obtained based on the aforementioned multimodal medical record data analysis, and an airway diagnosis and treatment plan suitable for the patient is obtained through evaluation; wherein, the airway threat information includes airway physiological threat information and airway psychological threat information; the airway diagnosis and treatment plan includes an airway physiological diagnosis and treatment plan and an airway psychological diagnosis and treatment plan.

[0008] Further, the medical text data includes at least one of electronic medical record data, doctor's note data, examination report data, nursing record data, patient feedback data, psychological assessment questionnaire data, treatment record data, and self-report questionnaire data; The medical audio data includes recorded conversation data between a doctor and a patient, and at least one of heart sound data and lung sound data recorded by a stethoscope; wherein the recorded conversation data includes at least one of consultation conversation data between a doctor and a patient, communication data between medical personnel, telemedicine conversation data, patient follow-up record data, and psychological consultation record data; the heart sound data includes at least one of heart sound recording data, heart sound feature analysis data, heart sound variability data, and heart sound clinical record data; The medical imaging data includes imaging data obtained by at least one of X-ray, computerized tomography, magnetic resonance imaging, ultrasonography, positron emission tomography and nuclear medicine; The patient's facial expression data refers to a visual image of the patient's face that can be captured by an electronic device with a camera shooting function; the electronic device includes at least one of a smart phone, a tablet computer, a digital camera, a camera monitoring system, and a 3D facial scanner; The patient's physiological data includes at least one of heart rate variability, blood pressure, respiratory rate, body temperature and biochemical indicators; The patient behavior data includes at least one of the patient's daily activity data, sleep quality data, and social interaction data.

[0009] Furthermore, the AI ​​airway analysis model includes an AI airway physiological analysis sub-model and an AI patient psychological analysis sub-model; wherein, the AI ​​airway physiological analysis sub-model determines the patient's airway physiological state by analyzing the aforementioned multimodal medical record data; and the AI ​​patient psychological analysis sub-model determines the patient's psychological state by analyzing the aforementioned multimodal medical record data.

[0010] Furthermore, when using the preset AI airway analysis model, it specifically includes: Performing data preprocessing on the collected multimodal medical record data; The aforementioned AI airway physiological analysis sub-model and AI patient psychological analysis sub-model are respectively executed; wherein, The AI ​​airway physiological analysis sub-model uses the aforementioned multimodal medical record data as input data, and obtains physiological output results through data processing; the physiological output results are airway physiological threat information and corresponding matching airway physiological diagnosis and treatment plans; The AI ​​patient psychological analysis sub-model takes the aforementioned multimodal medical record data and the aforementioned physiological output results as input data, and obtains psychological output results through data processing; the psychological output results are airway psychological threat information and corresponding matching airway psychological diagnosis and treatment plans.

[0011] Further, the analysis includes physiological analysis operation and psychological analysis operation; wherein, The physiological analysis operation includes: extracting multimodal medical physiological characteristics corresponding to the aforementioned multimodal medical record data; the multimodal medical physiological characteristics include key time series characteristics, statistical characteristics and physiological signal characteristics of the patient; based on the aforementioned AI airway physiological analysis sub-model, performing data fusion on the aforementioned multimodal medical physiological characteristics to obtain the aforementioned airway physiological threat information affecting the patient's airway safety; The psychological analysis operation includes: extracting multimodal medical psychological characteristics corresponding to the aforementioned multimodal medical record data; the multimodal medical psychological characteristics include multimodal medical physiological characteristics, language characteristics, behavioral characteristics, cognitive characteristics and social characteristics that reflect the patient's psychological state; based on the aforementioned AI patient psychological analysis sub-model, the aforementioned multimodal medical psychological characteristics are data-fused to obtain the aforementioned airway psychological threat information that affects the patient's airway safety.

[0012] Furthermore, the airway psychological diagnosis and treatment plan is divided into a first airway psychological diagnosis and treatment plan, a second airway psychological diagnosis and treatment plan, a third airway psychological diagnosis and treatment plan and a fourth airway psychological diagnosis and treatment plan according to different execution sources; the execution sources corresponding to the first airway psychological diagnosis and treatment plan to the fourth airway psychological diagnosis and treatment plan are the patient himself, medical staff, relatives and friends of the patient, and social organizations, respectively.

[0013] Furthermore, the evaluation also includes: For the first to fourth airway psychological diagnosis and treatment plans, weights are set according to the corresponding importance in the airway psychological diagnosis and treatment plans; Implement the aforementioned airway physiological diagnosis and treatment plan and airway psychological diagnosis and treatment plan; Corresponding to the aforementioned airway physiological diagnosis and treatment scheme and airway psychological diagnosis and treatment scheme, airway physiological diagnosis and treatment evaluation values ​​and airway psychological diagnosis and treatment evaluation values ​​are provided respectively; wherein the airway psychological diagnosis and treatment evaluation values ​​include the first airway psychological diagnosis and treatment evaluation value, the second airway psychological diagnosis and treatment evaluation value, the third airway psychological diagnosis and treatment evaluation value and the fourth airway psychological diagnosis and treatment evaluation value corresponding to the aforementioned first airway psychological diagnosis and treatment scheme to the fourth airway psychological diagnosis and treatment scheme; Obtaining execution feedback information of the aforementioned airway physiological diagnosis and treatment plan and airway psychological diagnosis and treatment plan according to the aforementioned airway physiological diagnosis and treatment evaluation value and airway psychological diagnosis and treatment evaluation value; According to the aforementioned execution feedback information, the weights of the first to fourth airway psychological diagnosis and treatment plans in the aforementioned airway psychological diagnosis and treatment plans are adjusted, and the adjusted airway psychological diagnosis and treatment plans are executed.

[0014] An AI-based multimodal human-machine collaborative airway assessment device, comprising: An information collection unit, used to collect multimodal medical record data of at least one patient; the multimodal medical record data includes medical text data, medical audio data, medical image data, patient facial expression data, patient physiological data and patient behavior data; An information evaluation unit is used to use a preset AI airway analysis model to obtain airway threat information that affects the patient's airway safety based on the aforementioned multimodal medical record data analysis, and obtain an airway diagnosis and treatment plan suitable for the patient through evaluation; wherein the airway threat information includes airway physiological threat information and airway psychological threat information; and the airway diagnosis and treatment plan includes an airway physiological diagnosis and treatment plan and an airway psychological diagnosis and treatment plan.

[0015] An AI-based multimodal human-machine collaborative airway assessment system, including: Network nodes for sending and receiving multimodal medical record data; An information processing module, used to process the aforementioned multimodal medical record data based on a preset AI airway analysis model; A system server, the system server connecting the network nodes and the information processing module; The system server is configured to: collect multimodal medical record data of at least one patient; the multimodal medical record data includes medical text data, medical audio data, medical image data, patient facial expression data, patient physiological data and patient behavior data; use a preset AI airway analysis model to obtain airway threat information that affects the patient's airway safety based on the aforementioned multimodal medical record data analysis, and obtain an airway diagnosis and treatment plan suitable for the patient through evaluation; wherein the airway threat information includes airway physiological threat information and airway psychological threat information; the airway diagnosis and treatment plan includes an airway physiological diagnosis and treatment plan and an airway psychological diagnosis and treatment plan.

[0016] A computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the implementation steps of any of the above methods.

[0017] Based on the above advantages and positive effects, the advantages of the present invention are: evaluating the physiological and psychological state of the patient, and while providing an airway physiological diagnosis and treatment plan, it can also provide a matching airway psychological diagnosis and treatment plan to help patients overcome psychological barriers in the airway diagnosis and treatment process, thereby helping patients to cooperate with the implementation of the aforementioned airway diagnosis and treatment plan and improve the implementation effectiveness of the airway diagnosis and treatment plan. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flow chart of a method provided by an embodiment of the present invention.

[0019] Figure 2 A schematic diagram of the structure of a device provided in an embodiment of the present invention.

[0020] Figure 3 A schematic diagram of the structure of a system provided in an embodiment of the present invention.

[0021] Description of reference numerals: Device 200, information collection unit 201, information evaluation unit 202; System 300 , network node 301 , information processing module 302 , system server 303 . DETAILED DESCRIPTION

[0022] The following is a further detailed description of an AI-based multimodal human-machine collaborative airway assessment method and system disclosed in the present invention in conjunction with the accompanying drawings and specific embodiments. It should be noted that the technical features or combinations of technical features described in the following embodiments should not be considered isolated, and they can be combined with each other to achieve better technical effects. In the drawings of the following embodiments, the same reference numerals appearing in each drawing represent the same features or components, which can be applied to different embodiments. Therefore, once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0023] It should be noted that the structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification for people familiar with this technology to understand and read, and are not used to limit the limiting conditions for the implementation of the invention. Any modification of the structure, change of the proportion relationship or adjustment of the size should fall within the scope of the technical content disclosed by the invention without affecting the effects and purposes that can be achieved by the invention. The scope of the preferred embodiments of the present invention includes other implementations, in which the functions can be performed in a substantially simultaneous manner or in a reverse order according to the functions involved, which should be understood by those skilled in the art of the technical field to which the embodiments of the present invention belong.

[0024] Techniques, methods, and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, the techniques, methods, and devices should be considered part of the authorization specification. In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values. Example

[0025] See also Figure 1 FIG. 1 is a flow chart of the present invention. The implementation step S100 of the method is as follows: S101, collecting multimodal medical record data of at least one patient.

[0026] In this embodiment, the multimodal medical record data refers to recorded medical data from different sources and types.

[0027] Specifically, the multimodal medical record data includes medical text data, medical audio data, medical image data, patient facial expression data, patient physiological data and patient behavior data.

[0028] The medical text data includes but is not limited to at least one of electronic medical record data, doctor's note data, examination report data, nursing record data, patient feedback data, psychological assessment questionnaire data, treatment record data and self-report questionnaire data.

[0029] The electronic medical record data includes the patient's basic information, medical history, diagnosis and treatment records, etc.

[0030] The physician's note data includes observations, assessments, and treatment plans recorded by the physician during the consultation.

[0031] The examination report data includes text records of laboratory test results, imaging examination results, etc.

[0032] The nursing record data includes the nurse's observation of the patient's condition and the record of nursing measures.

[0033] The patient feedback data includes the patient's feedback information on the treatment process and effect.

[0034] The psychological assessment questionnaire data refers to the filling results of various standardized mental health assessment scales.

[0035] The treatment record data includes records of various types of treatments received by the patient (such as drug treatment, psychotherapy, etc.).

[0036] The self-report questionnaire data included questionnaires filled out by patients regarding their health status, quality of life, and psychological status.

[0037] The above medical text data provides medical professionals with rich information, which helps in clinical decision-making, formulation of treatment plans and patient management, while also observing the patient's psychological condition and comforting the patient to reduce the patient's psychological pressure.

[0038] The medical audio data includes recorded data of conversations between doctors and patients, and at least one of heart sound data and lung sound data recorded by a stethoscope.

[0039] The conversation recording data includes but is not limited to at least one of consultation conversation data between doctors and patients, communication data between medical staff, telemedicine conversation data, patient follow-up record data and psychological consultation recording data.

[0040] The consultation dialogue data between the doctor and the patient refers to the diagnosis data, inquiry data and treatment suggestion data during the consultation process.

[0041] The communication data between medical personnel refers to the discussion data about the patient's condition between doctors, nurses and other medical personnel.

[0042] The telemedicine conversation data refers to remote consultation data and treatment data conducted via telephone or video conferencing.

[0043] The patient follow-up record data refers to the return communication data between the patient and the doctor after treatment.

[0044] The psychological counseling recording data refers to the conversations with patients during the psychotherapy process, which can assess emotions and psychological states.

[0045] The above conversation recording data provides valuable information for medical staff, which can help them better understand the patient's health status, physiological needs and psychological needs.

[0046] The heart sound data refers to heart audio collected through a stethoscope, which can help assess heart health.

[0047] The heart sound data includes but is not limited to at least one of heart sound recording data, heart sound feature analysis data, heart sound variability data and heart sound clinical record data.

[0048] The heart sound recording data refers to the heart sound data recorded by a stethoscope or an electronic device, including the sounds of the heart's contraction (first heart sound) and relaxation (second heart sound).

[0049] The heart sound characteristic analysis data includes quantitative analysis data of characteristics of the heart sound, such as frequency, intensity, duration and waveform.

[0050] The heart sound variability data is used to analyze the change pattern of heart sounds, which may involve the time intervals and regularities between different heart sounds.

[0051] The heart sound clinical record data can reflect the doctor's observation record of the heart sounds and describe possible abnormalities, such as murmurs, gallop rhythm, etc.

[0052] The lung sound data refers to lung sounds captured by a stethoscope and is used to assess respiratory health, such as asthma, pneumonia, etc.

[0053] In this embodiment, it is considered that the tone, speech speed, volume and emotional expression of the voice in the aforementioned conversation recording data can reflect the emotional state of the patient. For example, anxious or depressed patients may show different characteristics in their voice.

[0054] In addition, considering that the patient's breathing pattern is closely related to his or her emotions, although the heart sound data and lung sound data reflect the patient's physiological state, they cannot directly reflect the psychological state, but they can also indirectly provide some clues about the patient's psychological condition. For example, when anxious, rapid shallow breathing may occur, and the intensity of heart sounds will increase, or abnormal lung sounds (such as wheezing) may also be related to emotional state.

[0055] The medical imaging data refers to information about the internal structure and function of a patient's body obtained through various imaging techniques, which can be used for diagnosis, treatment planning and disease monitoring.

[0056] The medical imaging data includes but is not limited to imaging data obtained by at least one of X-ray, computer tomography, magnetic resonance imaging, ultrasound, positron emission tomography and nuclear medicine.

[0057] Since X-ray, computer tomography, magnetic resonance imaging, ultrasound, positron emission tomography and nuclear medicine are all existing technologies in this field, they will not be described in detail here.

[0058] The medical imaging data can help patients psychologically in the following ways: For one thing, medical imaging data can help confirm or rule out underlying physical illnesses, such as brain tumors or brain injuries, which may be associated with a patient's psychological symptoms, such as anxiety or depression.

[0059] Secondly, the examination results reflected by medical imaging data can reduce patients' anxiety about unknown causes of illness and provide a clearer assessment of their health status, thereby alleviating their psychological burden.

[0060] Thirdly, medical imaging data can be used to monitor disease progression or treatment effectiveness. When patients see actual improvements, it may enhance their confidence and enthusiasm for subsequent treatment and improve their mental state.

[0061] Fourthly, when certain physiological problems (such as abnormal brain structure) are discovered through imaging examinations, they can provide a physiological basis for the patient's psychological symptoms and help understand the root cause of his or her psychological state.

[0062] Aspect five: combining imaging data with psychological assessment results can provide clinicians with a more comprehensive understanding of the patient's condition, thereby formulating more reasonable treatment plans.

[0063] Sixth, doctors can use imaging data to communicate with patients and explain health conditions and treatment plans, thereby enhancing patients' participation and understanding of treatment and reducing anxiety.

[0064] Based on this, it can be determined that the above medical imaging data not only helps in the evaluation of physiological and pathological conditions, but also has a positive impact on the patient's mental health and treatment process.

[0065] The patient facial expression data refers to a visual image of the patient's face that can be captured by an electronic device with a camera shooting function, which can be used to analyze and evaluate the patient's emotions and psychological state. These patient facial expression data can be static photos or dynamic videos, reflecting different facial expressions, such as happiness, sadness, anxiety, anger, etc., so as to be used to evaluate and analyze the patient's psychological state and emotional response.

[0066] The electronic device includes but is not limited to at least one of a smart phone, a tablet computer, a digital camera, a camera monitoring system and a 3D facial scanner.

[0067] The patient's physiological data includes but is not limited to at least one of heart rate variability, blood pressure, respiratory rate, body temperature and biochemical indicators.

[0068] The heart rate variability (HRV) data reflects the degree of variation in the time intervals between heartbeats. Higher HRV indicates greater adaptability of the autonomic nervous system and better health, while lower HRV is often associated with anxiety and stress.

[0069] When the blood pressure manifests as hypertension or fluctuating blood pressure levels, it may be related to emotional stress and anxiety states.

[0070] The respiratory rate can reflect an increase in the patient's respiratory rate when in a state of anxiety or stress, which may manifest as rapid shallow breathing.

[0071] The body temperature can reflect that when the patient has a stress response, the corresponding body temperature may change slightly.

[0072] The biochemical indexes include but are not limited to at least one of cortisol, norepinephrine, epinephrine, serotonin, dopamine, endorphins, vitamin D and thyroid hormones that can reflect the patient's psychological stress.

[0073] The above-mentioned cortisol, norepinephrine, epinephrine, serotonin, dopamine, endorphin, vitamin D and thyroid hormone can be used to assess the patient's psychological stress, which is a prior art in the field, so it will not be described in detail here. Through these biochemical indicators, doctors can better assess the patient's psychological state and its relationship with physical health, so as to formulate corresponding treatment plans.

[0074] Therefore, it can be determined that the above patient physiological data can help medical personnel better understand and manage patients' mental health.

[0075] The patient behavior data includes at least one of the patient's daily activity data, sleep quality data, and social interaction data.

[0076] The patient's daily activity data includes but is not limited to at least one of the number of steps, exercise type data, exercise frequency data, and activity intensity data.

[0077] The daily activity data can be recorded through a smart bracelet or mobile phone application worn by the patient, including daily walking steps, running, swimming, cycling and other different types of exercise and their frequency, and even activity intensity.

[0078] The sleep quality data can be reflected through the patient's sleeping time, sleeping duration, light sleep, deep sleep and rapid eye movement behaviors, thereby reflecting the patient's mental health status.

[0079] The social interaction data includes the conversation and communication data and physical contact data between the patient and the medical staff, relatives and friends accompanying the patient. These social interaction data can reflect the patient's psychological state from one aspect.

[0080] These patient behavior data play an important role in comprehensively assessing the patient's health status and can provide medical professionals with more information support, thereby formulating more accurate diagnosis and treatment plans.

[0081] S102, using a preset AI airway analysis model, based on the aforementioned multimodal medical record data analysis, obtain airway threat information that affects the patient's airway safety, and obtain an airway diagnosis and treatment plan suitable for the patient through evaluation.

[0082] Among them, the airway threat information includes airway physiological threat information and airway psychological threat information; the airway diagnosis and treatment plan includes an airway physiological diagnosis and treatment plan and an airway psychological diagnosis and treatment plan.

[0083] The AI ​​airway analysis model includes an AI airway physiological analysis sub-model and an AI patient psychological analysis sub-model; wherein the AI ​​airway physiological analysis sub-model determines the patient's airway physiological state by analyzing the aforementioned multimodal medical record data; the AI ​​patient psychological analysis sub-model determines the patient's psychological state by analyzing the aforementioned multimodal medical record data.

[0084] It is worth noting that when the preset AI airway analysis model is used, the specific steps include step S110: S111, performing data preprocessing on the collected multimodal medical record data.

[0085] Since the data cleaning and data standardization involved in the data preprocessing operation are prior arts in the art, they will not be described in detail here.

[0086] S112, respectively executing the aforementioned AI airway physiological analysis sub-model and AI patient psychological analysis sub-model.

[0087] Among them, the AI ​​airway physiological analysis sub-model takes the aforementioned multimodal medical record data as input data, and obtains physiological output results through data processing; the physiological output results are airway physiological threat information and corresponding matching airway physiological diagnosis and treatment plans.

[0088] The AI ​​patient psychological analysis sub-model takes the aforementioned multimodal medical record data and the aforementioned physiological output results as input data, and obtains psychological output results through data processing; the psychological output results are airway psychological threat information and corresponding matching airway psychological diagnosis and treatment plans.

[0089] It should be noted that the training and verification of the AI ​​airway physiological analysis sub-model and the AI ​​patient psychological analysis sub-model both adopt existing technologies in the field, so they will not be elaborated here.

[0090] In actual operation, the preset AI airway analysis model can also provide accurate prediction and warning of airway threats, helping doctors to make more effective diagnosis and treatment. At the same time, the continuous learning and feedback mechanism of the AI ​​airway physiological analysis sub-model and the AI ​​patient psychological analysis sub-model can ensure the continuous optimization and improvement of the model.

[0091] In addition, the AI ​​airway physiological analysis sub-model and the AI ​​patient psychological analysis sub-model can both use existing deep learning models, such as convolutional neural networks (CNN), long short-term memory networks (LSTM), graph neural networks (GNN), etc.) to process different data types such as images, texts, time series, etc., respectively, and fuse them through a shared representation layer or attention mechanism (AttentionMechanism).

[0092] As an example but not a limitation, a multi-input multi-output neural network is used to process medical record data of different modalities. For example, medical image data uses a convolutional neural network (CNN) to extract data features; electrocardiograms use recurrent neural networks (RNN) or LSTM to extract features; medical text data uses NLP models to extract semantic features.

[0093] On the other hand, the AI ​​airway physiological analysis sub-model and the AI ​​patient psychological analysis sub-model can also use visualization tools and explanatory AI technology.

[0094] Specifically, the analysis results of multimodal medical record data can be presented to doctors through visualization tools, and the areas and patterns (such as lesion sites, key text content) that the AI ​​airway physiology analysis sub-model and the AI ​​patient psychology analysis sub-model focus on can be intuitively displayed to help doctors quickly understand the decision-making basis of the model.

[0095] Explainable AI can reveal the decision logic of the AI ​​airway physiology analysis sub-model and the AI ​​patient psychology analysis sub-model, and provide understandable explanations, allowing doctors to make further inferences and judgments based on these explanations.

[0096] Furthermore, it is worth mentioning that during the data processing process, when data fusion is performed on the multimodal medical physiological characteristics and multimodal medical psychological characteristics obtained by the AI ​​airway physiological analysis sub-model and the AI ​​patient psychological analysis sub-model respectively, it is preferred to input the aforementioned multimodal medical physiological characteristics and multimodal medical psychological characteristics into the shared representation layer or decision layer of the AI ​​airway physiological analysis sub-model and the AI ​​patient psychological analysis sub-model respectively for analysis.

[0097] Finally, the AI ​​airway physiological analysis sub-model and the AI ​​patient psychological analysis sub-model were verified using the labeled data sets provided by doctors, and parameters were adjusted based on the physiological output results and psychological output results to improve the accuracy of the AI ​​airway physiological analysis sub-model and the AI ​​patient psychological analysis sub-model.

[0098] As another preferred implementation manner of this embodiment, the analysis includes physiological analysis operations and psychological analysis operations.

[0099] The physiological analysis operation includes step S120: S121, extracting multimodal medical physiological characteristics corresponding to the aforementioned multimodal medical record data.

[0100] In this embodiment, the multimodal medical physiological characteristics are indicators reflected by the aforementioned multimodal medical record data, and descriptions corresponding to the indicators, such as numerical values ​​and textual contents reflected by the indicators.

[0101] Specifically, the multimodal medical physiological characteristics include key time series characteristics, statistical characteristics and physiological signal characteristics of the patient.

[0102] The key time series features include but are not limited to the respiratory rate change trend, the heart rate change rate (HRV), and the decrease pattern of blood oxygen saturation that describe the fluctuation of the patient's physiological state.

[0103] The statistical features include but are not limited to the mean, standard deviation, maximum value and / or minimum value that describe the fluctuation of the patient's physiological state.

[0104] The physiological signal features include but are not limited to frequency domain features and variability features that describe the fluctuation of the patient's physiological state. Among them, the frequency domain features can use Fourier transform or wavelet transform to analyze the frequency components in signals such as electrocardiogram and respiratory rate to identify abnormal periodic fluctuations; the variability features can calculate heart rate variability (HRV) and respiratory rate variability to help identify autonomic nervous system dysfunction.

[0105] S122, based on the aforementioned AI airway physiological analysis sub-model, data fusion is performed on the aforementioned multimodal medical physiological characteristics to obtain the aforementioned airway physiological threat information that affects the patient's airway safety.

[0106] For S122, based on the aforementioned AI airway physiological analysis sub-model, the multimodal medical record data as input data can be subjected to data extraction operations to obtain multimodal medical physiological characteristics. Afterwards, the multimodal medical physiological characteristics can be input into the shared representation layer or decision layer of the AI ​​airway physiological analysis sub-model for analysis, thereby obtaining the aforementioned airway physiological threat information affecting the patient's airway safety and the corresponding matching airway physiological diagnosis and treatment plan.

[0107] Among them, it is worth mentioning that the airway physiological threat information is information obtained after analyzing the aforementioned multimodal medical record data through the AI ​​airway physiological analysis sub-model; the airway physiological threat information is specifically feedback information reflecting that the patient's airway is threatened.

[0108] As an example, there is a patient A, 35 years old, female, with a past medical history of asthma. According to her multimodal medical record data, the corresponding multimodal medical physiological characteristics are obtained, as follows: blood oxygen saturation (SpO2): 88% (lower than normal), respiratory rate: 25 times / minute (higher than normal), heart rate: 110 times / minute (higher than normal), blood gas analysis: PaO2 = 55 mmHg (lower than normal), PaCO2 = 50 mmHg (higher than normal), pH = 7.32 (acidosis), allergic reaction type: pollen (yes), smoking history: never smoked, lung X-ray shows: no obvious abnormalities, but the airway is slightly narrowed, and SpO2 value in the last 24 hours: decreased from 95% to 88%, respiratory rate and heart rate gradually increased in the past 6 hours, and the patient reported worsening shortness of breath and paroxysmal cough at night, accompanied by wheezing.

[0109] Through the above multimodal medical physiological characteristics, the patient's airway physiological threat information is obtained as follows: high threat, the patient's blood oxygen saturation is significantly reduced, the respiratory rate and heart rate are increased, and the blood gas analysis shows acidosis. Combined with the previous history of asthma and allergies, the possibility of an acute asthma attack is extremely high and requires immediate treatment.

[0110] Furthermore, corresponding to the aforementioned airway physiological threat information, a corresponding airway physiological diagnosis and treatment plan is matched.

[0111] The psychological analysis operation includes step S130: S131, extracting multimodal medical psychological characteristics corresponding to the aforementioned multimodal medical record data.

[0112] The multimodal medical psychological characteristics include multimodal medical physiological characteristics, language characteristics, behavioral characteristics, cognitive characteristics and social characteristics that reflect the patient's psychological state.

[0113] Among them, multimodal medical physiological characteristics that can reflect the patient's mental state, as an example but not limitation, brain imaging can capture abnormalities in neural structures and functions that are closely related to mental health. For example: the gray matter density of the brain obtained by magnetic resonance imaging (MRI) can reflect that it may be related to emotional disorders such as depression and anxiety when the gray matter volume is reduced; functional magnetic resonance imaging (fMRI) can reveal changes in connections between brain regions, reflecting psychological disorders, such as patients with depression often show excessive or insufficient activity of specific brain networks; positron emission tomography (PET) can analyze the distribution of neurotransmitters (such as dopamine, 5-hydroxytryptamine), thereby helping to understand the patient's emotions and mental state.

[0114] The language features refer to the patient's language expression (speech, voice, intonation, etc.), which can also reflect the patient's psychological state. Specifically, relevant features can be extracted therefrom through natural language processing (NLP) technology.

[0115] The speech characteristics refer to the patient's pitch, speech rate, speech energy, speech pauses, language content and self-focused vocabulary.

[0116] Taking patients with depression as an example, their pitch and speaking speed are often low-pitched and slow, while patients with anxiety disorders may speak faster and with a tense voice.

[0117] The speech energy may indicate a patient's low mood or lack of energy when the sound energy decreases.

[0118] In the case of speech pauses, long pauses or hesitations in the patient may indicate slowed thinking, which is common in patients with depression.

[0119] In terms of language content, patients with depression and anxiety often use more negative words, such as "helpless" and "disappointed".

[0120] Regarding self-focused vocabulary, depressed patients tended to use more self-referential pronouns (e.g., “I,” “my”), indicating an increased self-focus.

[0121] The behavioral characteristics can be obtained by monitoring aspects such as the patient's activity level, social interaction, and sleep pattern. The behavioral characteristics include activity level, sleep pattern, and social interaction.

[0122] For activity levels, excessive reduction in activity (such as reduced exercise, long periods of bed rest) may indicate depression or low mood, while excessive activity may be related to anxiety or mania; for sleep patterns, sleep disorders (such as insomnia, excessive sleepiness) are often closely related to mental health problems such as anxiety and depression; for social interaction, reduced social activities or social isolation may indicate the onset of depression, conversely, patients with anxiety disorders may show excessive social anxiety.

[0123] The cognitive characteristics can be obtained through cognitive assessment or task testing, and the cognitive characteristics include attention level, memory ability, reaction time, anxiety score, depression score and stress level.

[0124] Among them, the attention level can reflect that anxious patients and depressed patients have difficulty concentrating and are easily distracted; the memory ability can reflect the patient's mental health disorders, such as depression or post-traumatic stress disorder (PTSD), which are often accompanied by memory loss or avoidance of certain memories; for reaction time, slower reaction time may be related to depression and low mood; for anxiety scores, it is preferred to use the Generalized Anxiety Inventory (GAD-7) score to reflect the patient's anxiety level; for depression scores, it is preferred to use the Beck Depression Inventory (BDI) or the Patient Health Questionnaire-9 (PHQ-9) to measure the severity of the patient's depressive symptoms; for stress levels, it is preferred to use a stress assessment scale (such as the Perceived Stress Scale PSS) to assess the patient's psychological stress level.

[0125] The social characteristics involve the patient's behavior in social interaction and can reflect their mental health. The social characteristics include social network interaction and social support.

[0126] Among them, for social network interaction, by analyzing the patient's behavioral patterns on social media (such as posting frequency, content tone, number of interactions, etc.), their mental health status can be inferred. For example, patients with depression may reduce social interactions, while patients with anxiety may show social tension; and for social support, the patient's lack of social support or feeling lonely is often significantly associated with depression or anxiety.

[0127] For S132, based on the aforementioned AI airway psychology analysis sub-model, the multimodal medical record data and physiological output results (i.e., airway physiological threat information and corresponding matching airway physiological diagnosis and treatment plans) as input data can be used to obtain multimodal medical psychological features after data extraction operations. Afterwards, the multimodal medical psychological features can be input into the shared representation layer or decision layer of the AI ​​airway psychology analysis sub-model for analysis, thereby obtaining the aforementioned airway psychological threat information and corresponding matching airway psychological diagnosis and treatment plans that affect the patient's airway safety.

[0128] Therefore, it can be determined that based on the aforementioned AI patient psychological analysis sub-model, data fusion of the aforementioned multimodal medical psychological characteristics can obtain the aforementioned airway psychological threat information that affects the patient's airway safety.

[0129] As another preferred implementation manner of this embodiment, the airway psychological diagnosis and treatment plan is divided into a first airway psychological diagnosis and treatment plan, a second airway psychological diagnosis and treatment plan, a third airway psychological diagnosis and treatment plan and a fourth airway psychological diagnosis and treatment plan according to different execution sources.

[0130] The execution sources corresponding to the above-mentioned first to fourth airway psychological diagnosis and treatment plans are the patients themselves, medical staff, relatives and friends of the patients, and social organizations.

[0131] For the patient himself, the first airway psychological diagnosis and treatment plan is an airway psychological diagnosis and treatment plan provided by the patient himself. The first airway psychological diagnosis and treatment plan is specifically carried out around the patient's physiological factors, psychological trait factors and cognitive factors.

[0132] Physiological factors include health status factors and drug side effects factors. By way of example and not limitation, health problems such as chronic diseases, pain or physical dysfunction can directly affect the patient's mental state. For example, certain drugs may cause mood swings, depression or anxiety.

[0133] The psychological trait factors include the patient's personality type, coping mechanism factors and self-control factors. The personality type is such as introversion or extroversion, optimism or pessimism. The coping mechanism is the way the patient copes with stress and difficulties, such as active coping or avoidance coping. The self-control factor refers to the patient's belief that he or she can control and cope with life events.

[0134] The cognitive factors are reflected in the patient's unhealthy thinking patterns such as catastrophic thinking and over-generalization.

[0135] The second airway psychological diagnosis and treatment plan is an airway psychological diagnosis and treatment plan provided by medical staff to patients. It is specifically reflected in the professionalism of the medical team and the quality of interaction between medical staff and patients. For example, the doctor's explanation and comfort can reduce the patient's fear and anxiety about airway diagnosis and treatment. The effect of this operation can be reflected through the doctor's medical record information and the chat information between the doctor and the patient.

[0136] The specific implementation of the second airway psychological diagnosis and treatment plan can be carried out through psychological counseling and treatment, such as cognitive behavioral therapy (CBT), psychodynamic therapy, etc.; it can also improve the patient's psychological state by using psychotropic drugs such as antidepressants and antianxiety drugs.

[0137] In addition, physicians can use behavioral interventions and technology assistance to implement second airway psychological treatment programs.

[0138] As for behavioral intervention, patients can be advised to improve their mental state by increasing exercise, improving diet, and adjusting sleep habits.

[0139] In terms of technical assistance, doctors can provide online psychological counseling services, and can also track patients' mental adjustments through mental health tracking and relaxation training in mobile health applications. They can also use mental health assessment tools to assess patients' mental health status with the assistance of artificial intelligence (AI).

[0140] The third airway psychological diagnosis and treatment plan is an airway psychological diagnosis and treatment plan provided by the patient's relatives and friends to the patient.

[0141] As an example and not a limitation, taking the case where the patient is a pregnant woman, during the entire pregnancy period from pregnancy preparation to delivery, the pregnant woman's relatives and friends can provide her with corresponding psychological support, such as arranging online psychological counseling services for her, tracking and checking the patient's mentality adjustment through mental health tracking and relaxation training in mobile health application software, so as to help her smoothly get through the entire pregnancy period.

[0142] The fourth airway psychological diagnosis and treatment plan is an airway psychological diagnosis and treatment plan provided by social organizations to patients.

[0143] The existence of the fourth airway psychological diagnosis and treatment program is mainly to take into account the humanistic care of the community, street, and company department to which the patient belongs. For example, the unit formulates annual physical examination information fed back by the hospital and corresponding adjustment plans for life pressure and work pressure to relieve the patient's nervous psychology.

[0144] Based on this, it can be determined that the improvement of the patient's psychological state requires comprehensive consideration of multiple factors. The patient's mental health level can be effectively improved by implementing the first airway psychological diagnosis and treatment plan, the second airway psychological diagnosis and treatment plan, the third airway psychological diagnosis and treatment plan, and the fourth airway psychological diagnosis and treatment plan respectively by the patient himself, medical staff, relatives and friends of the patient, and social organizations.

[0145] The assessment also includes S140: S141, for the first to fourth airway psychological diagnosis and treatment plans, weights are set according to the corresponding importance in the airway psychological diagnosis and treatment plans.

[0146] In actual operation, since the first to fourth airway psychological diagnosis and treatment plans have different effects on improving the psychological state of patients, weights are set for the first to fourth airway psychological diagnosis and treatment plans according to their corresponding importance in the airway psychological diagnosis and treatment plans, so that the airway psychological diagnosis and treatment plans are composed of the first to fourth airway psychological diagnosis and treatment plans with different weight values. The weight values ​​can be customized according to the actual situation of the patient.

[0147] S142, executing the aforementioned airway physiological diagnosis and treatment plan and airway psychological diagnosis and treatment plan.

[0148] S143, corresponding to the aforementioned airway physiological diagnosis and treatment plan and airway psychological diagnosis and treatment plan, provide an airway physiological diagnosis and treatment evaluation value and an airway psychological diagnosis and treatment evaluation value respectively.

[0149] Among them, the airway psychological diagnosis and treatment evaluation values ​​include the first airway psychological diagnosis and treatment evaluation value, the second airway psychological diagnosis and treatment evaluation value, the third airway psychological diagnosis and treatment evaluation value and the fourth airway psychological diagnosis and treatment evaluation value corresponding to the aforementioned first airway psychological diagnosis and treatment schemes to the fourth airway psychological diagnosis and treatment schemes.

[0150] The airway physiological diagnosis and treatment evaluation values ​​and the airway psychological diagnosis and treatment evaluation values ​​can be set by the hospital according to actual needs, so that the doctor's team can timely grasp the patient's airway diagnosis and treatment plan, as well as the direction of subsequent adjustments to the airway diagnosis and treatment plan.

[0151] S144, obtaining the execution feedback information of the aforementioned airway physiological diagnosis and treatment plan and the airway psychological diagnosis and treatment plan according to the aforementioned airway physiological diagnosis and treatment evaluation value and the airway psychological diagnosis and treatment evaluation value.

[0152] The execution feedback information is used to reflect the lack or missing content in the aforementioned airway physiological diagnosis and treatment plan and airway psychological diagnosis and treatment plan, such as the airway physiological diagnosis and treatment plan not taking into account the patient's allergic condition, or the airway psychological diagnosis and treatment plan not taking into account the patient's somatoform disorder, etc.

[0153] S145, according to the aforementioned execution feedback information, adjusting the weights of the first to fourth airway psychological diagnosis and treatment plans in the aforementioned airway psychological diagnosis and treatment plans, and executing the adjusted airway psychological diagnosis and treatment plans.

[0154] As an example and not a limitation, the airway psychological diagnosis and treatment plan reflected in the feedback information does not take into account the patient's somatoform disorder. Somatoform disorder is a psychological disorder in which patients complain of physical symptoms, and no obvious physiological cause can be found after medical examination. The treatment of this disorder requires the joint efforts of patients, doctors, relatives, friends and society to intervene in multiple aspects such as psychology, behavior and social support.

[0155] Specifically, if a treatment or intervention plan is developed without previously considering somatic disorders, the weights of these plans may focus on more general mental health interventions rather than targeting specific somatic symptoms. As an example, the weights of each plan in the airway psychological diagnosis and treatment plan are set as follows: For the first airway psychological treatment plan, the patient's self-emotional regulation accounts for 20%; for the second airway psychological treatment plan, cognitive behavioral therapy (CBT) accounts for 25%, and drug therapy accounts for 5%; for the third airway psychological treatment plan, relatives and friends need to understand the patient's mental health problems, of which emotional support accounts for 25%, communication skills account for 10%, and behavioral support accounts for 5%; for the fourth airway psychological treatment plan, community support accounts for 10%.

[0156] According to the feedback information from the execution above, the airway psychological diagnosis and treatment plan did not take into account the patient's somatoform disorder. The weights of the first to fourth airway psychological diagnosis and treatment plans are adjusted as follows: For the first airway psychological diagnosis and treatment plan, the patient's self-emotional regulation accounts for 35%; for the second airway psychological diagnosis and treatment plan, cognitive behavioral therapy (CBT) accounts for 20%, and drug treatment accounts for 10%; for the third airway psychological diagnosis and treatment plan, relatives and friends need to understand the patient's mental health problems, of which emotional support accounts for 10%, communication skills account for 10%, and behavioral support accounts for 10%; for the fourth airway psychological diagnosis and treatment plan, community support accounts for 5%, thereby helping the patient's autonomy and recovery by reducing the excessive attention of relatives, friends and society to the patient's symptoms, thereby improving the execution and efficacy of the patient's airway diagnosis and treatment.

[0157] From this, it can be determined that the distribution of weights needs to be adjusted according to the patient's specific situation, the severity of symptoms, the degree of response to treatment, and the situation of family and social support. Specifically, the weights of the first to fourth airway psychological diagnosis and treatment plans in the aforementioned airway psychological diagnosis and treatment plans can be adjusted according to the aforementioned execution feedback information, so as to achieve better diagnosis and treatment results in the subsequent implementation of the plan.

[0158] The advantage of this operation is that it can timely obtain the execution feedback information of the airway physiological diagnosis and treatment plan and the airway psychological diagnosis and treatment plan through the airway physiological diagnosis and treatment evaluation value and the airway psychological diagnosis and treatment evaluation value, and then timely adjust the weights of the first airway psychological diagnosis and treatment plan to the fourth airway psychological diagnosis and treatment plan in the aforementioned airway psychological diagnosis and treatment plan, so as to timely execute the airway diagnosis and treatment plan suitable for the patient's diagnosis and treatment.

[0159] For other technical features, please refer to the previous embodiments and will not be described in detail here.

[0160] In addition, see Figure 2 As shown, the present invention also provides an embodiment, providing an AI-based multimodal human-machine collaborative airway assessment device 200, comprising: The information collection unit 201 is used to collect multimodal medical record data of at least one patient; the multimodal medical record data includes medical text data, medical audio data, medical image data, patient facial expression data, patient physiological data and patient behavior data.

[0161] The information evaluation unit 202 is used to use a preset AI airway analysis model to obtain airway threat information that affects the patient's airway safety based on the aforementioned multimodal medical record data analysis, and obtain an airway diagnosis and treatment plan suitable for the patient through evaluation; wherein the airway threat information includes airway physiological threat information and airway psychological threat information; the airway diagnosis and treatment plan includes an airway physiological diagnosis and treatment plan and an airway psychological diagnosis and treatment plan.

[0162] For other technical features, please refer to the previous embodiments and will not be described in detail here.

[0163] In addition, see Figure 3 As shown, the present invention also provides an embodiment, providing an AI-based multimodal human-machine collaborative airway assessment system 300, including: The network node 301 is used to send and receive multimodal medical record data.

[0164] The information processing module 302 is used to process the aforementioned multimodal medical record data based on a preset AI airway analysis model.

[0165] The system server 303 is connected to the network node 301 and the information processing module 302 .

[0166] The system server 303 is configured to: collect multimodal medical record data of at least one patient; the multimodal medical record data includes medical text data, medical audio data, medical image data, patient facial expression data, patient physiological data and patient behavior data; use a preset AI airway analysis model to obtain airway threat information that affects the patient's airway safety based on the aforementioned multimodal medical record data analysis, and obtain an airway diagnosis and treatment plan suitable for the patient through evaluation; wherein the airway threat information includes airway physiological threat information and airway psychological threat information; the airway diagnosis and treatment plan includes an airway physiological diagnosis and treatment plan and an airway psychological diagnosis and treatment plan.

[0167] For other technical features, please refer to the previous embodiments and will not be repeated here.

[0168] In addition, an embodiment of the present invention also provides a computer-readable storage medium on which a program is stored, which is used in the aforementioned AI-based multimodal human-computer collaborative airway assessment system. When the program is executed by the processor, it can implement the steps of any of the above-mentioned AI-based multimodal human-computer collaborative airway assessment methods.

[0169] For other technical features, please refer to the previous embodiments and will not be repeated here.

[0170] In the above description, within the scope of the target protection of the present disclosure, the components can be selectively and operatively combined in any number. In addition, terms such as "including", "comprising" and "having" should be interpreted as inclusive or open by default, rather than exclusive or closed, unless they are explicitly defined to the contrary. All technical, scientific or other terms have the meaning understood by those skilled in the art unless they are defined to the contrary. Common terms found in dictionaries should not be interpreted too idealistically or too impractically in the context of relevant technical documents, unless the present disclosure explicitly defines them as such.

[0171] Although the exemplary aspects of the present disclosure have been described for illustrative purposes, those skilled in the art should appreciate that the above description is only a description of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. The scope of the preferred embodiments of the present invention includes other implementations in which functions may not be performed in the order in which they appear or are discussed. Any changes or modifications made by those skilled in the art based on the above disclosure are within the scope of the claims.

Claims

1. A multimodal human-machine collaborative airway assessment method based on AI, characterized in that: include: Collecting multimodal medical record data of at least one patient; the multimodal medical record data includes medical text data, medical audio data, medical image data, patient facial expression data, patient physiological data and patient behavior data; Using a preset AI airway analysis model, airway threat information that affects the patient's airway safety is obtained based on the aforementioned multimodal medical record data analysis, and an airway diagnosis and treatment plan suitable for the patient is obtained through evaluation; wherein, the airway threat information includes airway physiological threat information and airway psychological threat information; the airway diagnosis and treatment plan includes an airway physiological diagnosis and treatment plan and an airway psychological diagnosis and treatment plan.

2. The method according to claim 1, characterized in that: The medical text data includes at least one of electronic medical record data, doctor's note data, examination report data, nursing record data, patient feedback data, psychological assessment questionnaire data, treatment record data and self-report questionnaire data; The medical audio data includes recorded conversation data between a doctor and a patient, and at least one of heart sound data and lung sound data recorded by a stethoscope; wherein the recorded conversation data includes at least one of consultation conversation data between a doctor and a patient, communication data between medical personnel, telemedicine conversation data, patient follow-up record data, and psychological consultation record data; the heart sound data includes at least one of heart sound recording data, heart sound feature analysis data, heart sound variability data, and heart sound clinical record data; The medical imaging data includes imaging data obtained by at least one of X-ray, computerized tomography, magnetic resonance imaging, ultrasonography, positron emission tomography and nuclear medicine; The patient's facial expression data refers to a visual image of the patient's face that can be captured by an electronic device with a camera shooting function; the electronic device includes at least one of a smart phone, a tablet computer, a digital camera, a camera monitoring system, and a 3D facial scanner; The patient's physiological data includes at least one of heart rate variability, blood pressure, respiratory rate, body temperature and biochemical indicators; The patient behavior data includes at least one of the patient's daily activity data, sleep quality data, and social interaction data.

3. The method according to claim 1, characterized in that The AI ​​airway analysis model includes an AI airway physiological analysis sub-model and an AI patient psychological analysis sub-model; wherein the AI ​​airway physiological analysis sub-model determines the patient's airway physiological state by analyzing the aforementioned multimodal medical record data; the AI ​​patient psychological analysis sub-model determines the patient's psychological state by analyzing the aforementioned multimodal medical record data.

4. The method according to claim 3, characterized in that When using the preset AI airway analysis model, it specifically includes: Performing data preprocessing on the collected multimodal medical record data; The aforementioned AI airway physiological analysis sub-model and AI patient psychological analysis sub-model are respectively executed; wherein, The AI ​​airway physiological analysis sub-model uses the aforementioned multimodal medical record data as input data, and obtains physiological output results through data processing; the physiological output results are airway physiological threat information and corresponding matching airway physiological diagnosis and treatment plans; The AI ​​patient psychological analysis sub-model takes the aforementioned multimodal medical record data and the aforementioned physiological output results as input data, and obtains psychological output results through data processing; the psychological output results are airway psychological threat information and corresponding matching airway psychological diagnosis and treatment plans.

5. The method according to claim 3, characterized in that: The analysis includes physiological analysis operations and psychological analysis operations; wherein, The physiological analysis operation includes: extracting multimodal medical physiological characteristics corresponding to the aforementioned multimodal medical record data; the multimodal medical physiological characteristics include key time series characteristics, statistical characteristics and physiological signal characteristics of the patient; based on the aforementioned AI airway physiological analysis sub-model, performing data fusion on the aforementioned multimodal medical physiological characteristics to obtain the aforementioned airway physiological threat information affecting the patient's airway safety; The psychological analysis operation includes: extracting multimodal medical psychological characteristics corresponding to the aforementioned multimodal medical record data; the multimodal medical psychological characteristics include multimodal medical physiological characteristics, language characteristics, behavioral characteristics, cognitive characteristics and social characteristics that reflect the patient's psychological state; based on the aforementioned AI patient psychological analysis sub-model, the aforementioned multimodal medical psychological characteristics are data-fused to obtain the aforementioned airway psychological threat information that affects the patient's airway safety.

6. The method according to claim 1, characterized in that The airway psychological diagnosis and treatment plan is divided into a first airway psychological diagnosis and treatment plan, a second airway psychological diagnosis and treatment plan, a third airway psychological diagnosis and treatment plan and a fourth airway psychological diagnosis and treatment plan according to different execution sources; the execution sources corresponding to the first airway psychological diagnosis and treatment plan to the fourth airway psychological diagnosis and treatment plan are the patient himself, medical staff, relatives and friends of the patient, and social organizations.

7. The method according to claim 6, characterized in that The assessment also includes: For the first to fourth airway psychological diagnosis and treatment plans, weights are set according to the corresponding importance in the airway psychological diagnosis and treatment plans; Implement the aforementioned airway physiological diagnosis and treatment plan and airway psychological diagnosis and treatment plan; Corresponding to the aforementioned airway physiological diagnosis and treatment scheme and airway psychological diagnosis and treatment scheme, airway physiological diagnosis and treatment evaluation values ​​and airway psychological diagnosis and treatment evaluation values ​​are provided respectively; wherein the airway psychological diagnosis and treatment evaluation values ​​include the first airway psychological diagnosis and treatment evaluation value, the second airway psychological diagnosis and treatment evaluation value, the third airway psychological diagnosis and treatment evaluation value and the fourth airway psychological diagnosis and treatment evaluation value corresponding to the aforementioned first airway psychological diagnosis and treatment scheme to the fourth airway psychological diagnosis and treatment scheme; Obtaining execution feedback information of the aforementioned airway physiological diagnosis and treatment plan and airway psychological diagnosis and treatment plan according to the aforementioned airway physiological diagnosis and treatment evaluation value and airway psychological diagnosis and treatment evaluation value; According to the aforementioned execution feedback information, the weights of the first to fourth airway psychological diagnosis and treatment plans in the aforementioned airway psychological diagnosis and treatment plans are adjusted, and the adjusted airway psychological diagnosis and treatment plans are executed.

8. An AI-based multimodal human-machine collaborative airway assessment device according to any one of claims 1 to 7, characterized in that include: An information collection unit, used to collect multimodal medical record data of at least one patient; The multimodal medical record data includes medical text data, medical audio data, medical image data, patient facial expression data, patient physiological data and patient behavior data; An information evaluation unit is used to use a preset AI airway analysis model to obtain airway threat information that affects the patient's airway safety based on the aforementioned multimodal medical record data analysis, and obtain an airway diagnosis and treatment plan suitable for the patient through evaluation; wherein the airway threat information includes airway physiological threat information and airway psychological threat information; and the airway diagnosis and treatment plan includes an airway physiological diagnosis and treatment plan and an airway psychological diagnosis and treatment plan.

9. An AI-based multimodal human-machine collaborative airway assessment system according to any one of claims 1 to 7, characterized in that include: Network nodes for sending and receiving multimodal medical record data; An information processing module, used to process the aforementioned multimodal medical record data based on a preset AI airway analysis model; A system server, the system server connecting the network nodes and the information processing module; The system server is configured to: collect multimodal medical record data of at least one patient; the multimodal medical record data includes medical text data, medical audio data, medical image data, patient facial expression data, patient physiological data and patient behavior data; use a preset AI airway analysis model to obtain airway threat information that affects the patient's airway safety based on the aforementioned multimodal medical record data analysis, and obtain an airway diagnosis and treatment plan suitable for the patient through evaluation; wherein the airway threat information includes airway physiological threat information and airway psychological threat information; the airway diagnosis and treatment plan includes an airway physiological diagnosis and treatment plan and an airway psychological diagnosis and treatment plan.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 7 are implemented.

Images