Patient data intelligent analysis method and device based on CCE data

Abstract

The present invention relates to the field of data analysis technology, and discloses a method and device for intelligent analysis of patient data based on CCE data. The method comprises: inputting the patient's multimodal data into the feature embedding layer of a trained multimodal classification warning model for feature conversion to obtain feature embedded data; the multimodal data includes text data, physiological data, laboratory test data, and CEC data; inputting the feature embedded data into the feature fusion layer of the multimodal classification warning model for cross-modal feature fusion to obtain feature fusion data; inputting the feature fusion data into the feature extraction layer of the multimodal classification warning model for feature extraction to obtain feature extraction results; and inputting the feature extraction results into the classification layer of the multimodal classification warning model for classification prediction to obtain a symptom warning result for the target symptom. It can be seen that the implementation of the present invention can improve the analysis accuracy of patient data, thereby improving the accuracy of warning of patient symptoms.

Description

Technical Field

[0001] The present invention relates to the field of data analysis technology, and in particular to a method and device for intelligent analysis of patient data based on CCE data. Background Art

[0002] In clinical diagnosis, doctors usually analyze the patient's condition by integrating various collected physiological data and the patient's main complaints to provide a treatment plan that suits the patient's actual situation.

[0003] However, in the actual diagnosis process, due to the complex causes of some diseases (such as shock) and the relatively hidden early symptoms, it is difficult to identify the precursor symptoms of these diseases using existing technologies, which can easily lead to missing early warnings of the patient's condition and causing the patient to miss the best time for treatment.

[0004] Therefore, a technical solution is proposed that can improve the analysis accuracy of patient data, thereby improving the accuracy of early warning of patient symptoms. Summary of the Invention

[0005] The present invention provides a method and device for intelligent analysis of patient data based on CCE data, which can improve the accuracy of analysis of patient data and thus improve the accuracy of early warning of patient symptoms.

[0006] In order to solve the above technical problems, the first aspect of the present invention discloses a patient data intelligent analysis method based on CCE data, the method comprising:

[0007] Inputting the multimodal data about the patient into the feature embedding layer of the trained multimodal classification early warning model for feature conversion processing to obtain feature embedding data; the multimodal data includes text data and variable data; the variable data includes physiological data, laboratory test data and CEC data;

[0008] Inputting the feature embedding data into the feature fusion layer of the multimodal classification warning model to perform cross-modal feature fusion to obtain feature fusion data;

[0009] Inputting the feature fusion data into the feature extraction layer of the multimodal classification early warning model to perform feature extraction for the target disease to obtain a feature extraction result;

[0010] The feature extraction result is input into the classification layer of the multimodal classification warning model for classification prediction to obtain the patient's symptom warning result regarding the target symptom.

[0011] As an optional implementation, in the first aspect of the present invention, the multimodal data is determined by:

[0012] Obtaining the patient's medical record information and extracting text data corresponding to the patient from the medical record information; the medical record information includes one or more combinations of diagnosis records, comorbidity information, clinical sign information, and patient complaint information;

[0013] Based on the monitoring device, collecting physiological data of the patient within a preset time period; the physiological data includes one or more combinations of body temperature data, respiratory data, electrocardiogram data, hemodynamic data and blood oxygen saturation data;

[0014] Determining laboratory test data corresponding to the patient based on the collected patient sample material; the laboratory test data includes one or more combinations of blood gas analysis data, blood routine data, blood biochemistry data, D-dimer data, myocardial enzyme data, and BNP data;

[0015] Based on the determined cardiac ultrasound digital analysis system, CCE data of the patient is acquired; the CCE data includes CCE image data and / or CCE analysis data.

[0016] As an optional embodiment, in the first aspect of the present invention, the feature embedding layer includes a first self-fusion module for the text data and a second self-fusion module for the variable data;

[0017] The step of inputting the multimodal data about the patient into the feature embedding layer of the trained multimodal classification early warning model for feature conversion processing to obtain feature embedded data includes:

[0018] Inputting the text data about the patient into the feature embedding layer of the trained multimodal classification early warning model, performing a first feature conversion process on the first self-fusion module of the text data to obtain text feature embedding data;

[0019] Inputting the variable data about the patient into the feature embedding layer of the multimodal classification early warning model, performing a second feature conversion process on the second self-fusion module of the variable data, and obtaining variable feature embedding data;

[0020] The feature embedding data includes the text feature embedding data and the variable feature embedding data.

[0021] As an optional embodiment, in the first aspect of the present invention, the step of inputting text data about the patient into the feature embedding layer of the trained multimodal classification early warning model and performing a first feature conversion process on the first self-fusion module of the text data to obtain text feature embedding data includes:

[0022] Inputting text data about the patient into a first self-fusion module for the text data in a feature embedding layer of a trained multimodal classification early warning model;

[0023] Based on the first self-fusion module, the text data is divided into a plurality of text blocks according to the positions of punctuation marks recorded in the text data;

[0024] Based on the first self-fusion module, each of the text blocks is divided into a plurality of character combinations to obtain a character list corresponding to the text data; the character combination includes at least one character;

[0025] Based on the first self-fusion module, generating a token sequence corresponding to the character list according to a preset character recognition method;

[0026] Based on the first self-fusion module, the token sequence is converted into a digital sequence to obtain text feature embedding data corresponding to the text data;

[0027] The step of inputting the variable data about the patient into the feature embedding layer of the multimodal classification early warning model and performing a second feature conversion process on the second self-fusion module of the variable data to obtain variable feature embedding data includes:

[0028] Inputting the variable data about the patient into a second self-fusion module for the variable data in the feature embedding layer of the multimodal classification early warning model;

[0029] Based on the fully connected linear layer set by the second self-fusion module, the variable data is analyzed to obtain variable feature embedding data.

[0030] As an optional embodiment, in the first aspect of the present invention, the feature fusion layer includes a plurality of bidirectional multimodal attention modules;

[0031] The step of inputting the feature embedding data into the feature fusion layer of the multimodal classification warning model to perform cross-modal feature fusion to obtain feature fusion data includes:

[0032] For each of the bidirectional multimodal attention modules in the feature fusion layer of the multimodal classification warning model, the data to be fused is input into the bidirectional multimodal attention module; if the bidirectional multimodal attention module is ranked first, the data to be fused is the feature embedded data; if the bidirectional multimodal attention module is not ranked first, the data to be fused is the fusion result output by the bidirectional multimodal attention module ranked adjacent to the first;

[0033] For each of the bidirectional multimodal attention modules, based on the first linear layer of the bidirectional multimodal attention module, performing layer regularization processing on the data to be fused to obtain a first intermediate result; the first intermediate result includes an intermediate text matrix corresponding to the text data to be fused in the data to be fused and an intermediate variable matrix corresponding to the variable data to be fused in the data to be fused;

[0034] For each of the bidirectional multimodal attention modules, performing an inter-modal information extraction operation on the first intermediate result based on the bidirectional multimodal attention module to obtain a second intermediate result; the second intermediate result includes a second intermediate text result and a second intermediate variable result;

[0035] For each of the bidirectional multimodal attention modules, based on a pre-set residual learning calculation formula, the fusion result corresponding to the bidirectional multimodal attention module is calculated according to the second intermediate result and the data to be fused; if the bidirectional multimodal attention module is ranked last, the fusion result is determined as the feature fusion data.

[0036] As an optional embodiment, in the first aspect of the present invention, the feature extraction layer includes a plurality of self-attention modules;

[0037] The step of inputting the feature fusion data into the feature extraction layer of the multimodal classification warning model to extract target disease-oriented features and obtain feature extraction results includes:

[0038] For each of the self-attention modules in the feature extraction layer of the multimodal classification early warning model, inputting the data to be extracted into the self-attention module to perform a feature extraction operation for the target disease, and obtaining an attention map corresponding to the self-attention module;

[0039] Among them, if the self-attention module is ranked first, the data to be extracted is the feature fusion data; if the self-attention module is not ranked first, the data to be extracted is the attention map output by the self-attention module ranked first; if the self-attention module is ranked last, the attention map is determined as the feature extraction result.

[0040] As an optional embodiment, in the first aspect of the present invention, when the target disease is shock, the disease warning result includes the warning shock type corresponding to the patient and the probability of shock occurrence corresponding to the warning shock type.

[0041] A second aspect of the present invention discloses a patient data intelligent analysis device based on CCE data, the device comprising:

[0042] A feature embedding unit is used to input multimodal data about the patient into the feature embedding layer of the trained multimodal classification early warning model for feature conversion processing to obtain feature embedded data; the multimodal data includes text data and variable data; the variable data includes physiological data, laboratory test data and CEC data;

[0043] A feature fusion unit, configured to input the feature embedding data into the feature fusion layer of the multimodal classification warning model to perform cross-modal feature fusion to obtain feature fusion data;

[0044] A feature extraction unit, configured to input the feature fusion data into a feature extraction layer of the multimodal classification early warning model to perform feature extraction for a target disease and obtain a feature extraction result;

[0045] A classification warning unit is used to input the feature extraction result into the classification layer of the multimodal classification warning model for classification prediction, so as to obtain the symptom warning result of the patient regarding the target symptom.

[0046] As an optional implementation, in the second aspect of the present invention, the multimodal data is determined by:

[0047] Obtaining the patient's medical record information and extracting text data corresponding to the patient from the medical record information; the medical record information includes one or more combinations of diagnosis records, comorbidity information, clinical sign information, and patient complaint information;

[0048] Based on the monitoring device, collecting physiological data of the patient within a preset time period; the physiological data includes one or more combinations of body temperature data, respiratory data, electrocardiogram data, hemodynamic data and blood oxygen saturation data;

[0049] Determining laboratory test data corresponding to the patient based on the collected patient sample material; the laboratory test data includes one or more combinations of blood gas analysis data, blood routine data, blood biochemistry data, D-dimer data, myocardial enzyme data, and BNP data;

[0050] Based on the determined cardiac ultrasound digital analysis system, CCE data of the patient is acquired; the CCE data includes CCE image data and / or CCE analysis data.

[0051] As an optional embodiment, in the second aspect of the present invention, the feature embedding layer includes a first self-fusion module for the text data and a second self-fusion module for the variable data;

[0052] The feature embedding unit inputs the multimodal data about the patient into the feature embedding layer of the trained multimodal classification warning model for feature conversion processing to obtain the feature embedded data in a specific manner, including:

[0053] Inputting the text data about the patient into the feature embedding layer of the trained multimodal classification early warning model, performing a first feature conversion process on the first self-fusion module of the text data to obtain text feature embedding data;

[0054] Inputting the variable data about the patient into the feature embedding layer of the multimodal classification early warning model, performing a second feature conversion process on the second self-fusion module of the variable data, and obtaining variable feature embedding data;

[0055] The feature embedding data includes the text feature embedding data and the variable feature embedding data.

[0056] As an optional embodiment, in the second aspect of the present invention, the feature embedding unit inputs the text data about the patient into the feature embedding layer of the trained multimodal classification warning model, performs a first feature conversion process on the first self-fusion module of the text data, and obtains the specific method of text feature embedding data, including:

[0057] Inputting text data about the patient into a first self-fusion module for the text data in a feature embedding layer of a trained multimodal classification early warning model;

[0058] Based on the first self-fusion module, the text data is divided into a plurality of text blocks according to the positions of punctuation marks recorded in the text data;

[0059] Based on the first self-fusion module, each of the text blocks is divided into a plurality of character combinations to obtain a character list corresponding to the text data; the character combination includes at least one character;

[0060] Based on the first self-fusion module, generating a token sequence corresponding to the character list according to a preset character recognition method;

[0061] Based on the first self-fusion module, the token sequence is converted into a digital sequence to obtain text feature embedding data corresponding to the text data;

[0062] The feature embedding unit inputs the variable data about the patient into the feature embedding layer of the multimodal classification early warning model, performs a second feature conversion process on the second self-fusion module of the variable data, and obtains the specific method of variable feature embedding data, including:

[0063] Inputting the variable data about the patient into a second self-fusion module for the variable data in the feature embedding layer of the multimodal classification early warning model;

[0064] Based on the fully connected linear layer set by the second self-fusion module, the variable data is analyzed to obtain variable feature embedding data.

[0065] As an optional embodiment, in the second aspect of the present invention, the feature fusion layer includes a plurality of bidirectional multimodal attention modules;

[0066] The feature fusion unit inputs the feature embedding data into the feature fusion layer of the multimodal classification warning model to perform cross-modal feature fusion to obtain the feature fusion data in a specific manner including:

[0067] For each of the bidirectional multimodal attention modules in the feature fusion layer of the multimodal classification warning model, the data to be fused is input into the bidirectional multimodal attention module; if the bidirectional multimodal attention module is ranked first, the data to be fused is the feature embedded data; if the bidirectional multimodal attention module is not ranked first, the data to be fused is the fusion result output by the bidirectional multimodal attention module ranked adjacent to the first;

[0068] For each of the bidirectional multimodal attention modules, based on the first linear layer of the bidirectional multimodal attention module, performing layer regularization processing on the data to be fused to obtain a first intermediate result; the first intermediate result includes an intermediate text matrix corresponding to the text data to be fused in the data to be fused and an intermediate variable matrix corresponding to the variable data to be fused in the data to be fused;

[0069] For each of the bidirectional multimodal attention modules, performing an inter-modal information extraction operation on the first intermediate result based on the bidirectional multimodal attention module to obtain a second intermediate result; the second intermediate result includes a second intermediate text result and a second intermediate variable result;

[0070] For each of the bidirectional multimodal attention modules, based on a pre-set residual learning calculation formula, the fusion result corresponding to the bidirectional multimodal attention module is calculated according to the second intermediate result and the data to be fused; if the bidirectional multimodal attention module is ranked last, the fusion result is determined as the feature fusion data.

[0071] As an optional embodiment, in the second aspect of the present invention, the feature extraction layer includes a plurality of self-attention modules;

[0072] The feature extraction unit inputs the feature fusion data into the feature extraction layer of the multimodal classification warning model to perform feature extraction for the target disease, and obtains the feature extraction result in a specific manner including:

[0073] For each of the self-attention modules in the feature extraction layer of the multimodal classification early warning model, inputting the data to be extracted into the self-attention module to perform a feature extraction operation for the target disease, and obtaining an attention map corresponding to the self-attention module;

[0074] Among them, if the self-attention module is ranked first, the data to be extracted is the feature fusion data; if the self-attention module is not ranked first, the data to be extracted is the attention map output by the self-attention module ranked first; if the self-attention module is ranked last, the attention map is determined as the feature extraction result.

[0075] As an optional embodiment, in the second aspect of the present invention, when the target disease is shock, the disease warning result includes the warning shock type corresponding to the patient and the probability of shock occurrence corresponding to the warning shock type.

[0076] A third aspect of the present invention discloses another device for intelligent analysis of patient data based on CCE data, the device comprising:

[0077] a memory storing executable program code;

[0078] a processor coupled to the memory;

[0079] The processor calls the executable program code stored in the memory to execute part or all of the steps of the patient data intelligent analysis method based on CCE data disclosed in the first aspect of the present invention.

[0080] The fourth aspect of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute some or all steps of the patient data intelligent analysis method based on CCE data disclosed in the first aspect of the present invention.

[0081] Compared with the prior art, the present invention has the following beneficial effects:

[0082] In the present invention, multimodal data about the patient is input into the feature embedding layer of the trained multimodal classification warning model for feature conversion processing to obtain feature embedding data; the multimodal data includes text data and variable data; the variable data includes physiological data, laboratory test data and CEC data; the feature embedding data is input into the feature fusion layer of the multimodal classification warning model for cross-modal feature fusion to obtain feature fusion data; the feature fusion data is input into the feature extraction layer of the multimodal classification warning model for feature extraction for the target disease to obtain a feature extraction result; the feature extraction result is input into the classification layer of the multimodal classification warning model for classification prediction to obtain the patient's disease warning result for the target disease. It can be seen that the implementation of the present invention can input the multimodal data about the patient into the trained multimodal classification warning model for feature conversion to obtain feature embedding data, and then input the feature embedding data into the feature fusion layer for cross-modal feature fusion to obtain feature fusion data, and then input the feature fusion data into the feature extraction layer for disease-oriented feature extraction to obtain feature extraction results, and finally input the feature extraction results into the classification layer for classification prediction to obtain the disease warning results of the patient for the target disease, and can realize the fusion analysis of multimodal data containing CCE data, thereby realizing the warning and classification of the target disease for the patient, and can improve the analysis accuracy and comprehensiveness of the patient data, thereby improving the accuracy of the recognition of the patient's symptoms, and then improving the accuracy of the analysis of the target disease for the patient, so as to improve the warning accuracy and warning efficiency of the patient's disease, which is beneficial for medical staff to take further examination and treatment measures for the patient based on the disease warning results, and is beneficial to improve the timeliness of the treatment of the patient's disease. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0084] Figure 1 This is a flow chart of a patient data intelligent analysis method based on CCE data disclosed in an embodiment of the present invention;

[0085] Figure 2 1 is a flow chart of another method for intelligent analysis of patient data based on CCE data disclosed in an embodiment of the present invention;

[0086] Figure 3 This is a schematic diagram of the architecture of a multimodal classification warning model disclosed in an embodiment of the present invention;

[0087] Figure 4This is a schematic diagram of the structure of a patient data intelligent analysis device based on CCE data disclosed in an embodiment of the present invention;

[0088] Figure 5 This is a structural diagram of another patient data intelligent analysis device based on CCE data disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0089] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0090] The terms "first," "second," and so on, in the description and claims of the present invention and the accompanying drawings are used to distinguish between different items, not to describe a specific order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or end comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed therein, or may optionally include other steps or elements inherent to such process, method, product, or end.

[0091] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute a separate or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0092] The present invention discloses a method and device for intelligent patient data analysis based on CCE data. The method can input multimodal patient data into a trained multimodal classification warning model for feature conversion to obtain feature embedding data. The feature embedding data is then input into a feature fusion layer for cross-modal feature fusion to obtain feature fusion data. The feature fusion data is then input into a feature extraction layer for symptom-specific feature extraction to obtain feature extraction results. Finally, the feature extraction results are input into a classification layer for classification prediction to obtain a symptom warning result for the patient regarding a target symptom. The method can perform fusion analysis on multimodal data containing CCE data, thereby providing early warning and classification for the patient regarding the target symptom. The method can improve the accuracy and comprehensiveness of patient data analysis, thereby improving the accuracy of symptom recognition and the accuracy of target symptom analysis. This improves the accuracy and efficiency of early warning for the patient's symptom, facilitates medical personnel to take further examination and treatment measures for the patient based on the symptom warning results, and improves the timeliness of treatment for the patient's symptom. The following describes each method in detail.

[0093] Example 1

[0094] See also Figure 1 , Figure 1 This is a flow chart of a patient data intelligent analysis method based on CCE data disclosed in an embodiment of the present invention. Figure 1 The described patient data intelligent analysis method based on CCE data can be applied to a patient data intelligent analysis device based on CCE data, which may include one of an intelligent analysis device, an intelligent analysis terminal, an intelligent analysis system, and a server, wherein the server may be a local server or a cloud server, which is not limited in the embodiment of the present invention; the method can also be applied to a cardiac ultrasound digital analysis system, which is not limited in the embodiment of the present invention. Figure 1 As shown, the patient data intelligent analysis method based on CCE data may include the following operations:

[0095] 101. Input the multimodal data about the patient into the feature embedding layer of the trained multimodal classification warning model for feature conversion processing to obtain feature embedding data.

[0096] In an embodiment of the present invention, multimodal data may include text data and variable data; wherein the variable data may include physiological data, laboratory test data, and CEC data; wherein the multimodal classification warning model may be an artificial intelligence model for clinical application constructed after training and testing based on multiple indicators; wherein the multiple indicators may be a variety of data contained in the above-mentioned multimodal data, which is not limited in the embodiment of the present invention. wherein the feature embedding layer may be used to convert the multimodal data into a unified feature embedding, so that the obtained feature embedding data may be in vector form.

[0097] It's important to note that CCE (Critical Care Echocardiography) is a non-invasive ultrasound technique. Because CCE can directly visualize cardiac structure and identify common pathophysiological changes in critically ill patients, it has become an important technical tool for assessing the condition of critically ill patients.

[0098] In the embodiment of the present invention, the architecture of the exemplary multimodal classification warning model can be found in Figure 3 ,in, Figure 3 This is a schematic diagram of the architecture of a multimodal classification warning model disclosed in an embodiment of the present invention. Figure 3 As shown, Figure 3 (a) is the overall architecture of the multimodal classification warning model.

[0099] 102. Input the feature embedding data into the feature fusion layer of the multimodal classification warning model to perform cross-modal feature fusion to obtain feature fusion data.

[0100] 103. The feature fusion data is input into the feature extraction layer of the multimodal classification warning model to extract features oriented to the target disease and obtain feature extraction results.

[0101] In the embodiment of the present invention, the target disease may be any disease whose clinical symptoms are related to cardiac data. For example, the target disease may be shock, which is not limited in the embodiment of the present invention.

[0102] 104. Input the feature extraction results into the classification layer of the multimodal classification warning model for classification prediction to obtain the patient's symptom warning results regarding the target disease.

[0103] In an embodiment of the present invention, the symptom warning result may include the patient's classification type of the target symptom and the patient's symptom occurrence probability of each classification type, which is not limited in the embodiment of the present invention; wherein, the classification layer may include a classification head.

[0104] It can be seen that the implementation of the method described in the embodiment of the present invention can input the multimodal data about the patient into the trained multimodal classification warning model for feature conversion to obtain feature embedding data, and then input the feature embedding data into the feature fusion layer for cross-modal feature fusion to obtain feature fusion data, and then input the feature fusion data into the feature extraction layer for disease-oriented feature extraction to obtain feature extraction results, and finally input the feature extraction results into the classification layer for classification prediction to obtain the disease warning results of the patient for the target disease, and can realize the fusion analysis of multimodal data containing CCE data, thereby realizing the warning and classification of the target disease for the patient, and can improve the accuracy and comprehensiveness of the analysis of the patient data, thereby improving the accuracy of the recognition of the patient's symptoms, and then improving the accuracy of the analysis of the patient's target disease, so as to improve the accuracy and efficiency of the warning of the patient's disease, which is beneficial for medical staff to take further examination and treatment measures for the patient based on the disease warning results, and is beneficial to improve the timeliness of the treatment of the patient's disease.

[0105] In an optional embodiment, the multimodal data may be determined in the following manner:

[0106] Obtaining the patient's medical record information and extracting the patient's corresponding text data from the medical record information; the medical record information includes one or more combinations of diagnosis records, comorbidity information, clinical signs information, and patient complaint information;

[0107] Based on the monitoring equipment, the patient's physiological data is collected within a preset time period; the physiological data includes one or more combinations of body temperature data, respiratory data, electrocardiogram data, hemodynamic data and blood oxygen saturation data;

[0108] Determine the patient's corresponding laboratory test data based on the collected patient sample material; the laboratory test data includes one or more combinations of blood gas analysis data, blood routine data, blood biochemistry data, D-dimer data, myocardial enzyme data, and BNP data;

[0109] Based on the determined cardiac ultrasound digital analysis system, CCE data of the patient is acquired; the CCE data includes CCE image data and / or CCE analysis data.

[0110] For example, the clinical sign information may include cold and wet skin, mottling, decreased urine volume, confusion, irritability, and other descriptions of clinical signs, which are not limited in the embodiment of the present invention.

[0111] Among them, exemplarily, the monitoring device can be a multi-parameter monitor, or other equipment capable of monitoring the patient's physiological data, which is not limited in the embodiment of the present invention; optionally, the respiratory data can include respiratory rate and / or respiratory waveform, which is not limited in the embodiment of the present invention; optionally, the electrocardiogram data can include heart rate and / or heart rhythm, which is not limited in the embodiment of the present invention; optionally, the hemodynamic data can include a combination of one or more of systolic pressure, diastolic pressure, mean arterial pressure, and central venous pressure, which is not limited in the embodiment of the present invention.

[0112] Among them, optionally, the blood gas analysis data may include a combination of one or more of serum pH value, PaCO2, PaO2, PaO2 / FiO2, and blood lactate value, and may also include other types of blood gas analysis data, which is not limited in the embodiment of the present invention; the routine blood data may include a combination of one or more of hemoglobin concentration, white blood cell count, and platelet count, and may also include other types of routine blood data, which is not limited in the embodiment of the present invention; the blood biochemistry data may include a combination of one or more of blood creatinine value, urea nitrogen value, alanine aminotransferase value, aspartate aminotransferase value, and lactate dehydrogenase value, and may also include other types of blood biochemistry data, which is not limited in the embodiment of the present invention; the myocardial enzyme data may include a combination of one or more of creatine kinase value, creatine kinase isoenzyme value, and troponin value, and may also include other types of myocardial enzyme data, which is not limited in the embodiment of the present invention; among them, BNP (Brain Natriuretic Peptide) is a peptide hormone synthesized by the heart.

[0113] Among them, optionally, the CCE analysis data may include LVEF (Left Ventricular Ejection Fractions), LVFAC (Left Ventricular Fractional Area Change), mitral valve E peak, mitral valve E' peak, mitral valve E / E' ratio, aortic VTI (Velocity Time Integral), right ventricular / left ventricular end-diastolic volume ratio, inferior vena cava collapsibility index, TAPSE (Tricuspid Annular Plane Systolic Excursion), RVFAC (Right Ventricular Fractional Area Change) and one or more combinations of right heart morphology assessment, and may also include other data obtained by CCE detection, which is not limited in the embodiments of the present invention.

[0114] Among them, the cardiac ultrasound digital analysis system is a three-dimensional modeling algorithm that integrates knowledge retrieval and neural graphics networks and an electromagnetic tracking system. It can build a real-time, dynamic three-dimensional cardiac cycle full-heart model and embed it into the ultrasound section being collected.

[0115] It can be seen that this optional embodiment can extract the patient's corresponding text data from the acquired patient's medical record information, collect the patient's physiological data based on the monitoring equipment, and determine the patient's corresponding laboratory test data based on the collected patient sample materials, and obtain the patient's CCE data based on the cardiac ultrasound digital analysis system, which can improve the flexibility and reliability of obtaining the patient's multimodal data, and is conducive to improving the accuracy and comprehensiveness of the multimodal data, thereby helping to improve the comprehensiveness and accuracy of the analysis of the patient data.

[0116] In this optional embodiment, optionally, obtaining CCE data of the patient based on the determined cardiac ultrasound digital analysis system may include the following operations:

[0117] Obtaining the patient's original CCE image data;

[0118] Determining an optimal denoising threshold corresponding to the original CCE image data based on the determined first preset algorithm corresponding to the cardiac ultrasound digital analysis system;

[0119] Based on a second preset algorithm corresponding to the cardiac ultrasound digital analysis system, and according to an optimal denoising threshold, performing an image texture denoising operation on the original CCE image data to obtain first denoised image data;

[0120] performing a denoising operation on the first denoised image data based on a third preset algorithm corresponding to the cardiac ultrasound digital analysis system to obtain second denoised image data;

[0121] Based on an index evaluation method pre-set by the cardiac ultrasound digital analysis system, an image quality evaluation operation is performed on the second denoised image data to obtain quality-optimized image data as the patient's CCE image data;

[0122] Based on the cardiac ultrasound digital analysis system, the CCE image data is analyzed to obtain CCE analysis data.

[0123] It can be seen that this optional embodiment can also determine the optimal denoising threshold corresponding to the patient's original CCE image data based on the first preset algorithm corresponding to the determined cardiac ultrasound digital analysis system, and then perform image texture denoising on the original CCE image data based on the second preset algorithm and according to the optimal denoising threshold to obtain first denoised image data, and then generate denoising on the first denoised image data based on the third preset algorithm to obtain second denoised image data, and then perform image quality evaluation on the second denoised image data based on the index evaluation method to obtain quality-optimized image data as the patient's CCE image data, thereby improving the denoising accuracy of the CCE image, which is conducive to effectively retaining the detailed features of the CCE image, thereby improving the quality of the CCE image, and further facilitating improving the analysis accuracy of the CCE image; and, it can also obtain CCE analysis data by analyzing the CCE image data, and can obtain CCE analysis data by analyzing the CCE image data with optimized image quality, which is conducive to improving the accuracy of the CCE analysis data.

[0124] In this optional embodiment, optionally, the feature embedding layer includes a first self-fusion module for text data and a second self-fusion module for variable data;

[0125] The multimodal data about the patient is input into the feature embedding layer of the trained multimodal classification early warning model for feature conversion processing to obtain feature embedding data, which may include the following operations:

[0126] Inputting the text data about the patient into the feature embedding layer of the trained multimodal classification early warning model, performing a first feature conversion process on the first self-fusion module of the text data, and obtaining text feature embedding data;

[0127] Inputting the patient's variable data into the feature embedding layer of the trained multimodal classification early warning model, performing a second feature conversion process on the second self-fusion module of the variable data, and obtaining variable feature embedding data;

[0128] Among them, feature embedding data includes text feature embedding data and variable feature embedding data.

[0129] Wherein, optionally, the first self-fusion module for text data can be a Tokenizer self-fusion module for text data, and the second self-fusion module for variable data can be a fully connected layer linear Tokenizer self-fusion module for variable data, which is not limited in the embodiment of the present invention. Wherein, the Tokenizer self-fusion module can be used to convert the original data into a digital sequence that can be understood by the model, which is not limited in the embodiment of the present invention. Further, illustratively, the Tokenizer self-fusion module for text data can be a BERT Tokenizer pre-trained based on a large corpus, which is not limited in the embodiment of the present invention.

[0130] It can be seen that this optional embodiment can also input the text data about the patient into the first self-fusion module for text data in the multimodal classification warning model to perform a first feature conversion process on the text data to obtain text feature embedding data, and input the variable data about the patient into the second self-fusion module for variable data to perform a second feature conversion process on the variable data to obtain variable feature embedding data. It can use self-fusion modules with different structures to perform different feature conversion processes for different types of patient data, which can improve the feature conversion flexibility and feature conversion accuracy of multimodal data, thereby helping to improve the accuracy of feature embedding data.

[0131] In this optional embodiment, optionally, inputting the text data about the patient into the feature embedding layer of the trained multimodal classification early warning model to perform a first feature conversion process on the first self-fusion module of the text data to obtain text feature embedding data may include the following operations:

[0132] Inputting the text data about the patient into the first self-fusion module for text data in the feature embedding layer of the trained multimodal classification early warning model;

[0133] Based on the first self-fusion module, the text data is divided into a plurality of text blocks according to the positions of punctuation marks recorded in the text data;

[0134] Based on the first self-fusion module, each text block is divided into multiple character combinations to obtain a character list corresponding to the text data; the character combination includes at least one character;

[0135] Based on the first self-fusion module, a token sequence corresponding to the character list is generated according to a pre-set character recognition method;

[0136] Based on the first self-fusion module, the token sequence is converted into a digital sequence to obtain the text feature embedding data corresponding to the text data.

[0137] Among them, each text block includes multiple characters; optionally, the character type of the characters included in each text block can be any text type. For example, the text type can be Chinese type or other text types, which is not limited in the embodiment of the present invention.

[0138] Wherein, the token sequence includes multiple tokens; optionally, each token may include at least one character and / or word and / or word and / or phrase and / or statement and / or sentence, which is not limited in the embodiment of the present invention; wherein, each token may correspond to a character combination with actual semantics. Optionally, based on the first self-fusion module, according to a pre-set character recognition method, a token sequence corresponding to the character list is generated, which can be specifically as follows: based on the first self-fusion module, each character in the character list is first recognized to obtain a character recognition result; then, according to the character recognition result and based on a pre-set word segmentation rule and / or dictionary, multiple tokens are sequentially extracted from the character list to obtain a token sequence, which is not limited in the embodiment of the present invention.

[0139] It can be seen that this optional embodiment can also input text data about the patient into the first self-fusion module for text data in the feature embedding layer of the multimodal classification warning model, and based on the first self-fusion module, according to the punctuation positions recorded in the text data, divide the text data into multiple text blocks, and then divide each text block into multiple character combinations to obtain a character list, and then generate a token sequence corresponding to the character list according to a pre-set character recognition method, and then convert the token sequence into a digital sequence to obtain text feature embedding data, which can improve the processing accuracy and efficiency of text data, thereby improving the accuracy of converting text data into text feature embedding data.

[0140] In this optional embodiment, optionally, inputting the patient's variable data into the feature embedding layer of the trained multimodal classification early warning model to perform a second feature conversion process on the second self-fusion module of the variable data to obtain variable feature embedding data may include the following operations:

[0141] Inputting the patient's variable data into the second self-fusion module for variable data in the feature embedding layer of the trained multimodal classification early warning model;

[0142] Based on the fully connected linear layer set in the second self-fusion module, the variable data is analyzed to obtain variable feature embedding data.

[0143] The fully connected linear layer provided in the second self-fusion module may be a learnable fully connected linear layer, which is not limited in the embodiment of the present invention.

[0144] Optionally, the calculation formula for analyzing variable data in the fully connected linear layer set in the second self-fusion module can be as follows:

[0145] y=Wx+b

[0146] Among them, x is the input feature vector, which can be the input vector corresponding to each data contained in the variable data; W is the weight matrix, b is the bias vector, and y is the output feature vector, which can be the output vector corresponding to each data contained in the variable data; among them, W and b are optional learnable parameters that can be learned in the overall optimization of the model.

[0147] Optionally, the variable feature embedded data includes an output vector corresponding to each piece of data contained in the variable data, that is, the variable feature embedded data can be in vector form, which is not limited in the embodiment of the present invention.

[0148] It can be seen that this optional embodiment can also input the variable data about the patient into the second self-fusion module for variable data in the feature embedding layer of the multimodal classification early warning model, and analyze the variable data based on the fully connected linear layer set by the second self-fusion module to obtain variable feature embedding data, which can improve the processing accuracy and efficiency of the variable data, thereby improving the accuracy of converting the variable data into variable feature embedding data.

[0149] Example 2

[0150] See also Figure 2 , Figure 2 This is a flow chart of a patient data intelligent analysis method based on CCE data disclosed in an embodiment of the present invention. Figure 2 The described patient data intelligent analysis method based on CCE data can be applied to a patient data intelligent analysis device based on CCE data, which may include one of an intelligent analysis device, an intelligent analysis terminal, an intelligent analysis system, and a server, wherein the server may be a local server or a cloud server, which is not limited in the embodiment of the present invention; the method can also be applied to a cardiac ultrasound digital analysis system, which is not limited in the embodiment of the present invention. Figure 2 As shown, the patient data intelligent analysis method based on CCE data may include the following operations:

[0151] 201. Input the multimodal data about the patient into the feature embedding layer of the trained multimodal classification warning model for feature conversion processing to obtain feature embedding data.

[0152] In the embodiment of the present invention, the multimodal data includes text data and variable data; the variable data includes physiological data, laboratory test data and CEC data.

[0153] In an embodiment of the present invention, the feature fusion layer of the multimodal classification warning model includes multiple bidirectional multimodal attention modules; illustratively, the number of bidirectional multimodal attention modules in the feature fusion layer can be two or other numbers, which is not limited in the embodiment of the present invention.

[0154] 202. For each bidirectional multimodal attention module in the feature fusion layer of the multimodal classification warning model, input the data to be fused into the bidirectional multimodal attention module.

[0155] In an embodiment of the present invention, if the bidirectional multimodal attention module is ranked first, the data to be fused is the feature embedded data; if the bidirectional multimodal attention module is not ranked first, the data to be fused is the fusion result output by the bidirectional multimodal attention module that is ranked adjacent to the first one.

[0156] 203. For each bidirectional multimodal attention module, based on the first linear layer of the bidirectional multimodal attention module, perform layer regularization processing on the fused data to obtain a first intermediate result.

[0157] In an embodiment of the present invention, the first intermediate result includes an intermediate text matrix corresponding to the text data to be fused in the data to be fused and an intermediate variable matrix corresponding to the variable data to be fused in the data to be fused; wherein the intermediate text matrix may include a text query matrix, a text key matrix and a text value matrix; the intermediate variable matrix may include a variable query matrix, a variable key matrix and a variable value matrix; optionally, based on the regularization calculation formula corresponding to the above-mentioned first linear layer, layer regularization processing can be performed on the data to be fused to obtain the first intermediate result, which is not limited in the embodiment of the present invention.

[0158] 204. For each bidirectional multimodal attention module, based on the bidirectional multimodal attention module, perform an inter-modal information extraction operation on the first intermediate result to obtain a second intermediate result.

[0159] In an embodiment of the present invention, the second intermediate result includes a second intermediate text result and a second intermediate variable result; optionally, based on the cross-modal calculation formula of the bidirectional multimodal attention module, an inter-modal information extraction operation can be performed on the first intermediate result to obtain the second intermediate result, which is not limited in the embodiment of the present invention.

[0160] 205. For each bidirectional multimodal attention module, based on a preset residual learning calculation formula, the fusion result corresponding to the bidirectional multimodal attention module is calculated according to the second intermediate result and the data to be fused.

[0161] In an embodiment of the present invention, if the bidirectional multimodal attention module is ranked last, the fusion result is determined as feature fusion data; wherein the fusion result includes a text fusion result and a variable fusion result.

[0162] In the embodiment of the present invention, for example, in the bidirectional multimodal attention module, the text feature embedding data X in the input data to be fused is T and variable feature embedding data X V First, use the first linear layer and layer normalization to project the two embedded data according to the following regularization calculation formula to obtain the corresponding query matrix (Query), key matrix (Key), and value matrix (Value):

[0163]

[0164] Among them, l is the sorting number of the bidirectional multimodal attention module, is the text query matrix, is the text key matrix, is the text value matrix, is the variable query matrix, is the variable key matrix, is the variable value matrix.

[0165] Next, we use the cross-modal calculation formula of bidirectional multimodal attention processing to extract inter-modal information and obtain the second intermediate result:

[0166]

[0167] in:

[0168]

[0169] Among them, λ is the weighted hyperparameter; is the second intermediate text result, is the second intermediate variable result.

[0170] Then, using the idea of ​​residual learning and the residual learning calculation formula, the second intermediate result is input into the layer regularization processing and MLP (Multilayer Perceptron) of the second linear layer, and the processing result is added to the original input to finally obtain the output of this layer and the input of the next layer, which is the fusion result:

[0171]

[0172] in, is the text fusion result, is the result of variable fusion.

[0173] For example, the structure of the bidirectional multimodal attention module included in the feature fusion layer of the multimodal classification warning model can be as follows: Figure 3(b) is shown, and the embodiment of the present invention is not limited thereto.

[0174] 206. Input the feature fusion data into the feature extraction layer of the multimodal classification warning model to extract features oriented to the target disease and obtain feature extraction results.

[0175] 207. Input the feature extraction results into the classification layer of the multimodal classification warning model for classification prediction to obtain the patient's symptom warning results regarding the target symptom.

[0176] In the embodiment of the present invention, for other detailed descriptions of step 201 and steps 206-207, please refer to the detailed descriptions of step 101 and steps 103-104 in the first embodiment, which will not be repeated in the embodiment of the present invention.

[0177] It can be seen that the implementation of the method described in the embodiment of the present invention can input the multimodal data about the patient into the trained multimodal classification warning model for feature conversion to obtain feature embedding data, and then input the feature embedding data into the feature fusion layer for cross-modal feature fusion to obtain feature fusion data, and then input the feature fusion data into the feature extraction layer for disease-oriented feature extraction to obtain feature extraction results, and finally input the feature extraction results into the classification layer for classification prediction to obtain the disease warning results of the patient for the target disease, and can realize the fusion analysis of multimodal data containing CCE data, thereby realizing the warning and classification of the target disease for the patient, and can improve the accuracy and comprehensiveness of the analysis of the patient data, thereby improving the accuracy of the recognition of the patient's symptoms, and then improving the accuracy of the analysis of the patient's target disease, so as to improve the accuracy and efficiency of the warning of the patient's disease, which is beneficial for medical staff to take further examination and treatment measures for the patient based on the disease warning results, and is beneficial to improve the timeliness of the treatment of the patient's disease. In addition, for each bidirectional multimodal attention module in the feature fusion layer of the multimodal classification warning model, the data to be fused can be input into the bidirectional multimodal attention module, and based on the first linear layer, the data to be fused is subjected to layer regularization processing to obtain a first intermediate result, and then the inter-modal information extraction operation is performed on the first intermediate result to obtain a second intermediate result, and then based on the residual learning calculation formula and according to the second intermediate result and the data to be fused, the fusion result corresponding to the bidirectional multimodal attention module is calculated. The bidirectional multimodal attention mechanism can discover and encode the mutual correlation between different modalities, realize cross-modal feature fusion between multimodal data, improve the accuracy of feature analysis and feature fusion between different modal data, and help improve the accuracy of feature fusion data, and thus help improve the accuracy of subsequent feature extraction.

[0178] In an optional embodiment, the feature extraction layer includes a plurality of self-attention modules;

[0179] The feature fusion data is input into the feature extraction layer of the multimodal classification warning model to extract target disease-oriented features and obtain feature extraction results, which may include the following operations:

[0180] For each self-attention module in the feature extraction layer of the multimodal classification warning model, the data to be extracted is input into the self-attention module to perform a feature extraction operation for the target disease, and an attention map corresponding to the self-attention module is obtained;

[0181] Among them, if the self-attention module is ranked first, the data to be extracted is the feature fusion data; if the self-attention module is not ranked first, the data to be extracted is the attention map output by the self-attention module ranked first; if the self-attention module is ranked last, the attention map is determined as the feature extraction result.

[0182] For example, the structure of the self-attention module included in the feature extraction layer of the multimodal classification warning model can be as follows: Figure 3 (c) is shown, and the embodiment of the present invention is not limited thereto.

[0183] It can be seen that this optional embodiment can input the data to be extracted into each self-attention module in the feature extraction layer of the multimodal classification warning model to perform feature extraction operations for the target disease, obtain the attention map corresponding to the self-attention module, and determine the attention map output by the last-ranked self-attention module as the feature extraction result, which can realize the connection between the modality-independent feature expression and the overall diagnosis-oriented representation, improve the accuracy of the analysis of patient symptoms, and thus help improve the accuracy of the analysis of patients about target diseases.

[0184] In another optional embodiment, when the target disease is shock, the disease warning result includes the warning shock type corresponding to the patient and the probability of shock occurrence corresponding to the warning shock type.

[0185] Each warning shock type may be one of a hypovolemic shock type, a cardiogenic shock type, an obstructive shock type, and a distributive shock type, which is not limited in the embodiment of the present invention.

[0186] It can be seen that this optional embodiment can, when the target disease is shock, include the disease warning result of the patient's corresponding warning shock type and the corresponding probability of shock occurrence, which can improve the accuracy of the disease warning result, thereby helping to further improve the warning accuracy of the patient's disease, and helping medical staff to take further examination and treatment measures for the patient based on the disease warning results.

[0187] In the embodiment of the present invention, the architecture of the exemplary multimodal classification warning model can be found in Figure 3 ,in, Figure 3 This is a schematic diagram of the architecture of a multimodal classification warning model disclosed in an embodiment of the present invention. Figure 3 As shown:

[0188] Figure 3 (a) is the overall architecture of the multimodal classification warning model, where: Figure 3 In (a), Text embedding is the first self-fusion module of text embedding, Variable embedding is the second self-fusion module of variable embedding, Bidirectional multimodal attention block is the bidirectional multimodal attention module, Self-attention block is the self-attention module, Classification head is the classification head in the classification layer, and Predict is the disease warning result;

[0189] Figure 3 (b) is the bidirectional multimodal attention module structure included in the feature fusion layer of the multimodal classification warning model, where: Figure 3 (b) Bidirectional multimodal attention is bidirectional multimodal attention processing, Norm is regularization processing, and MLP is multi-layer perceptron;

[0190] Figure 3 (c) is the structure of the self-attention module included in the feature extraction layer of the multimodal classification warning model, where Self-attention refers to self-attention processing and Norm refers to regularization processing;

[0191] in, Figure 3 The arrow in the figure indicates the direction of data transmission.

[0192] Example 3

[0193] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of a patient data intelligent analysis device based on CCE data disclosed in an embodiment of the present invention. Figure 4 The described patient data intelligent analysis device based on CCE data may include one of an intelligent analysis device, an intelligent analysis terminal, an intelligent analysis system and a server, wherein the server may be a local server or a cloud server, which is not limited in the embodiment of the present invention; the device can be applied to a cardiac ultrasound digital analysis system, which is not limited in the embodiment of the present invention.

[0194] like Figure 4 As shown, the patient data intelligent analysis device based on CCE data may include:

[0195] A feature embedding unit 301 is configured to input multimodal data about a patient into a feature embedding layer of a trained multimodal classification early warning model for feature conversion processing to obtain feature embedded data; the multimodal data includes text data and variable data; the variable data includes physiological data, laboratory test data, and CEC data;

[0196] A feature fusion unit 302 is used to input the feature embedded data into the feature fusion layer of the multimodal classification warning model to perform cross-modal feature fusion to obtain feature fusion data;

[0197] The feature extraction unit 303 is used to input the feature fusion data into the feature extraction layer of the multimodal classification warning model to perform feature extraction for the target disease and obtain a feature extraction result;

[0198] The classification warning unit 304 is used to input the feature extraction result into the classification layer of the multimodal classification warning model for classification prediction, and obtain the patient's symptom warning result regarding the target symptom.

[0199] It can be seen that the device described in the embodiment of the present invention can input the multimodal data about the patient into the trained multimodal classification warning model for feature conversion to obtain feature embedding data, and then input the feature embedding data into the feature fusion layer for cross-modal feature fusion to obtain feature fusion data, and then input the feature fusion data into the feature extraction layer for disease-oriented feature extraction to obtain feature extraction results. Finally, the feature extraction results are input into the classification layer for classification prediction to obtain the disease warning results of the patient for the target disease, and can realize the fusion analysis of multimodal data containing CCE data, thereby realizing the warning and classification of the target disease for the patient, and can improve the accuracy and comprehensiveness of the analysis of the patient data, thereby improving the accuracy of the recognition of the patient's symptoms, and then improving the accuracy of the analysis of the patient's target disease, so as to improve the accuracy and efficiency of the warning of the patient's disease, which is beneficial for medical staff to take further examination and treatment measures for the patient based on the disease warning results, and is beneficial to improve the timeliness of the treatment of the patient's disease.

[0200] In an optional embodiment, the multimodal data may be determined in the following manner:

[0201] Obtaining the patient's medical record information and extracting the patient's corresponding text data from the medical record information; the medical record information includes one or more combinations of diagnosis records, comorbidity information, clinical signs information, and patient complaint information;

[0202] Based on the monitoring equipment, the patient's physiological data is collected within a preset time period; the physiological data includes one or more combinations of body temperature data, respiratory data, electrocardiogram data, hemodynamic data and blood oxygen saturation data;

[0203] Determine the patient's corresponding laboratory test data based on the collected patient sample material; the laboratory test data includes one or more combinations of blood gas analysis data, blood routine data, blood biochemistry data, D-dimer data, myocardial enzyme data, and BNP data;

[0204] Based on the determined cardiac ultrasound digital analysis system, CCE data of the patient is acquired; the CCE data includes CCE image data and / or CCE analysis data.

[0205] It can be seen that the device described in the implementation of this optional embodiment can extract the text data corresponding to the patient from the acquired patient medical record information, collect the patient's physiological data based on the monitoring equipment, and determine the patient's corresponding laboratory test data based on the collected patient sample materials, and obtain the patient's CCE data based on the cardiac ultrasound digital analysis system, which can improve the flexibility and reliability of obtaining the patient's multimodal data, and is conducive to improving the accuracy and comprehensiveness of the multimodal data, thereby helping to improve the comprehensiveness and accuracy of the analysis of the patient data.

[0206] In this optional embodiment, optionally, the feature embedding layer includes a first self-fusion module for text data and a second self-fusion module for variable data;

[0207] The feature embedding unit 301 inputs the multimodal data about the patient into the feature embedding layer of the trained multimodal classification warning model for feature conversion processing. The specific method of obtaining the feature embedded data may include:

[0208] Inputting the text data about the patient into the feature embedding layer of the trained multimodal classification early warning model, performing a first feature conversion process on the first self-fusion module of the text data, and obtaining text feature embedding data;

[0209] Inputting the patient's variable data into the feature embedding layer of the trained multimodal classification early warning model, performing a second feature conversion process on the second self-fusion module of the variable data, and obtaining variable feature embedding data;

[0210] Among them, feature embedding data includes text feature embedding data and variable feature embedding data.

[0211] It can be seen that the device described in the implementation of this optional embodiment can also input the text data about the patient into the first self-fusion module for text data in the multimodal classification warning model to perform a first feature conversion process on the text data to obtain text feature embedding data, and input the variable data about the patient into the second self-fusion module for variable data to perform a second feature conversion process on the variable data to obtain variable feature embedding data. It can use self-fusion modules with different structures to perform different feature conversion processes on different types of patient data, which can improve the feature conversion flexibility and feature conversion accuracy of multimodal data, thereby helping to improve the accuracy of feature embedding data.

[0212] In this optional embodiment, the feature embedding unit 301 optionally inputs the text data about the patient into the feature embedding layer of the trained multimodal classification warning model, performs a first feature conversion process on the first self-fusion module of the text data, and obtains the text feature embedded data in a specific manner, which may include:

[0213] Inputting the text data about the patient into the first self-fusion module for text data in the feature embedding layer of the trained multimodal classification early warning model;

[0214] Based on the first self-fusion module, the text data is divided into a plurality of text blocks according to the positions of punctuation marks recorded in the text data;

[0215] Based on the first self-fusion module, each text block is divided into multiple character combinations to obtain a character list corresponding to the text data; the character combination includes at least one character;

[0216] Based on the first self-fusion module, a token sequence corresponding to the character list is generated according to a pre-set character recognition method;

[0217] Based on the first self-fusion module, the token sequence is converted into a digital sequence to obtain the text feature embedding data corresponding to the text data;

[0218] The feature embedding unit 301 inputs the variable data about the patient into the feature embedding layer of the trained multimodal classification early warning model, performs a second feature conversion process on the second self-fusion module of the variable data, and obtains the variable feature embedded data in a specific manner, which may include:

[0219] Inputting the patient's variable data into the second self-fusion module for variable data in the feature embedding layer of the trained multimodal classification early warning model;

[0220] Based on the fully connected linear layer set in the second self-fusion module, the variable data is analyzed to obtain variable feature embedding data.

[0221] It can be seen that the device described in this optional embodiment can also input text data about the patient into a first self-fusion module for text data in the feature embedding layer of the multimodal classification early warning model. Based on the first self-fusion module, the text data is segmented into multiple text blocks according to the positions of punctuation marks recorded in the text data. Each text block is then segmented into multiple character combinations to obtain a character list. Then, a token sequence corresponding to the character list is generated according to a pre-set character recognition method. The token sequence is then converted into a numerical sequence to obtain text feature embedding data. This can improve the processing accuracy and efficiency of the text data, thereby improving the accuracy of converting the text data into text feature embedding data. Furthermore, the device can also input variable data about the patient into a second self-fusion module for variable data in the feature embedding layer of the multimodal classification early warning model. Based on the fully connected linear layer provided by the second self-fusion module, the variable data is analyzed to obtain variable feature embedding data. This can improve the processing accuracy and efficiency of the variable data, thereby improving the accuracy of converting the variable data into variable feature embedding data.

[0222] In another optional embodiment, the feature fusion layer includes a plurality of bidirectional multimodal attention modules;

[0223] The feature fusion unit 302 inputs the feature embedded data into the feature fusion layer of the multimodal classification warning model to perform cross-modal feature fusion to obtain the feature fusion data. The specific method may include:

[0224] For each bidirectional multimodal attention module in the feature fusion layer of the multimodal classification warning model, the data to be fused is input into the bidirectional multimodal attention module; if the bidirectional multimodal attention module is ranked first, the data to be fused is the feature embedded data; if the bidirectional multimodal attention module is not ranked first, the data to be fused is the fusion result output by the bidirectional multimodal attention module ranked adjacent to the first;

[0225] For each bidirectional multimodal attention module, based on the first linear layer of the bidirectional multimodal attention module, performing layer regularization processing on the data to be fused to obtain a first intermediate result; the first intermediate result includes an intermediate text matrix corresponding to the text data to be fused in the data to be fused and an intermediate variable matrix corresponding to the variable data to be fused in the data to be fused;

[0226] For each bidirectional multimodal attention module, performing an inter-modal information extraction operation on the first intermediate result based on the bidirectional multimodal attention module to obtain a second intermediate result; the second intermediate result includes a second intermediate text result and a second intermediate variable result;

[0227] For each bidirectional multimodal attention module, based on the pre-set residual learning calculation formula, the fusion result corresponding to the bidirectional multimodal attention module is calculated according to the second intermediate result and the data to be fused; if the bidirectional multimodal attention module is ranked last, the fusion result is determined as the feature fusion data.

[0228] It can be seen that the device described in the implementation of this optional embodiment can input the data to be fused into the bidirectional multimodal attention module for each bidirectional multimodal attention module in the feature fusion layer of the multimodal classification warning model, and perform layer regularization processing on the data to be fused based on the first linear layer to obtain a first intermediate result, and then perform an inter-modal information extraction operation on the first intermediate result to obtain a second intermediate result, and then calculate the fusion result corresponding to the bidirectional multimodal attention module based on the residual learning calculation formula and according to the second intermediate result and the data to be fused. It can discover and encode the mutual correlation between different modalities through the bidirectional multimodal attention mechanism, realize cross-modal feature fusion between multimodal data, improve the accuracy of feature analysis and feature fusion between different modal data, and is conducive to improving the accuracy of feature fusion data, and thus is conducive to improving the accuracy of subsequent feature extraction.

[0229] In this optional embodiment, optionally, the feature extraction layer includes a plurality of self-attention modules;

[0230] The feature extraction unit 303 inputs the feature fusion data into the feature extraction layer of the multimodal classification warning model to extract target disease-oriented features. The specific method of obtaining the feature extraction result may include:

[0231] For each self-attention module in the feature extraction layer of the multimodal classification warning model, the data to be extracted is input into the self-attention module to perform a feature extraction operation for the target disease, and an attention map corresponding to the self-attention module is obtained;

[0232] Among them, if the self-attention module is ranked first, the data to be extracted is the feature fusion data; if the self-attention module is not ranked first, the data to be extracted is the attention map output by the self-attention module ranked first; if the self-attention module is ranked last, the attention map is determined as the feature extraction result.

[0233] It can be seen that the device described in the implementation of this optional embodiment can also input the data to be extracted into each self-attention module in the feature extraction layer of the multimodal classification warning model to perform feature extraction operations for the target disease, obtain the attention map corresponding to the self-attention module, and determine the attention map output by the last-ranked self-attention module as the feature extraction result, which can realize the connection between the modality-independent feature expression and the overall diagnosis-oriented representation, improve the accuracy of the analysis of patient symptoms, and thus help improve the accuracy of the analysis of patients' target diseases.

[0234] In yet another optional embodiment, when the target disease is shock, the disease warning result includes the warning shock type corresponding to the patient and the probability of shock occurrence corresponding to the warning shock type.

[0235] It can be seen that when the device described in this optional embodiment is implemented, when the target disease is shock, the disease warning result can include the patient's corresponding warning shock type and the corresponding probability of shock occurrence, which can improve the accuracy of the disease warning result, thereby helping to further improve the warning accuracy of the patient's disease and helping medical staff to take further examination and treatment measures for the patient based on the disease warning results.

[0236] Example 4

[0237] See also Figure 5 , Figure 5 This is a schematic diagram of the structure of another patient data intelligent analysis device based on CCE data disclosed in an embodiment of the present invention. Figure 5 As shown, the patient data intelligent analysis device based on CCE data may include:

[0238] A memory 401 storing executable program code;

[0239] a processor 402 coupled to the memory 401;

[0240] The processor 402 calls the executable program code stored in the memory 401 to execute part or all of the steps of the patient data intelligent analysis method based on CCE data described in the first embodiment of the present invention or the second embodiment of the present invention.

[0241] Example 5

[0242] An embodiment of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute some or all of the steps in the patient data intelligent analysis method based on CCE data described in Example 1 or Example 2 of the present invention.

[0243] Example 6

[0244] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute some or all of the steps in the patient data intelligent analysis method based on CCE data described in Example 1 or Example 2.

[0245] The device embodiments described above are merely illustrative, wherein the modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, i.e., they may be located in one place or distributed across multiple network modules. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Those skilled in the art can understand and implement the present invention without inventive effort.

[0246] Through the detailed description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus the necessary general hardware platform, or of course, by means of hardware. Based on this understanding, the above technical solution, in essence, or the portion that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0247] Finally, it should be noted that the patient data intelligent analysis method and device based on CCE data disclosed in the embodiments of the present invention are only preferred embodiments of the present invention, and are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A patient data intelligent analysis method based on CCE data, characterized in that: The method comprises: Inputting multimodal data about the patient into a feature embedding layer of a trained multimodal classification early warning model for feature conversion processing to obtain feature embedding data; the multimodal data includes text data and variable data; the variable data includes physiological data, laboratory test data, and CCE data; Inputting the feature embedding data into the feature fusion layer of the multimodal classification warning model to perform cross-modal feature fusion to obtain feature fusion data; Inputting the feature fusion data into the feature extraction layer of the multimodal classification early warning model to perform feature extraction for the target disease to obtain a feature extraction result; The feature extraction result is input into the classification layer of the multimodal classification warning model for classification prediction to obtain the patient's symptom warning result regarding the target symptom.

2. The patient data intelligent analysis method based on CCE data according to claim 1, characterized in that: The multimodal data is determined in the following manner: Obtaining the patient's medical record information and extracting text data corresponding to the patient from the medical record information; the medical record information includes one or more combinations of diagnosis records, comorbidity information, clinical sign information, and patient complaint information; Based on the monitoring equipment, collecting the patient's physiological data within a preset time period; The physiological data includes one or more combinations of body temperature data, respiratory data, electrocardiogram data, hemodynamic data, and blood oxygen saturation data; Determining laboratory test data corresponding to the patient based on the collected patient sample material; the laboratory test data includes one or more combinations of blood gas analysis data, blood routine data, blood biochemistry data, D-dimer data, myocardial enzyme data, and BNP data; Based on the determined cardiac ultrasound digital analysis system, CCE data of the patient is acquired; the CCE data includes CCE image data and / or CCE analysis data.

3. The patient data intelligent analysis method based on CCE data according to claim 2, characterized in that: The feature embedding layer includes a first self-fusion module for the text data and a second self-fusion module for the variable data; The step of inputting the multimodal data about the patient into the feature embedding layer of the trained multimodal classification early warning model for feature conversion processing to obtain feature embedded data includes: Inputting the text data about the patient into the feature embedding layer of the trained multimodal classification early warning model, performing a first feature conversion process on the first self-fusion module of the text data to obtain text feature embedding data; Inputting the variable data about the patient into the feature embedding layer of the multimodal classification early warning model, performing a second feature conversion process on the second self-fusion module of the variable data, and obtaining variable feature embedding data; The feature embedding data includes the text feature embedding data and the variable feature embedding data.

4. The patient data intelligent analysis method based on CCE data according to claim 3, characterized in that: The step of inputting the text data about the patient into the feature embedding layer of the trained multimodal classification early warning model and performing a first feature conversion process on the first self-fusion module of the text data to obtain text feature embedding data includes: Inputting text data about the patient into a first self-fusion module for the text data in a feature embedding layer of a trained multimodal classification early warning model; Based on the first self-fusion module, the text data is divided into a plurality of text blocks according to the positions of punctuation marks recorded in the text data; Based on the first self-fusion module, each of the text blocks is divided into a plurality of character combinations to obtain a character list corresponding to the text data; the character combination includes at least one character; Based on the first self-fusion module, generating a token sequence corresponding to the character list according to a preset character recognition method; Based on the first self-fusion module, the token sequence is converted into a digital sequence to obtain text feature embedding data corresponding to the text data; The step of inputting the variable data about the patient into the feature embedding layer of the multimodal classification early warning model and performing a second feature conversion process on the second self-fusion module of the variable data to obtain variable feature embedding data includes: Inputting the variable data about the patient into a second self-fusion module for the variable data in the feature embedding layer of the multimodal classification early warning model; Based on the fully connected linear layer set by the second self-fusion module, the variable data is analyzed to obtain variable feature embedding data.

5. The method for intelligent analysis of patient data based on CCE data according to any one of claims 1 to 4, characterized in that: The feature fusion layer includes multiple bidirectional multimodal attention modules; The step of inputting the feature embedding data into the feature fusion layer of the multimodal classification warning model to perform cross-modal feature fusion to obtain feature fusion data includes: For each of the bidirectional multimodal attention modules in the feature fusion layer of the multimodal classification warning model, the data to be fused is input into the bidirectional multimodal attention module; if the bidirectional multimodal attention module is ranked first, the data to be fused is the feature embedded data; if the bidirectional multimodal attention module is not ranked first, the data to be fused is the fusion result output by the bidirectional multimodal attention module ranked adjacent to the first; For each of the bidirectional multimodal attention modules, based on the first linear layer of the bidirectional multimodal attention module, performing layer regularization processing on the data to be fused to obtain a first intermediate result; the first intermediate result includes an intermediate text matrix corresponding to the text data to be fused in the data to be fused and an intermediate variable matrix corresponding to the variable data to be fused in the data to be fused; For each of the bidirectional multimodal attention modules, performing an inter-modal information extraction operation on the first intermediate result based on the bidirectional multimodal attention module to obtain a second intermediate result; the second intermediate result includes a second intermediate text result and a second intermediate variable result; For each of the bidirectional multimodal attention modules, based on a pre-set residual learning calculation formula, the fusion result corresponding to the bidirectional multimodal attention module is calculated according to the second intermediate result and the data to be fused; if the bidirectional multimodal attention module is ranked last, the fusion result is determined as the feature fusion data.

6. The patient data intelligent analysis method based on CCE data according to claim 5, characterized in that: The feature extraction layer includes multiple self-attention modules; The step of inputting the feature fusion data into the feature extraction layer of the multimodal classification warning model to extract target disease-oriented features and obtain feature extraction results includes: For each of the self-attention modules in the feature extraction layer of the multimodal classification early warning model, inputting the data to be extracted into the self-attention module to perform a feature extraction operation for the target disease, and obtaining an attention map corresponding to the self-attention module; Among them, if the self-attention module is ranked first, the data to be extracted is the feature fusion data; if the self-attention module is not ranked first, the data to be extracted is the attention map output by the self-attention module ranked first; if the self-attention module is ranked last, the attention map is determined as the feature extraction result.

7. The method for intelligent analysis of patient data based on CCE data according to any one of claims 1, 2, 3, 4 and 6, characterized in that: When the target disease is shock, the disease warning result includes the warning shock type corresponding to the patient and the probability of shock occurrence corresponding to the warning shock type.

8. A patient data intelligent analysis device based on CCE data, characterized in that: The device comprises: A feature embedding unit is used to input multimodal data about the patient into the feature embedding layer of the trained multimodal classification early warning model for feature conversion processing to obtain feature embedded data; the multimodal data includes text data and variable data; the variable data includes physiological data, laboratory test data and CCE data; A feature fusion unit, configured to input the feature embedding data into the feature fusion layer of the multimodal classification warning model to perform cross-modal feature fusion to obtain feature fusion data; A feature extraction unit, configured to input the feature fusion data into a feature extraction layer of the multimodal classification early warning model to perform feature extraction for a target disease and obtain a feature extraction result; A classification warning unit is used to input the feature extraction result into the classification layer of the multimodal classification warning model for classification prediction, so as to obtain the symptom warning result of the patient regarding the target symptom.

9. A patient data intelligent analysis device based on CCE data, characterized in that: The device comprises: a memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the patient data intelligent analysis method based on CCE data as described in any one of claims 1-7.

10. A computer storage medium, characterized in that The computer storage medium stores computer instructions, which, when called, are used to execute the patient data intelligent analysis method based on CCE data as described in any one of claims 1 to 7.

Images