Perioperative hypothermia prediction and prevention system based on multi-modal data and deep learning

Abstract

The invention discloses a perioperative hypothermia prediction and prevention system based on multi-modal data and deep learning. The system integrates skin temperature, physiological parameters (such as heart rate, blood pressure and oxyhemoglobin saturation) and operating room environment data (temperature, humidity, airflow and the like), and adopts a deep learning method (CNN-LSTM model, Transformer model and multi-modal fusion model) to carry out data analysis and hypothermia prediction. The system comprises a data acquisition module, a data processing module, a deep learning prediction module and a personalized temperature control module, can realize real-time low body temperature early warning, and provides a personalized intraoperative temperature control strategy. Experimental results show that the system abandons complexity, invasiveness and hysteresis of traditional hypothermia monitoring, hypothermia occurrence can be predicted 20-30 minutes in advance, and the accuracy of intraoperative management is improved. The method can be widely applied to the fields of anesthesia monitoring, operating room intelligent management and perioperative patient body temperature optimization control.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical artificial intelligence and physiological monitoring, and particularly relates to a perioperative hypothermia prediction and prevention system based on multimodal data fusion and deep learning, which can be used for intraoperative body temperature management to improve surgical safety. Background Art

[0002] Perioperative hypothermia (PH) refers to the phenomenon that the core body temperature of a patient is lower than 36°C during the surgical process. Research shows that intraoperative hypothermia may lead to adverse consequences such as abnormal coagulation function, increased postoperative infection rate, and delayed anesthesia recovery. Currently, clinical temperature management mainly adopts the methods of traditional monitoring and empirical judgment, but lacks a precise prediction mechanism, and has the following deficiencies:

[0003] Single data source: Existing methods mainly rely on core body temperature monitoring and do not fully combine multimodal information such as skin temperature, physiological parameters, and surgical environment.

[0004] Low prediction accuracy: Traditional models (such as logistic regression and random forest) perform limitedly in the hypothermia prediction task and lack a deep understanding of complex physiological signals.

[0005] Insufficient personalized regulation: Currently, temperature management measures are usually based on surgical types and empirical values, and individualized intraoperative temperature management cannot be achieved.

[0006] Therefore, developing a hypothermia prediction and prevention system based on multimodal data fusion and deep learning can improve the accuracy of intraoperative temperature management and provide more scientific decision-making support for clinical practice. Summary of the Invention

[0007] In view of the deficiencies of the prior art, the present invention provides an intelligent control method for the surgical environment. The purpose of the present invention is to provide a perioperative hypothermia prediction and prevention system based on multimodal data fusion and deep learning technology. By integrating skin temperature, physiological parameters (such as heart rate, blood pressure, and blood oxygen saturation), and operating room environment data (temperature, humidity, air flow, etc.), and applying an advanced deep learning model, accurate prediction of perioperative hypothermia and personalized temperature management are realized.

[0008] To achieve the above purpose, the present invention provides the following technical solutions:

[0009] Compared with the prior art, the beneficial effects of the present invention are as follows: An intelligent control method for preventing hypothermia in the surgical environment, and the control steps are:

[0010] S1. The external monitoring system monitors the external environment in real time, obtains the data of external temperature, humidity, and air flow rate, and uploads the data to the monitoring platform.

[0011] S2. Place an infrared camera beside the patient to obtain the patient's skin temperature data. The infrared camera is electrically connected to the monitoring platform and the cloud, and uploads the data to the monitoring platform;

[0012] S3. Use medical surgical monitoring equipment to monitor the patient's physiological parameters and upload the data to the monitoring platform;

[0013] S4. Establish a data processing module in the monitoring platform. By receiving the acquired data, perform data cleaning, normalization, feature extraction, and fuse multi-modal data;

[0014] S5. Deep learning prediction module in the platform: Use CNN-LSTM, Transformer, and multi-modal fusion models for hypothermia prediction to improve accuracy and stability.

[0015] S6. Combine the prediction results and intelligently adjust the warming equipment (such as heating blankets, infusion heaters) to provide personalized temperature control strategies.

[0016] Preferably, in step S1, the external detection system includes a thermometer, a hygrometer, and an air velocity meter. The thermometer, hygrometer, and air velocity meter are respectively located beside the operating table. The thermometer is used to monitor the temperature change in the operating room, the hygrometer is used to monitor the humidity change in the operating room, and the air velocity meter is used to monitor the air flow rate in the operating room, and upload the monitored results.

[0017] Preferably, in step S2, the infrared camera is used to monitor the skin temperature. Through real-time monitoring, it is uploaded to the monitoring platform for the monitoring platform to evaluate.

[0018] Preferably, in step S3, the physiological parameters monitored by the surgical monitoring equipment include heart rate, blood pressure, anesthesia depth, and oxygen saturation. Through real-time monitoring, they are uploaded to the monitoring platform for the monitoring platform to evaluate.

[0019] Preferably, in step S4, for the data processing module, after data acquisition is completed, the following data processing steps are performed:

[0020] · Missing value processing: Linear interpolation, KNN interpolation;

[0021] · Outlier detection: Remove outliers based on IQR (interquartile range) or Z-score;

[0022] · Data alignment: Data with different sampling frequencies need to be interpolated and synchronized;

[0023] · Feature normalization: Use Min-Max normalization or standardization (Z-score);

[0024] · Dimensionality reduction (PCA / LASSO) to remove redundant information;

[0025] · Time series features (average value and change rate in the past 5 minutes);

[0026] · Statistical features (mean, variance, skewness, kurtosis);

[0027] · Frequency domain features (FFT transform to analyze physiological signals);

[0028] · Multimodal feature fusion (joint modeling of body temperature, environment, and physiological signals).

[0029] Preferably, in step S5, the deep learning models are CNN-LSTM combined model, Transformer prediction model, and multimodal data fusion model. The three models are not used in isolation, but are co-modeled in a hierarchical manner. Their relationship is as follows:

[0030] (1) CNN-LSTM as a short-term prediction model

[0031] · CNN is responsible for extracting local features from skin temperature images or time series data.

[0032] · LSTM is responsible for modeling the core body temperature trend within a short time range (10 - 30 minutes).

[0033] · Suitable for short-term prediction, it can provide real-time intraoperative warning information for doctors.

[0034] (2) Transformer as a long-term prediction model

[0035] · Process the complete intraoperative data and model the core body temperature change trend over a long time span.

[0036] · Pay attention to important moments of historical data through the self-attention mechanism to improve prediction accuracy.

[0037] · Suitable for predicting the body temperature change trend during the entire surgical process, assisting doctors in formulating long-term body temperature management strategies.

[0038] (3) Multimodal data fusion model integrates information

[0039] · Combine the short time series features extracted by CNN-LSTM and the long time series features extracted by Transformer.

[0040] · Further fuse physiological parameters (heart rate variability, blood pressure, respiratory rate) and environmental parameters (temperature, humidity, air flow) to enhance the comprehensiveness of the model for hypothermia prediction.

[0041] · Improve the overall prediction accuracy, enabling the model to have stronger generalization ability and be applicable to various intraoperative situations.

[0042] To make full use of the advantages of these three models, we can adopt a cascade or hybrid strategy to construct the final hypothermia prediction system:

[0043] Solution 1: Cascade modeling

[0044] 1. CNN-LSTM first makes short-term predictions (10 - 30 minutes) to provide rapid intraoperative warnings.

[0045] 2. Transformer makes long-term predictions (the overall intraoperative body temperature trend) to predict the body temperature development in the next 1 - 2 hours.

[0046] 3. The multi-modal data fusion model corrects the prediction results of both, and optimizes the final prediction results by combining physiological parameters and environmental parameters.

[0047] Solution 2: Hybrid modeling

[0048] · Embed the CNN-LSTM structure in Transformer, enabling Transformer to handle both short-term sequence information and pay attention to long-term dependence relationships.

[0049] · The multi-modal data fusion module inputs the results of both CNN-LSTM and Transformer simultaneously, and combines environmental and physiological factors to generate the final prediction output.

[0050] Preferably, in step S6, according to the output prediction results, the warming equipment (such as heating blankets, infusion heaters, etc.) is intelligently and dynamically adjusted to ensure that the patient's core body temperature is within the safe range.

[0051] The present invention sets up an integrated monitoring platform, which not only monitors environmental factors such as the temperature, humidity, and air velocity in the operating room, but also can adjust the warming equipment, such as heating blankets, infusion heaters, etc., based on the combination of the patient's physiological indicators and skin temperature. Its clinical value lies in:

[0052] · Accurate prediction, improving the intraoperative management level: The existing hypothermia monitoring is lagging behind. The prediction model of this study can give early warnings to help doctors take personalized temperature management measures.

[0053] · Reducing postoperative complications: Hypothermia is closely related to abnormal coagulation function, postoperative infection, etc. Accurate prediction helps to reduce postoperative risks and improve the patient's recovery speed.

[0054] · Facilitate intelligent anesthesia and perioperative management: This study can be part of an intelligent operating room and a smart anesthesia system, promoting cross-disciplinary research between medicine and engineering and enhancing the intelligent level of intraoperative body temperature management. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a framework diagram of the regulation steps of the present invention;

[0056] Figure 2 It is a schematic diagram of the comparative analysis method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0057] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0058] The present invention provides a technical solution: a regulation method for preventing and controlling low body temperature during surgery, and the regulation steps are as follows:

[0059] S1. The external monitoring system monitors the external environment in real time to obtain data on the external temperature, humidity, and air velocity, and uploads the data to the monitoring platform;

[0060] S2. An infrared camera is placed beside the patient to obtain the skin temperature data of the patient. The infrared camera is electrically connected to the monitoring platform and the cloud, and uploads the data to the monitoring platform;

[0061] S3. Use medical surgical monitoring equipment to monitor the physiological parameters of the patient and upload the data to the monitoring platform;

[0062] S4. Establish a data processing module in the monitoring platform, and through receiving the acquired data, perform data cleaning, normalization, feature extraction, and fuse multi-modal data;

[0063] S5. Deep learning prediction module in the platform: Use CNN-LSTM, Transformer, and multi-modal fusion models for low body temperature prediction to improve accuracy and stability.

[0064] S6. Combine the prediction results to intelligently adjust the warming equipment (such as heating blankets, infusion heaters) to provide personalized temperature control strategies.

[0065] Further, in step S1, the environmental detection system includes a thermometer, a hygrometer, and an anemometer. The thermometer, hygrometer, and anemometer are respectively located beside the operating table. The thermometer is used to monitor the temperature change in the operating room, the hygrometer is used to monitor the humidity change in the operating room, and the anemometer is used to monitor the air flow rate in the operating room. The monitored results are uploaded.

[0066] Further, in step S2, the infrared camera captures the body heat distribution of the patient and extracts the skin temperature. The specific implementation is as follows:

[0067] · Core body temperature collection: The core body temperature of the patient is monitored in real time through devices such as an esophageal probe or a rectal probe. Different body temperature monitoring devices can be combined for data collection to ensure the accuracy and stability of body temperature information.

[0068] · Skin temperature collection: An infrared thermometer or a skin temperature sensor (such as a thermocouple, an infrared sensor, etc.) is used to obtain the temperature change on the patient's skin surface. Through real-time monitoring, it is uploaded to the monitoring platform for the monitoring platform to evaluate.

[0069] Further, in step S3, the physiological parameters monitored by the surgical patient monitoring device include heart rate, blood pressure, anesthesia depth, and oxygen saturation. The specific implementation is as follows:

[0070] · Heart rate, blood pressure, and oxygen saturation: The physiological parameters such as heart rate, blood pressure, and blood oxygen saturation of the patient are obtained in real time through common monitoring devices (such as an electrocardiograph, a sphygmomanometer, a pulse oximeter, etc.).

[0071] · Anesthesia depth: The data of anesthesia depth is obtained through a BIS (Bispectral Index) monitor to judge the impact of the anesthesia state on body temperature management. Through real-time monitoring, it is uploaded to the monitoring platform for the monitoring platform to evaluate.

[0072] Further, in step S4, there is a data processing module. After the data collection is completed, it is responsible for cleaning, normalizing, extracting features, and fusing the collected multi-modal data. The specific steps are as follows:

[0073] · Data cleaning: Remove invalid or incorrect data, supplement missing values, and process noise data. For example, for sudden heart rate fluctuations or abnormal temperatures, interpolation methods or filtering algorithms are used for processing to ensure data quality.

[0074] · Data normalization: For different types of sensor data (such as body temperature, blood oxygen saturation, blood pressure, etc.), normalization is performed so that the scale differences of different data sources do not affect the subsequent model training. Standardization methods (such as z-score standardization) can be used for normalization.

[0075] · Feature extraction: Extract useful features (such as rate of change, mean, standard deviation, maximum value, minimum value, etc.) from time series data (such as body temperature, heart rate, etc.); Extract time series features of data through methods such as sliding windows so that subsequent models can effectively capture time dependencies; Integrate multi-modal data, including unified processing of physiological signals such as body temperature and heart rate with operating room environment data to form a complete feature set for use by deep learning models.

[0076] Furthermore, in step S5, it is a deep learning model. The deep learning prediction module is the core of the system, responsible for analyzing the processed multi-modal data and predicting the risk of perioperative hypothermia. The implementation is divided into two parts: model selection and training, and prediction tasks, as follows:

[0077] 1) Model selection and training:

[0078] · CNN-LSTM: Use a convolutional neural network (CNN) to extract local features, and then process time series data through a long short-term memory network (LSTM) to capture the dependencies between different time points. This model can process data with time series properties (such as body temperature, heart rate, blood pressure, etc.) and perform comprehensive analysis in combination with spatial features (such as local regions of body temperature changes).

[0079] · Transformer model: Use the Transformer model to handle long-term dependencies, which can better understand the time series patterns of hypothermia occurrence. Especially when training on large-scale data sets, it can more efficiently capture long-range dependencies in the data.

[0080] · Multi-modal fusion: For multiple data sources (such as skin temperature, blood oxygen, environmental temperature, etc.), use a multi-modal fusion model. Various types of data can be input into different sub-networks for processing, and then fused through concatenation or weighted summation to finally output the risk prediction result of hypothermia.

[0081] 2) Prediction tasks:

[0082] · Hypothermia risk prediction: Through the training of the deep learning model, predict the hypothermia risk of patients during the operation. The prediction result can be the probability of hypothermia occurrence, helping medical staff take measures in advance.

[0083] · Personalized temperature control adjustment: According to the specific data of different patients (such as body temperature, type of surgery, etc.), predict the possibility of hypothermia occurrence, and combine individual differences to intelligently adjust the temperature control strategy.

[0084] Furthermore, step S6 is a personalized temperature control module, which provides real-time and personalized body temperature management strategies based on the deep learning prediction results. The implementation has three aspects, as follows:

[0085] 1) Intelligent heating equipment adjustment:

[0086] · Through the intelligent control system, the prediction results are combined with the actual temperature control equipment (such as heating blankets, infusion heaters, etc.), and the output power of the temperature control equipment is automatically adjusted to ensure that the patient's body temperature is maintained within the ideal range.

[0087] · Algorithms such as PID control (Proportional-Integral-Derivative control) or fuzzy control can be used to automatically adjust the power of the heating equipment and maintain the stability of body temperature.

[0088] 2) Real-time warning system:

[0089] · When the probability of hypothermia predicted by the system exceeds a certain threshold, a warning signal is immediately issued to prompt the anesthesiologist or the surgical team to take corresponding measures (such as adjusting the heating equipment, monitoring the patient's body temperature, etc.).

[0090] · The warning information can be transmitted in real time through channels such as the display screen in the operating room and the mobile devices of medical staff to ensure timely information transmission and reduce delays.

[0091] 3) Closed-loop control:

[0092] · Combining real-time physiological data and environmental data, the system can perform closed-loop control. When the predicted risk of hypothermia is verified, the system will automatically adjust the output of the heating equipment to achieve precise temperature management.

[0093] Furthermore, system integration and optimization are carried out.

[0094] The integration and optimization of the system achieve real-time data acquisition, analysis, prediction, and temperature control management. The specific implementation methods include:

[0095] · Data synchronization and storage:

[0096] · Use a wireless sensor network (WSN) to synchronize various types of data (such as physiological signals, environmental data, etc.) to the cloud computing platform in real time to ensure the timely acquisition and storage of data. The data can be combined with the patient's historical health records through the electronic medical record (EMR) system to provide comprehensive patient status information.

[0097] · Model update and adaptive learning:

[0098] · Adopt an adaptive learning method, continuously collect new data, and continuously optimize the deep learning model to improve the prediction accuracy.

[0099] · The system supports the federated learning architecture to ensure the data privacy of each hospital and operating room while improving the generalization ability of the system.

[0100] Example 1

[0101] An intelligent regulation method for preventing and controlling surgical hypothermia, and the regulation steps are as follows:

[0102] S1. The external monitoring system monitors the external environment in real time, monitors and obtains the data of the external temperature, humidity and air velocity, and uploads the data to the monitoring platform;

[0103] S2. An infrared camera is placed beside the patient to obtain the skin temperature data of the patient. The infrared camera is electrically connected to the monitoring platform and the cloud, and uploads the data to the monitoring platform;

[0104] S3. Use medical surgical monitoring equipment to monitor the physiological parameters of the patient and upload the data to the monitoring platform;

[0105] S4. A data processing module is established in the monitoring platform, and through receiving the obtained data, data cleaning, normalization, feature extraction are carried out, and multi-modal data are fused;

[0106] S5. The deep learning prediction module in the platform: uses CNN-LSTM, Transformer and multi-modal fusion models for hypothermia prediction to improve accuracy and stability.

[0107] S6. Combining the prediction results, intelligently adjust the warming equipment (such as heating blankets, infusion heaters) to provide personalized temperature control strategies.

[0108] Comparative Example 1

[0109] The regulation steps for preventing surgical hypothermia are as follows:

[0110] A1. The key measures for preventing and controlling perioperative hypothermia include physical means and temperature monitoring. Warm clothing should be worn before surgery and a warm surgical environment should be maintained. During surgery, equipment such as heating blankets, heated liquids, and warm air blowers are used to prevent heat loss. At the same time, reasonable control of the depth of anesthesia is adopted to avoid excessive anesthesia affecting the body temperature regulation function. Real-time monitoring of the core body temperature during surgery is particularly important, which can timely detect changes in body temperature and take corresponding measures for intervention.

[0111] A2. After surgery, it is still necessary to continue to keep warm. Heating blankets and maintaining a warm environment are used to help the patient recover body temperature, and the body temperature change is continuously monitored to ensure that it is maintained within the normal range. Temperature monitoring can not only effectively guide the measures for preventing and controlling hypothermia, but also timely adjust the treatment strategy to prevent the negative impact of hypothermia on the patient's recovery.

[0112] In Comparative Example 1, after manual monitoring detected a too low temperature, the heating equipment was adjusted manually. The deficiencies of this method are as follows: It is necessary to have a dedicated person to constantly measure the patient's core body temperature and operate the heating equipment all the time. This is extremely likely to cause untimely heating or overheating, both of which will cause harm to the patient. This not only results in a waste of the human resources of the surgical team, but also the control of the patient's temperature is not precise enough. Moreover, the heating is passive and cannot accurately prevent the risk of hypothermia in advance. In Example 1, through the set comprehensive monitoring platform, this platform not only monitors environmental factors such as the temperature, humidity, and air flow rate in the operating room, but also combines the patient's physiological indicators with the skin temperature, predicts the change of the patient's core body temperature through deep learning, predicts the risk of hypothermia and automatically adjusts the heating equipment for heating.

[0113] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device.

[0114] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principle and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A perioperative hypothermia prediction and prevention system based on multimodal data and deep learning, comprising: S1, data acquisition module, used to obtain the patient's perioperative skin temperature, physiological parameters (heart rate, blood pressure, oxygen saturation, respiratory rate), anesthesia depth (BIS index) and operating room environmental data (temperature, humidity, airflow speed); S2, data processing module, used for data cleaning, normalization, feature extraction, and multimodal data fusion; S3, deep learning prediction module, based on CNN-LSTM, Transformer and multimodal fusion model, to achieve hypothermia risk prediction; S4, personalized temperature control module, provides individualized temperature management strategies based on the prediction results, including intraoperative early warning and intelligent adjustment of heating equipment (such as heating blankets and infusion heaters).

2. The system according to claim 1, characterized in that: The deep learning prediction module uses CNN to extract local features, LSTM to process time series data, and uses the Transformer model to improve the ability to model long-term dependencies.

3. The system according to claim 1, characterized in that: The personalized temperature control module is combined with SHAP and Grad-CAM to perform model interpretability analysis to improve the clinical credibility of the model.

4. The system according to claim 1, characterized in that: The data acquisition module realizes real-time synchronous acquisition and storage of multi-point data in the operating room through a wireless sensor network (WSN) and a cloud computing platform.

5. The system according to claim 1, characterized in that: The system can adapt to different types of surgeries (such as laparoscopic surgery, cardiac surgery, neurosurgery) and supports personalized temperature prediction threshold settings.

6. The system according to claim 1, characterized in that: The deep learning prediction module adopts a multi-task learning approach to simultaneously predict the risk of hypothermia at different surgical stages.

7. The system according to claim 1, characterized in that: The system combines physiological feedback mechanism and uses closed-loop control technology to automatically adjust the output power of the temperature control device.

8. The system according to claim 1, characterized in that: The system uses an adaptive model updating method that can continuously optimize the prediction model based on historical data and real-time feedback.

9. The system according to claim 1, characterized in that: The system provides personalized intraoperative temperature control recommendations and can be integrated with an electronic medical record (EMR) system to enable postoperative follow-up and data recording.

10. The system according to claim 1, characterized in that: The system supports a federated learning architecture to ensure data privacy while improving the generalization ability of the prediction model.

Images