Critical patient sedation state assessment method based on multi-modal network model

Abstract

The invention relates to the technical field of patient state evaluation, and provides a sedative state evaluation method for a critical patient based on a multi-modal network model, and the method comprises the steps: S1, obtaining the original physiological data of the critical patient in real time; s2, converting the EEG data into an electroencephalogram spectrogram; s3, the generated missing EEG data are supplemented into the complete EEG data, and the complete EEG data are updated; s4, cutting the updated complete EEG data through a sliding window to obtain complete EEG data fragments; taking the RASS score as a tag, and carrying out time axis matching on the tag and each complete EEG data segment to form EEG data with the tag; s5, on the basis of the multi-modal network model, training is carried out by utilizing the EEG data with the labels; and S6, performing RASS score prediction on the label-free data. By monitoring the multi-modal physiological parameters of the patient, the sedation depth of the patient in the intensive care unit is objectively evaluated.

Description

Technical Field

[0001] The present invention relates to the technical field of patient status assessment, and particularly to a method for assessing the sedation status of critically ill patients based on a multimodal network model. Background Art

[0002] Critically ill patients in the intensive care unit (ICU) are usually administered sedative drugs, which can relieve the pain of patients and facilitate clinical care. While sedative treatment reduces stress and protects organ function, it can also inhibit the important physiological functions of certain organs or increase the metabolic burden on certain organs, leading to organ function damage. However, for patients using sedative drugs in the ICU, monitoring the sedation status is a challenge. Improper use of sedatives can result in over-sedation and under-sedation. Clinically, the long-term or excessive use of sedatives can cause various damages to the human body. Over-sedation may lead to side effects such as prolonged mechanical ventilation time, increased medical costs, prolonged ICU stay, and mental damage. On the contrary, under-sedation may cause pain, anxiety, and agitation, which may in turn lead to the removal of invasive devices such as intubation and catheters. Therefore, achieving the optimal sedation status is crucial for improving the prognosis of patients.

[0003] Clinically, the sedation level monitoring of critically ill patients usually adopts the method of periodically evaluating the patient's behavioral response to stimuli. For example, the Richmond Agitation-Sedation Scale (RASS) used to monitor the sedation level and the Confusion Assessment Method-ICU (CAM-ICU) used to detect signs of delirium. A large amount of evidence shows that the use of these sedation assessment scores and the sedation regimens based on them can effectively reduce the use rate of mechanical ventilation, shorten the tracheal intubation time, and the hospital stay. As an alternative, electroencephalogram monitoring means have been proposed and preliminarily applied in clinical trials. However, due to the different degrees of illness of the patients, individual differences of the patients, and different surgical categories, the required sedation time will also vary to a certain extent, which poses a certain challenge to the application of electroencephalogram (EEG) signals in the sedation level assessment of ICU patients.

[0004] Meanwhile, when using sedative drugs in the ICU, the basic vital signs of patients (consciousness, heart rate, respiration, blood pressure, urine output, and body temperature) are also closely monitored to select the appropriate drugs and their doses, determine the efficacy target of observation and monitoring, formulate the best individualized treatment plan, and achieve the least adverse reactions and the best efficacy.

[0005] Currently in clinical practice, the assessment of a patient's sedation state is still determined based on a doctor's clinical experience and corresponding scales, and there is no intelligent method that uses feature extraction from electroencephalogram (EEG) signals and other peripheral physiological signals for identifying the patient's sedation depth. In fact, Patent CN112006658A describes a method for evaluating sedation depth using EEG, heart rate signals, and respiratory signals. However, this method has limitations because it does not effectively combine information such as the patient's blood pressure and condition collected clinically, and does not perform a more in-depth extraction and utilization of the features of EEG signals. Summary of the Invention

[0006] The present invention mainly solves the technical problem that the current assessment of a patient's sedation state is still determined based on a doctor's clinical experience and corresponding scales, without using feature extraction from EEG signals and other peripheral physiological signals for identifying the patient's sedation depth. It proposes a method for assessing the sedation state of critically ill patients based on a multimodal network model. By monitoring the multimodal physiological parameters of the patient, an objective assessment of the sedation depth of patients in the intensive care unit can be achieved; through the automated preprocessing and analysis of the collected physiological parameters, the sedation level of the patient can be formed and verified, thus allowing real-time tracking of the patient's sedation state.

[0007] The present invention provides a method for assessing the sedation state of critically ill patients based on a multimodal network model, including:

[0008] Step S1: Real-time acquisition of the original physiological data of critically ill patients; the physiological data includes EEG data and peripheral data, and the peripheral data includes HR data, SpO2 data, and BP data;

[0009] Step S2: Preprocess the original EEG data and convert the EEG data into an EEG spectrogram;

[0010] Step S3: Generate missing EEG data, supplement the generated missing EEG data into the complete EEG data, and update the complete EEG data;

[0011] Step S4: Cut the updated complete EEG data through a sliding window to obtain complete EEG data segments; and match the RASS score as a label with each segment of the complete EEG data segment on the time axis to form labeled EEG data;

[0012] Step S5: Based on the multimodal network model, use the labeled EEG data for training;

[0013] Step S6: Through the trained sedation state assessment model, predict the RASS score for unlabeled data.

[0014] Further, Step S2 includes the following steps S201 to S203:

[0015] Step S201: Remove the abnormal points in the collected original EEG data;

[0016] Step S202: Denoise the EEG data;

[0017] Step S203: Convert the denoised EEG data into an electroencephalogram spectrogram.

[0018] Further, step S202 includes: decomposing the EEG data into sub-bands of different scales, applying threshold processing on the high-frequency sub-bands, reconstructing the signal after removing the noise, then analyzing the wavelet coefficients at each scale, extracting specific patterns, and finally performing lossy compression on the signal by retaining the low-frequency sub-bands and discarding the high-frequency sub-bands exceeding the threshold; the mathematical expression is:

[0019]

[0020] where x(t) represents the input signal, ψ(t) is the mother wavelet, a is the scaling parameter, and b is the translation parameter.

[0021] Further, step S3 specifically includes the following steps S301 to S304:

[0022] Step S301: Determine the integrity of the EEG data and peripheral data to obtain complete EEG data, incomplete EEG data, complete peripheral data, and incomplete peripheral data;

[0023] Step S302: Use the complete EEG data and complete peripheral data to train the RBM model;

[0024] Step S303: Use the trained RBM model to predict the missing data in the incomplete EEG data through the features of the hidden nodes;

[0025] Step S304: Randomly initialize the missing data in the incomplete EEG data, and use the RBM model to generate the missing EEG data, and fill the generated missing EEG data into the incomplete EEG data to form complete EEG data.

[0026] Further, step S4 includes the following steps S401 to S403:

[0027] Step S401: Use a sliding window to cut the complete EEG data into segments to obtain complete EEG data segments;

[0028] Step S402: Align the discrete peripheral data with the complete EEG data segments by means of data interpolation;

[0029] Step S403: Match the RASS scores evaluated by medical staff with each data segment of the interpolated EEG data to form labeled EEG data.

[0030] Further, in step S402, the discrete peripheral data is aligned with the complete EEG data segment by data interpolation, expressed as:

[0031]

[0032] where y is the value to be interpolated, x is the position to insert the interpolation value, (x 1 ,y 1 ) and (x 2 ,y 2 ) are the valid data points before and after the invalid or noisy data respectively.

[0033] Further, step S5 includes the following steps S501 to S503:

[0034] Step S501: Use the WaveNet model to train the labeled EEG signal, labeled HR signal, labeled SpO2 signal, and labeled BP signal;

[0035] Step S502: Use the convolutional neural network model to train the EEG spectrogram generated in step S2;

[0036] Step S503: Based on the multi-modal network model, perform weighted fusion on the WaveNet model trained in step S501 and the convolutional neural network model trained in step S502 to obtain the final sedation state evaluation model.

[0037] Further, the step S503 includes the following process:

[0038] Let be the instance matrix of the first modality, be the instance matrix of the second modality, be the instance matrix of the i-th modality; N is the number of instances, d i are the dimensions of the features extracted from each modality respectively;

[0039] To non-linearly transform the original features of the modalities, deep neural networks are constructed for these two modalities as follows:

[0040] O 1 = f 1 (X 1 ; W 1 )

[0041] O 2 = f 2 (X2 ; W 2 )

[0042] O i = f i (X i ; W i )

[0043] where, W i represents all the parameters of the non - linear transformation, is the output of the neural network, and d represents the output dimension;

[0044]

[0045] For weighted - sum fusion, initialize the hyper - parameter α i , find the optimal value of the weight, and fuse different modalities as follows:

[0046]

[0047] A method for evaluating the sedation state of critically ill patients based on a multi - modal network model provided by the present invention can fuse the data collected by various bedside monitoring devices in the ICU, and at the same time perform special weighted fusion on the features of each modality. It has the function of monitoring the sedation state of patients, can complete the missing data, and can predict the sedation level of patients, realizing the safe monitoring and early warning of the sedation state of critically ill patients. It has the advantages of short response time and high accuracy. The present invention can simultaneously monitor the sedation depth and realize the identification of changes in the sedation level, meet the needs of ICU medical staff to master the sedation level of patients, and can provide a reference for the subsequent infusion rate, thereby providing the most suitable sedation state for critically ill patients and avoiding over - sedation or under - sedation. The method according to the present invention makes it possible to form and verify the sedation level of patients especially based on the automated analysis of physiological activity parameters of patients such as EEG, HR, and BP. Through the three - dimensional monitoring, real - time analysis and formulation of personalized treatment plans for ICU patients by the present invention, the RASS score is quantified so that patients can better adapt to the environment of the intensive care unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is the implementation flowchart of the method for evaluating the sedation state of critically ill patients based on a multi - modal network model provided by the present invention;

[0049] Figure 2 is the logic diagram of data segmentation and tagging in the present invention;

[0050] Figure 3 is the idea diagram of multi - modal feature fusion in the present invention;

[0051] Figure 4It is a curve graph for denoising and filtering EEG signals in the present invention;

[0052] Figure 5 It is a spectrogram of the EEG signal generated by the present invention;

[0053] Figure 6 It is the true value of the RASS score of the patient during the training process;

[0054] Figure 7 It is the RASS prediction value generated by the present invention after combining the multi-modal data of the patient. Specific Embodiments

[0055] To make the technical problems solved, the technical solutions adopted, and the technical effects achieved by the present invention clearer, the present invention will be further described in detail below with reference to the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present invention, rather than limiting the present invention. Additionally, it should be noted that for the sake of description, only parts related to the present invention are shown in the drawings, rather than all the content.

[0056] The present invention provides a method for evaluating the sedation state of critically ill patients based on a multi-modal network model, which can be used to automatically and continuously measure the sedation state of patients in the intensive care unit (hereinafter referred to as critically ill patients). Sedation is a medical technique and management strategy, usually used to treat patients with acute life-threatening diseases to relieve their symptoms, improve their survival status, or create conditions for further treatment. Its core goal is to moderately adjust the patient's state of consciousness, physical reactions, or psychological burden through drugs or other means to ensure their stability during the treatment process and reduce pain and risks.

[0057] After arriving at the intensive care unit, the patient is connected to multiple parameter monitoring devices. These parameter monitoring devices centralize the measurement of various physiological variables of the patient. Among these signals extracted from the monitoring devices, the present invention relies on EEG measurement, ECG (electrocardiogram) monitoring, NIBP (non-invasive blood pressure), and SpO2 measurement. For example, EEG is a graphical representation of the electrical activity of the patient's brain over a period of time. EEG can be obtained by using electrodes placed on the patient's scalp.

[0058] As Figure 1 shown, the method for evaluating the sedation state of critically ill patients based on a multi-modal network model provided by the embodiment of the present invention includes the following processes:

[0059] Step S1: Real-time obtain the original physiological data of the critically ill patient; the physiological data includes EEG data and peripheral data, and the peripheral data includes HR data, SpO2 data, and BP data.

[0060] In this step, various physiological data such as EEG (electroencephalogram) data, HR (heart rate) data, SpO2 (blood oxygen saturation) data, and BP (blood pressure) data are collected through the rich bedside parameter monitoring devices in the ICU. Among them, HR data, SpO2 data, and BP data are regarded as peripheral data. Specifically, electrode patches or electrode caps are used to collect real-time EEG data, HR data, and SpO2 data; a blood pressure measurement device is set to measure BP data every 30 minutes.

[0061] Step S2: Preprocess the original EEG data and convert the EEG data into an electroencephalogram spectrogram.

[0062] In this step, the original EEG data is preprocessed to remove artifacts and low-quality signals in various original EEG data. Specifically, the purpose is to ensure that the collected signals are as complete as possible and of acceptable quality. Eliminate the DC offset and low-frequency drift of the signal, and restore the true dynamic characteristics. Eliminate random noise and device noise, highlight the signal features, and remove the baseline drift of the signal, filter out power frequency noise, apply wavelet denoising, identify and remove artifacts caused by non-physiological or external interference. Step S2 specifically includes the following steps S201 to S203:

[0063] Step S201: Eliminate the abnormal points in the collected original EEG data.

[0064] After detecting the amplitude value, low standard deviation, and amplitude change of the original EEG data, the threshold method is used to eliminate abnormal data. The specific process of this step: Detect EEG data with an abnormal high amplitude value exceeding 500 μV, observe EEG data with a low standard deviation < 0.2 μV of the data, identify rapid amplitude changes exceeding 900 μV within 0.1 s and other abnormal data, and eliminate them. This step can be achieved by an automatic combined with manual method.

[0065] Step S202: Perform denoising processing on the EEG data.

[0066] This step is carried out by the method of wavelet transform and is used to filter out interference signals during the acquisition process. The specific process is as Figure 2 shown: Decompose the EEG data into sub-bands of different scales, apply threshold processing on the high-frequency sub-bands, reconstruct the signal after removing the noise, then analyze the wavelet coefficients at each scale, extract specific patterns, and finally perform lossy compression on the signal by retaining the low-frequency sub-bands and discarding the high-frequency sub-bands exceeding the threshold. The mathematical expression is:

[0067]

[0068] Among them, x(t) represents the input signal, ψ(t) is the mother wavelet, a is the scaling parameter, and b is the translation parameter.

[0069] Step S203: Convert the denoised EEG data into an electroencephalogram spectrogram.

[0070] In this step, the denoised EEG data is converted into an electroencephalogram spectrogram (as shown in Figure 3 ), and the spectrogram is used as one of the modalities for training to facilitate the time-frequency analysis of EEG data. In this step, the absolute value or square value of time b, frequency f, and wavelet coefficient W(a,b) is plotted as an electroencephalogram spectrogram, which can be expressed as:

[0071] Spectrogram(b,f) = |W(a,b)| 2

[0072] Step S3: Generate missing EEG data, fill the generated missing EEG data into the complete EEG data, and update the complete EEG data.

[0073] The complex environment in the ICU and patient interference can also cause signal artifacts and data loss. Therefore, this step interpolates the missing data. Specifically, it is to learn the relationship between EEG data and each peripheral data through a multi-modal learning method, use the peripheral data as supplementary information to infer and fill the missing part of the EEG data, and vice versa. Step S3 specifically includes the following steps S301 to S304:

[0074] Step S301: Determine the integrity of the EEG data and peripheral data to obtain complete EEG data, incomplete EEG data, complete peripheral data, and incomplete peripheral data.

[0075] Incomplete EEG data means that there is missing EEG data, and incomplete peripheral data means that there is missing peripheral data. The method for determining the integrity of the EEG data and peripheral data: Real-time monitor the packet loss rate of the EEG data and peripheral data. If the continuous loss time > 2 seconds or the cumulative loss rate > 5%, it is determined as an incomplete data segment.

[0076] Step S302: Use the complete EEG data and complete peripheral data to train the RBM model.

[0077] The RBM model is a stochastic neural network model with a hidden layer and a visible layer, which are symmetrically connected and have no self-feedback, with full connections between layers and no connections within layers. The data nodes in the hidden layer are hidden nodes. The data nodes in the visible layer are visible nodes. In the present invention, each modality corresponds to a group of visible nodes, and then the data of the visible nodes is input into the RBM model, and the state of the hidden nodes is calculated through weights and activation functions.

[0078] For a given state vector and Then the current energy function of the RBM (Restricted Boltzmann Machine) model can be expressed as:

[0079]

[0080] Where n v is the number of neurons in the visible layer, and n h is the number of neurons in the hidden layer, a i is the bias of the i-th neuron in the visible layer, b j is the bias of the j-th neuron in the hidden layer, and W ij is the connection weight between the i-th neuron in the visible layer and the j-th neuron in the hidden layer.

[0081] Based on the energy function, the joint probability distribution of the visible layer and the hidden layer can be expressed as:

[0082]

[0083] Where is the partition function, which is used to ensure the normalization of the probability distribution.

[0084] Given the visible layer v, the activation probability of the j-th neuron in the hidden layer is:

[0085]

[0086] Where is the sigmoid function.

[0087] Given the hidden layer h, the activation probability of the i-th neuron in the visible layer is:

[0088]

[0089] The update formulas for the weight W, the visible layer bias a, and the hidden layer bias b are as follows:

[0090] ΔW ij = ε(<v i h j > data - <v i h j > recon )

[0091] Δa i = ε(<v i > data - <v i > recon )

[0092] Δb i = ε(<hi > data -<h i > recon )

[0093] Among them, ε is the learning rate, <·> data represents the expectation on the training data, and <·> recon represents the expectation on the reconstructed data.

[0094] Train the RBM model using the complete EEG data and the complete peripheral data.

[0095] Step S303: Use the trained RBM model to predict the missing data in the incomplete EEG data through the features of the hidden nodes.

[0096] Input the complete EEG data and the complete peripheral data of the visible nodes into the RBM model, calculate the state of the hidden nodes through the weights and activation functions, and reconstruct the data of the visible nodes through the weights and activation functions according to the state of the hidden nodes, where the state of the hidden nodes is the extracted latent feature.

[0097] Step S304: Randomly initialize the missing data in the incomplete EEG data, and use the RBM model to generate the missing EEG data. Complement the generated missing EEG data into the incomplete EEG data to form complete EEG data. Among them, the random initialization can fill the missing part with the mean value of the known data.

[0098] Then update the multimodal data according to the following equation:

[0099]

[0100] where t is the number of iterations, l represents the learning rate, v i is the i-th missing value of the sample, represents the conditional probability of the missing value given the hidden nodes. In addition, F is the energy function that satisfies the following formula:

[0101]

[0102] where, V E represents the EEG data, and V P represents the peripheral data. θ is the parameter of the model, and Σ h represents all possible hidden nodes.

[0103] Step S4: Cut the updated complete EEG data through a sliding window to obtain complete EEG data segments; and match the RASS score as a label with each segment of the complete EEG data segment on the time axis to form labeled EEG data;

[0104] This step performs data segmentation and tagging. Specifically, as Figure 4 shown in the process of data segmentation and tagging, step S4 specifically includes the following steps S401 to S403:

[0105] Step S401, use a sliding window to cut the complete EEG data into segments to obtain complete EEG data segments.

[0106] Set the window length to w l , and set the window sliding step size to w s . The complete EEG data is data in time series form, and each segment of the complete EEG data segment after cutting is also data in time series form.

[0107] Step S402, align the discrete peripheral data with the complete EEG data segments by means of data interpolation. Expressed as:

[0108]

[0109] where y is the value to be interpolated, x is the position to insert the interpolated value, (x 1 , y 1 ) and (x 2 , y 2 ) are the valid data points before and after the invalid or noisy data respectively.

[0110] The peripheral data is data in discrete form. The formed interpolated EEG data is also in the form of data segments.

[0111] Step S403, match the RASS scores evaluated by medical staff with each data segment of the interpolated EEG data to form tagged EEG data.

[0112] Specifically, it is to match the time points of the RASS scores with the time points of the peripheral data, and make each segment of data have a corresponding RASS score (tag value).

[0113] Step S5: Based on the multi-modal network model, use the tagged EEG data for training.

[0114] Specifically, utilize the complementarity of multi-modal data, fuse different types of information through the multi-modal learning framework, and improve the performance and generalization ability of the model. Step S5 includes the following steps S501 to S503:

[0115] Step S501, use the WaveNet model to train the tagged EEG signals, tagged HR signals, tagged SpO2 signals, and tagged BP signals.

[0116] The present invention uses the basic components of the WaveNet model (a time series prediction model) to process the complete EEG data. The input of this model is the complete EEG data, which is then rectified into a time-by-channel tensor The signal X of each channel C can be expressed as:

[0117] X C ={x c,1 ,x c,2 ,…,x c,T}, c = 1, 2, …, C

[0118] The input signal is processed through multiple layers of dilated convolution and residual concatenation. Each layer of dilated convolution enables the network to capture features at different scales, thereby improving the expressiveness of the model. Then, the model processes the input data from different channels separately. For the input X of a certain channel C , the calculation formula of convolution is:

[0119]

[0120] where K is the size of the convolution kernel, ω k is the weight of the convolution kernel, and x c,t-k is the input value in X C .

[0121] WaveNet is a Neural Vocoder architecture proposed by Google DeepMind in 2016. The main body of the model is a probability model based on dilated causal convolution

[0122] Next, the outputs of each channel are combined through a concurrency layer to form a composite feature vector. Then, the composite vector passes through a dense layer and uses the ReLU activation function to output multiple features. The mathematical expression of the gated activation unit is as follows:

[0123] z c (t) = tanh(W f *y c (t) + b f ) ⊙ σ(W g *y c (t) + b g )

[0124] where tanh is the hyperbolic tangent activation function. σ is the Sigmoid activation function. W f and W g are the weights of the convolution kernel, and b f and b g are the bias terms. ⊙ represents element-wise multiplication

[0125] The composite feature vector passing through the synergy layer is as follows:

[0126] Z = Flatten([Concat(Z 1 , Z 2 , …, Z C )])

[0127] Among them, Concat is the concatenation operation in the channel dimension, and Flatten flattens the concatenated tensor into a one-dimensional vector.

[0128] The output z c (t) of the gated activation unit is used to generate skip connections and residual connections. The expression of the residual connection is:

[0129] r c (t) = y c (t) + W r * z c (t)

[0130] Among them, W r is the weight of the residual convolution kernel.

[0131] Finally, the fully connected layer with the softmax activation function outputs the probability distributions of different classes.

[0132]

[0133] Among them, K is the number of classes. W k and b k are the weights and bias terms of the softmax layer.

[0134] Furthermore, the Rectified Linear Unit (ReLU) is used as the activation function, and its expression is:

[0135] f(x) = max(0, x)

[0136] This equation means that if the input x is positive, the output is x; if x is negative, the output is 0.

[0137] The Kullback-Leibler Divergence Loss (KLDivLoss) is used as an indicator to measure the performance of the model, and its expression is:

[0138]

[0139] represents the difference degree between a probability distribution Q and a reference probability distribution P.

[0140] Step S502: Use a convolutional neural network model to train the EEG spectrogram generated in step S2.

[0141] Specifically, first divide the data into a training set, a validation set, and a test set, and then use a convolutional neural network to extract the time-frequency features of the spectrogram and classify them. Use the cross-entropy loss function to calculate the error between the predicted class probability and the true class, and then optimize the model. Calculate the loss batch by batch and update the weights. After each training epoch, evaluate the model performance using the validation set to monitor overfitting.

[0142] Step S503: Based on the multimodal network model, perform weighted fusion on the WaveNet model trained in step S501 and the convolutional neural network model trained in step S502 to obtain the final sedation state evaluation model.

[0143] Specifically, as Figure 5 shown in the multimodal feature fusion idea diagram.

[0144] Let be the instance matrix of the first modality, be the instance matrix of the second modality, be the instance matrix of the i-th modality. Here, N is the number of instances, and d i are the dimensions of the features extracted from each modality respectively. To non-linearly transform the original features of the modalities, the present invention constructs deep neural networks for these two modalities as follows:

[0145] O 1 = f 1 (X 1 ; W 1 )

[0146] O 2 = f 2 (X 2 ; W 2 )

[0147] O i = f i (X i ; W i )

[0148] where, W i represents all the parameters of the non-linear transformation, is the output of the neural network, and d represents the output dimension.

[0149]

[0150] For weighted sum fusion, the present invention initializes the hyperparameter α i , finds the optimal value of the weight, and fuses different modalities as follows:

[0151]

[0152] Step S6: Use the trained sedation state assessment model to predict the RASS score for the unlabeled data.

[0153] This step is the result evaluation step. The unlabeled data are the collected original EEG data, HR data, SpO2 data, and BP data, for which the RASS score is to be determined.

[0154] Specifically, it is to use the trained sedation state assessment model for the newly collected patient data, output the probability distribution, select the category corresponding to the maximum probability as the prediction result, and compare the prediction result of the model with the RASS score given by the expert to evaluate the prediction accuracy. As shown in the following formula:

[0155]

[0156] Under the action of the method for evaluating the sedation state of patients in the intensive care unit based on multimodal physiological signals, as Figure 6 and Figure 7 shown, the present invention can effectively and accurately evaluate the RASS score of critically ill patients.

[0157] A method for evaluating the sedation state of critically ill patients based on a multimodal network model provided by the present invention can simultaneously monitor the sedation depth, identify changes in the sedation level, meet the needs of ICU medical staff to master the sedation level of patients, provide a reference for the subsequent infusion rate, and thus provide the most suitable sedation state for critically ill patients, avoiding over-sedation or under-sedation. The method according to the present invention makes it possible to form and verify the sedation level of patients especially based on the automated analysis of patient physiological activity parameters such as EEG, HR, and BP. More particularly, a single-modal signal is difficult to comprehensively describe the patient's consciousness state, and the real-time evolution of such multi-parameters makes it possible to comprehensively describe the consciousness state. By performing three-dimensional monitoring, real-time analysis and formulating personalized treatment plans for ICU patients through the present invention, the RASS score is quantified so that patients can better adapt to the environment of the intensive care unit.

[0158] The present invention can integrate the data collected by various bedside monitoring devices in the ICU, perform special weighted fusion on the features of each modality at the same time, has the function of monitoring the sedation state of patients, can complete the missing data, and can predict the sedation level of patients, realizing the safe monitoring and early warning of the sedation state of critically ill patients, and having the advantages of short response time and high accuracy.

[0159] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: modifying the technical solutions recorded in the foregoing embodiments, or equivalently replacing some or all of the technical features therein, does not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for evaluating the sedation state of critically ill patients based on a multimodal network model, characterized in that: include: Step S1: acquiring original physiological data of critically ill patients in real time; The physiological data includes EEG data and peripheral data, and the peripheral data includes HR data, SpO2 data, and BP data; Step S2: preprocessing the original EEG data and converting the EEG data into an EEG spectrum; Step S3: generating missing EEG data, filling the generated missing EEG data into the complete EEG data, and updating the complete EEG data; Step S4: cutting the updated complete EEG data through a sliding window to obtain complete EEG data segments; and matching the RASS score with each complete EEG data segment on the time axis as a label to form labeled EEG data; Step S5: training using labeled EEG data based on the multimodal network model; Step S6: Use the trained sedation state assessment model to predict the RASS score of the unlabeled data.

2. The method for evaluating the sedation state of critically ill patients based on a multimodal network model according to claim 1, characterized in that: Step S2 includes the following steps S201 to S203: Step S201: removing abnormal points from the collected original EEG data; Step S202: performing denoising processing on the EEG data; Step S203: converting the denoised EEG data into an EEG spectrum.

3. The method for evaluating the sedation state of critically ill patients based on a multimodal network model according to claim 2, characterized in that: Step S202 includes: decomposing the EEG data into sub-bands of different scales, applying threshold processing on the high-frequency sub-bands, reconstructing the signal after removing noise, then analyzing the wavelet coefficients at each scale, extracting specific patterns, and finally compressing the signal lossily by retaining the low-frequency sub-bands and discarding the high-frequency sub-bands exceeding the threshold; the mathematical expression is: Among them, x(t) represents the input signal, ψ(t) is the mother wavelet, a is the scaling parameter, and b is the translation parameter.

4. The method for evaluating the sedation state of critically ill patients based on a multimodal network model according to claim 2, characterized in that: Step S3 specifically includes the following steps S301 to S304: Step S301: Determine the integrity of EEG data and peripheral data, and obtain complete EEG data, incomplete EEG data, complete peripheral data, and incomplete peripheral data; Step S302: using complete EEG data and complete peripheral data to train the RBM model; Step S303: using the trained RBM model to predict missing data in the incomplete EEG data through the features of hidden nodes; Step S304: randomly initialize the missing data in the incomplete EEG data, generate the missing EEG data using the RBM model, and fill the generated missing EEG data into the incomplete EEG data to form complete EEG data.

5. The method for evaluating the sedation state of critically ill patients based on a multimodal network model according to claim 1, characterized in that: Step S4 includes the following steps S401 to S403: Step S401, using a sliding window to cut the complete EEG data into segments to obtain complete EEG data segments; Step S402, aligning the discrete peripheral data with the complete EEG data segment by means of data interpolation; Step S403, matching the RASS score evaluated by the medical staff with each data segment of the interpolated EEG data to form labeled EEG data.

6. The method for evaluating the sedation state of critically ill patients based on a multimodal network model according to claim 5, characterized in that: Step S402, align the discrete peripheral data with the complete EEG data segment by data interpolation, which is expressed as: Where y is the value to be interpolated, x is the location where the interpolated value is to be inserted, and (x1,y1) and (x2,y2) are valid data points before and after the invalid or noisy data, respectively.

7. The method for evaluating the sedation state of critically ill patients based on a multimodal network model according to claim 5, characterized in that: Step S5 includes the following steps S501 to S503: Step S501: using the WaveNet model to train the labeled EEG signal, the labeled HR signal, the labeled SpO2 signal and the labeled BP signal; Step S502: using a convolutional neural network model to train the EEG spectrogram generated in step S2; Step S503: Based on the multimodal network model, the WaveNet model trained in step S501 and the convolutional neural network model trained in step S502 are weightedly fused to obtain a final sedation state assessment model.

8. The method for evaluating the sedation state of critically ill patients based on a multimodal network model according to claim 7, characterized in that: The step S503 includes the following process: make is the instance matrix of the first mode, is the instance matrix of the second mode, is the instance matrix of the i-th mode; N is the number of instances, d i They are the dimensions of the features extracted from each modality; In order to nonlinearly transform the original features of the modalities, deep neural networks are constructed for these two modalities as shown below: O1=f1(X1;W1) O2=f2(X2;W2) O i =f i (X i ;W i ) Among them, W i represents all parameters of nonlinear transformation, is the output of the neural network, and d represents the output dimension; For weighted sum fusion, initialize the hyperparameter α i , find the optimal value of the weights and fuse the different modalities as follows:

Images