Intensive care unit early warning system based on artificial intelligence

Abstract

The invention relates to the technical field of artificial intelligence, and particularly discloses an intensive care unit early warning system based on artificial intelligence, which comprises a data acquisition module, a data processing and embedding module, a modal feature extraction module, a multi-channel fusion module, a risk prediction module and an unplanned self-extubation early warning module, according to the scheme, a multi-modal fusion framework with three fusion strategies is provided, electronic health record data, medical image data and clinical notes of a patient are combined, and the health condition of the patient is comprehensively reflected; a YOLO v3 detection method is used for extracting an area where a patient is located, an L-K optical flow algorithm is used for patient behavior tracking, patient behavior classification and early warning are conducted through a support vector machine, non-contact and non-invasive early warning of unplanned self extubation of the patient is achieved by monitoring video data, and the safety of the patient is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an intensive care unit early warning system based on artificial intelligence. Background Art

[0002] With the development of medical technology and the aging of the population, intensive care units play an increasingly important role in the medical system. Traditional intensive care unit risk prediction methods use a single type of data for risk prediction, such as only using vital signs, laboratory test results or medical images, which cannot fully reflect the patient's health status, resulting in low prediction accuracy. Intensive care units are full of various medical equipment and personnel activities, which interfere with general human motion recognition methods in identifying patients' unplanned self-extubation behaviors. In addition, patients' self-extubation actions may be very subtle or have various forms of expression, resulting in low recognition accuracy. Summary of the invention

[0003] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides an intensive care unit early warning system based on artificial intelligence. The traditional intensive care unit risk prediction method uses a single type of data for risk prediction, such as only using vital signs, laboratory test results or medical images, which cannot fully reflect the patient's health status, resulting in low accuracy of the prediction results. This solution proposes a multimodal fusion framework with three fusion strategies, which combines the patient's electronic health record, medical imaging data and clinical notes to fully reflect the patient's health status, divide the patient into different risk levels, and take different early warning measures for different risk levels; the intensive care unit is full of various medical equipment and personnel activities, which interferes with the general human motion recognition method in identifying the patient's unplanned self-extubation behavior, and the patient's self-extubation action may be very subtle, or the expression form is diverse, resulting in low recognition accuracy. This solution uses the YOLO v3 detection method (You Only Look Once version 3) to extract the patient's area, and uses the LK optical flow algorithm (LK optical flow) developed a spatiotemporal feature for patient behavior tracking, classified and warned patients’ behaviors through support vector machines, and achieved non-contact and non-invasive early warning of unplanned self-extubation by monitoring video data, which helped improve patient safety.

[0004] The technical solution adopted by the present invention is as follows: An artificial intelligence-based intensive care unit early warning system provided by the present invention comprises: a data acquisition module, a data processing and embedding module, a modal feature extraction module, a multi-channel fusion module, a risk prediction module and an unplanned self-extubation early warning module;

[0005] The data acquisition module collects the patient's electronic health records, medical imaging data, and clinical notes, and sends the patient's electronic health records, medical imaging data, and clinical notes to the data processing and embedding module;

[0006] The data processing and embedding module adjusts the size of the medical image, processes the clinical notes using natural language processing technology, embeds the patient's electronic health record and demographic information into the vector using an embedding method, obtains the information of the electronic health record, and sends the adjusted medical image, the processed clinical notes, and the information of the electronic health record to the modality feature extraction module;

[0007] The modality feature extraction module uses a ResNet model (Residual Neural Network) to extract image features from the adjusted medical image to obtain a final medical image feature vector, uses a Doc2Vec model (Document to Vector) as a feature extractor for the processed clinical notes to obtain a feature vector time series, uses an LSTM network (Long Short-Term Memory) to process the feature vector time series to obtain a single feature vector, uses a bidirectional LSTM network to capture information of the electronic health record to obtain a final feature vector of the electronic health record, and sends the final medical image feature vector, the single feature vector, and the final feature vector of the electronic health record to the multi-channel fusion module;

[0008] The multi-channel fusion module uses three fusion strategies to integrate the final medical image feature vector, the single feature vector and the final feature vector of the electronic health record to obtain a fusion feature, and sends the fusion feature to the risk prediction module;

[0009] The risk prediction module inputs the fusion features into the machine learning model for training to obtain a risk score, and divides the patient into different levels according to the risk score. When the patient's risk level is high risk, the unplanned self-extubation warning module is triggered;

[0010] The unplanned self-extubation warning module uses the YOLO v3 detection method to extract regions of interest, develops a spatiotemporal feature for patient behavior tracking by using the LK optical flow algorithm, and uses a support vector machine for patient behavior classification and warning.

[0011] Furthermore, the data collection module constructs multimodal patient data by combining electronic health records, medical imaging data, and clinical notes, using patient ID and admission ID to identify patients, denoted as ,in, Indicates the patient ID, Indicates admission ID, is the patient's demographic information, represents the electronic health record of a patient, represents the medical imaging data of the patient, Represents the patient's clinical notes.

[0012] Furthermore, the data processing and embedding module adjusts the medical image into size and replicated on 3 input channels. Use natural language processing techniques to process clinical notes. Convert sentences into token lists, convert all words to lowercase and tokenize to form word sequences, and remove punctuation. Convert the patient's electronic health record into a collection of time points, observed variables and variable values, and operations performed at specific time points. Convert the patient's electronic health record and demographic information into ,in represents the set of all time points of all event records of a patient, A vector representing the patient's demographic information, and Represents the observed variables and their values ​​at time A collection of Indicates that the patient is at time The set of operations performed are divided into immediate operations and continuous operations. After conversion, three methods, variable embedding, value embedding, and time embedding, are used to embed the patient's electronic health records into vectors.

[0013] Furthermore, the modality feature extraction module uses a ResNet model to extract image features, replaces the original classification head of the ResNet model with a linear layer to generate a feature vector from the output of the convolutional layer, collects medical image features of patients at different timestamps, and performs weighted summation according to the time difference to obtain the final medical image feature vector;

[0014] The Doc2Vec model is used as a feature extractor for text modality to convert clinical notes into feature vector time series, which are processed by an LSTM network and a single feature vector containing the entire sequence information is generated from all hidden states through a maximum pooling operation. The formula used is as follows: ; ;

[0015] In the formula, Indicates the hidden state, represents the LSTM network, represents the time series of feature vectors, represents a single eigenvector, Represents the maximum pooling operation;

[0016] Since the characteristics of electronic health records are highly correlated with the time dimension, a bidirectional LSTM network is used to capture long-term information. The record is represented as , and input the embedded records into the bidirectional LSTM network, extract features through forward propagation and backward propagation, and extract the most important information from the output of the LSTM layer through the maximum pooling operation as the final feature vector of the electronic health record. The formula used is as follows: ; ; ; ;

[0017] In the formula, and They are the forward propagation and backward propagation of the bidirectional LSTM network, represents the number of timestamps in the electronic health record, represents the output vector of the forward propagation of the bidirectional LSTM network, Represents the embedded record, represents the output vector of the bidirectional LSTM network backpropagation, Represents the output vector of the forward propagation at each time point and the output vector of the backpropagation To splice, The final feature vector representing the electronic health record.

[0018] Furthermore, the multi-channel fusion module adopts three fusion strategies to integrate the final medical image feature vector, the single feature vector and the final feature vector of the electronic health record to obtain the fusion feature. The three fusion strategies include early fusion, joint fusion and late fusion.

[0019] Furthermore, the risk prediction module inputs the fusion features into a machine learning model for training to obtain a risk score, and divides the patient into high risk, medium risk and low risk according to the risk score. When the patient's risk level is high risk, the unplanned self-extubation warning module is triggered.

[0020] Furthermore, the unplanned self-extubation warning module includes a data acquisition and preprocessing module, a region of interest detection module, a feature extraction module and a classification module;

[0021] Further, the data acquisition and preprocessing module acquires ICU monitoring video data, clips the ICU monitoring video data into sequence data of fixed length, and processes the sequence data using data enhancement technology;

[0022] Furthermore, the region of interest detection module uses the YOLO v3 model to detect the region where the patient is located. The YOLO v3 model is pre-trained on the ImageNet dataset and then fine-tuned using the ICU monitoring video data to obtain the region where the patient is located;

[0023] Furthermore, the feature extraction module uses the LK optical flow algorithm to analyze the ICU monitoring video data, measure and track the changes in the number and distance of corner points in each video sequence, and construct three types of features: optical flow corner point count feature, trajectory distance feature, and wavelet transform feature to capture and analyze dynamic behaviors in video sequences. The steps are as follows:

[0024] Step A1: Use the Shi-Tomasi corner detection algorithm in the first frame of the video sequence, and then use the LK optical flow algorithm to track the detected corner points from the first frame, and determine whether the corner points are moving corner points based on the position changes of the corner points between adjacent frames. The formula used is as follows: ;

[0025] In the formula, represents the decision threshold, Indicates Frame The position coordinates of the corner points, Indicates Frame The position coordinates of the corner points, Indicates The corner point is Whether the frame is considered as a moving corner;

[0026] Calculate the number of corner points of each frame using the following formula: ;

[0027] In the formula, Indicates The number of corner points of the frame, Represents the total number of corner points in a frame of video sequence, and selects the corner point number feature of the last frame as the optical flow corner point number feature;

[0028] Step A2: Use the number of corner points in the last frame of the video sequence as a reference to calculate the trajectory distance feature and calculate the trajectory length of the corner points between adjacent frames. The formula used is as follows: ;

[0029] In the formula, represents the trajectory length of the corner point between adjacent frames, Indicates the number of corner points in the last frame of the video sequence;

[0030] Taking into account the change in the number of corner points in different frames, the trajectory lengths of the corner points between adjacent frames are averaged to obtain the trajectory distance feature of the current video sequence. The formula used is as follows: ;

[0031] In the formula, represents the trajectory distance feature of the current video sequence, Indicates the total number of frames in the video sequence;

[0032] Step A3: Use wavelet transform to process the optical flow corner point feature to obtain the wavelet transform feature. The formula used is as follows: ;

[0033] In the formula, represents the scale factor, represents the wavelet function, represents the amount of translation, Represents the wavelet transform feature, adding the time dimension to form the spatiotemporal frequency feature. The formula used is as follows: ;

[0034] In the formula, Represents the spatiotemporal frequency characteristics.

[0035] Furthermore, the classification module constructs a spatiotemporal feature vector for patient behavior classification using the following formula: ;

[0036] In the formula, represents the spatiotemporal feature vector, Represents the number of optical flow corner points;

[0037] Given video sequences and Class action, define the feature space as ,in, Indicates The spatiotemporal feature vector of a video sequence, Indicates The labels of the video sequences, and .

[0038] The spatiotemporal feature vector is input into the support vector machine to classify the patient's behavior into three categories: no action, no unplanned self-extubation tendency, and unplanned self-extubation tendency. When the patient is detected to have an unplanned self-extubation tendency, an alarm is issued to notify medical staff to check on the patient's condition.

[0039] The data collection and preprocessing module mentioned in this plan collects ICU surveillance video data in compliance with the provisions of the Personal Information Protection Law and other relevant laws. It is obtained under legal circumstances and is a case of legal use.

[0040] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0041] (1) In view of the problem that traditional ICU risk prediction methods use a single type of data for risk prediction, such as only using vital signs, laboratory test results or medical images, which cannot fully reflect the patient's health status and leads to low prediction accuracy, this scheme proposes a multimodal fusion framework with three fusion strategies, which combines the patient's electronic health records, medical imaging data and clinical notes to fully reflect the patient's health status, divide the patient into different risk levels, and take different early warning measures for different risk levels;

[0042] (2) Intensive care units are filled with various medical equipment and personnel activities, which interferes with the identification of patients' unplanned self-extubation behaviors by general human motion recognition methods. In addition, the patient's self-extubation actions may be very subtle or have various forms of expression, resulting in low recognition accuracy. This scheme uses the YOLO v3 detection method to extract the patient's area, and uses the LK optical flow algorithm to develop a spatiotemporal feature for patient behavior tracking. The support vector machine is used to classify and warn patient behaviors. By monitoring video data, a non-contact, non-invasive early warning of patients' unplanned self-extubation is achieved, which helps to improve patient safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A framework diagram of the unplanned self-extubation warning module provided by the present invention;

[0044] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0046] Embodiment 1, the present invention provides an artificial intelligence-based intensive care unit early warning system, comprising: a data acquisition module, a data processing and embedding module, a modal feature extraction module, a multi-channel fusion module, a risk prediction module and an unplanned self-extubation early warning module;

[0047] The data acquisition module collects the patient's electronic health records, medical imaging data, and clinical notes, and sends the patient's electronic health records, medical imaging data, and clinical notes to the data processing and embedding module;

[0048] The data processing and embedding module adjusts the size of the medical image, processes the clinical notes using natural language processing technology, embeds the patient's electronic health record and demographic information into the vector using an embedding method, obtains the information of the electronic health record, and sends the adjusted medical image, the processed clinical notes, and the information of the electronic health record to the modality feature extraction module;

[0049] The modality feature extraction module uses the ResNet model to extract image features from the adjusted medical image to obtain a final medical image feature vector, uses the Doc2Vec model as a feature extractor for the processed clinical notes to obtain a feature vector time series, uses an LSTM network to process the feature vector time series to obtain a single feature vector, uses a bidirectional LSTM network to capture information of the electronic health record to obtain a final feature vector of the electronic health record, and sends the final medical image feature vector, the single feature vector, and the final feature vector of the electronic health record to the multi-channel fusion module;

[0050] The multi-channel fusion module uses three fusion strategies to integrate the final medical image feature vector, the single feature vector and the final feature vector of the electronic health record to obtain a fusion feature, and sends the fusion feature to the risk prediction module;

[0051] The risk prediction module inputs the fusion features into the machine learning model for training, inputs the patient data into the trained machine learning model to obtain a risk score, and divides the patients into different levels according to the risk score. When the patient's risk level is high risk, the unplanned self-extubation warning module is triggered;

[0052] The unplanned self-extubation warning module uses the YOLO v3 detection method to extract regions of interest, develops a spatiotemporal feature for patient behavior tracking by using the LK optical flow algorithm, and uses a support vector machine for patient behavior classification and warning.

[0053] Embodiment 2: This embodiment is based on the above embodiment. The modality feature extraction module uses the ResNet model to extract image features. The original classification head of the ResNet model is replaced with a linear layer to generate a feature vector from the output of the convolutional layer. The medical image features of the patient at different timestamps are collected and weighted summed according to the time difference to obtain the final medical image feature vector. The formula used is as follows: ;

[0054] In the formula, represents the final medical image feature vector, represents the activation function, Indicates the time of admission, Represents image features;

[0055] The Doc2Vec model is used as a feature extractor for text modality to convert clinical notes into feature vector time series, which are processed by an LSTM network and a single feature vector containing the entire sequence information is generated from all hidden states through a maximum pooling operation. The formula used is as follows: ; ;

[0056] In the formula, Indicates the hidden state, represents the LSTM network, represents the time series of feature vectors, represents a single eigenvector, Represents the maximum pooling operation;

[0057] Since the characteristics of electronic health records are highly correlated with the time dimension, a bidirectional LSTM network is used to capture long-term information. The record is represented as , and input the embedded records into the bidirectional LSTM network, extract features through forward propagation and backward propagation, and extract the most important information from the output of the LSTM layer through the maximum pooling operation as the final feature vector of the electronic health record. The formula used is as follows: ; ; ; ;

[0058] In the formula, and They are the forward propagation and backward propagation of the bidirectional LSTM network, represents the number of timestamps in the electronic health record, represents the output vector of the forward propagation of the bidirectional LSTM network, Represents the embedded record, represents the output vector of the bidirectional LSTM network backpropagation, Represents the output vector of the forward propagation at each time point and the output vector of the backpropagation To splice, The final feature vector representing the electronic health record.

[0059] Embodiment 3: Based on the above embodiment, the multi-channel fusion module adopts three fusion strategies to integrate the final medical image feature vector, the single feature vector and the final feature vector of the electronic health record to obtain fusion features. The three fusion strategies include early fusion, joint fusion and late fusion, which are:

[0060] Early fusion: Before inputting into the classification network, the feature vectors of multiple modalities are directly connected to form a single feature vector. The feature extractor of each modality is trained separately and generates feature vectors for training the classifier. After separate pre-training, the multimodal classifier is trained using a fixed feature extractor.

[0061] Joint fusion: Unlike early fusion, joint fusion trains an end-to-end model that propagates gradients from the classifier to each feature extractor. The network structure is similar to early fusion, but the training strategy is different. It also uses direct connections to construct multimodal feature vectors.

[0062] Late fusion: Train classifiers of different modalities separately and combine their unimodal prediction results into a global multimodal prediction. This strategy is similar to ensemble learning, also known as decision-level fusion. In the implementation, the average is chosen as the method for combining predictions.

[0063] Embodiment 4: This embodiment is based on the above embodiment. The risk prediction module inputs the fusion features into the machine learning model for training to obtain a risk score. The patient is divided into high risk, medium risk and low risk according to the risk score. When the patient's risk level is high risk, the unplanned self-extubation warning module is triggered. The risk scores of different risk levels are as follows:

[0064] 1) Low risk: risk scores between 0 and 0.33, indicating that the patient is less likely to experience adverse clinical outcomes;

[0065] 2) Medium risk: risk scores between 0.33 and 0.66, indicating that the patient has a certain risk of adverse outcomes and requires further monitoring or preventive measures;

[0066] 3) High risk: The risk score is between 0.66 and 1, indicating that the patient has a high probability of adverse clinical outcomes and requires immediate intervention.

[0067] Measures taken for different risk levels:

[0068] 1) Low-risk patients: regular health checks and health education, monitoring of key vital signs, but no frequent medical intervention;

[0069] 2) Medium-risk patients: Increase monitoring frequency, such as daily or weekly monitoring of key vital signs and laboratory indicators, provide preventive medication or nutritional supplements as needed, and consult a doctor regularly to discuss health status and lifestyle adjustments;

[0070] 3) High-risk patients: Trigger the unplanned self-extubation warning module to monitor whether the patient has an unplanned self-extubation tendency. When it is detected that the patient has an unplanned self-extubation tendency, an alarm will be issued immediately to notify medical staff to check on the patient's condition.

[0071] By performing the above operations, the traditional ICU risk prediction method uses a single type of data for risk prediction, such as only using vital signs, laboratory test results or medical images, which cannot fully reflect the patient's health status, resulting in low accuracy of prediction results. This scheme proposes a multimodal fusion framework with three fusion strategies, which combines the patient's electronic health records, medical imaging data and clinical notes to comprehensively reflect the patient's health status, divide the patient into different risk levels, and take different early warning measures for different risk levels.

[0072] Example 5, see Figure 1 ,This embodiment is based on the above embodiment, the unplanned self-extubation warning module includes a data acquisition and preprocessing module, a region of interest detection module, a feature extraction module and a classification module;

[0073] The data acquisition and preprocessing module acquires ICU monitoring video data, clips the ICU monitoring video data into sequence data of fixed length, and processes the sequence data using data enhancement technology;

[0074] The region of interest detection module uses the YOLO v3 model to detect the area where the patient is located. The YOLO v3 model is pre-trained on the ImageNet dataset and then fine-tuned using the ICU monitoring video data to obtain the area where the patient is located;

[0075] The feature extraction module uses the LK optical flow algorithm to analyze the ICU monitoring video data, measure and track the changes in the number and distance of corner points in each video sequence, and construct three types of features: optical flow corner point count feature, trajectory distance feature, and wavelet transform feature to capture and analyze dynamic behaviors in video sequences. The steps are as follows:

[0076] Step A1: Use the Shi-Tomasi corner detection algorithm in the first frame of the video sequence, and then use the LK optical flow algorithm to track the detected corner points from the first frame, and determine whether the corner points are moving corner points based on the position changes of the corner points between adjacent frames. The formula used is as follows: ;

[0077] In the formula, represents the decision threshold, Indicates Frame The position coordinates of the corner points, Indicates Frame The position coordinates of the corner points, Indicates The corner point is Whether the frame is considered as a moving corner;

[0078] Calculate the number of corner points of each frame using the following formula: ;

[0079] In the formula, Indicates The number of corner points of the frame, Represents the total number of corner points in a frame of video sequence, and selects the number of corner points in the last frame as the optical flow corner point feature;

[0080] Step A2: Use the number of corner points in the last frame of the video sequence as a reference to calculate the trajectory distance feature and calculate the trajectory length of the corner points between adjacent frames. The formula used is as follows: ;

[0081] In the formula, represents the trajectory length of the corner point between adjacent frames, Indicates the number of corner points in the last frame of the video sequence;

[0082] Taking into account the change in the number of corner points in different frames, the trajectory lengths of the corner points between adjacent frames are averaged to obtain the trajectory distance feature of the current video sequence. The formula used is as follows: ;

[0083] In the formula, represents the trajectory distance feature of the current video sequence, Indicates the total number of frames in the video sequence;

[0084] Step A3: Use wavelet transform to process the optical flow corner point feature to obtain the wavelet transform feature. The formula used is as follows: ;

[0085] In the formula, represents the scale factor, represents the wavelet function, represents the amount of translation, Represents the wavelet transform feature, adding the time dimension to form the spatiotemporal frequency feature. The formula used is as follows: ;

[0086] In the formula, Represents spatiotemporal frequency characteristics;

[0087] Furthermore, the classification module constructs a spatiotemporal feature vector for patient behavior classification using the following formula: ;

[0088] In the formula, represents the spatiotemporal feature vector, Represents the number of optical flow corner points;

[0089] Given video sequences and Class action, define the feature space as ,in, Indicates The spatiotemporal feature vector of a video sequence, Indicates The labels of the video sequences, and .

[0090] The spatiotemporal feature vector is input into the support vector machine to classify the patient's behavior into three categories: no action, no unplanned self-extubation tendency, and unplanned self-extubation tendency. When the patient is detected to have an unplanned self-extubation tendency, an alarm is issued to notify medical staff to check on the patient's condition.

[0091] By performing the above operations, the intensive care unit is full of various medical equipment and personnel activities, which interferes with the general human motion recognition method in identifying the patient's unplanned self-extubation behavior. The patient's self-extubation action may be very subtle or have various forms of expression, resulting in low recognition accuracy. This scheme uses the YOLO v3 detection method to extract the patient's area, and uses the LK optical flow algorithm to develop a spatiotemporal feature for patient behavior tracking. The support vector machine is used to classify and warn patient behavior. By monitoring video data, a non-contact, non-invasive early warning of unplanned self-extubation of patients is achieved, which helps to improve patient safety.

[0092] Embodiment 6, based on the above embodiment, the data acquisition module combines electronic health records, medical imaging data and clinical notes, uses patient ID and admission ID to identify patients, and constructs multimodal patient data, which is recorded as ,in, Indicates the patient ID, Indicates admission ID, is the patient's demographic information, represents the electronic health record of a patient, represents the medical imaging data of the patient, Represents the patient's clinical notes, where the patient's demographic information includes the patient's age, gender, etc. All of this information becomes categorical data after processing; the patient's electronic health record consists of three types of events, including chart events, laboratory tests, and procedural events. These events are a collection of variables with timestamps, and the variable values ​​can be numerical or categorical. The classification of procedural events: divided into immediate operations and continuous operations. Immediate operations represent operations performed at a specific point in time, while continuous operations represent operations that continue for a period of time.

[0093] Example 7, based on the above example, in the unplanned self-extubation warning module, the data acquisition and preprocessing module collects ICU monitoring video data, the video length ranges from 4 minutes to 10 minutes, and the frame rate is , the frame size is Pixel, in order to be able to provide early warning of patients’ unplanned self-extubation tendencies, healthcare professionals predefine actions in three stages:

[0094] 1) Extubation tendency stage: including the patient's physical discomfort and subtle body movements, which usually manifest 2-3 minutes before extubation occurs;

[0095] 2) Deliberate extubation phase: the patient's tactile behaviors, which occur approximately 10 seconds before the extubation action;

[0096] 3) Extubation phase: the moment when the patient is actually extubated unplanned.

[0097] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0098] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. An artificial intelligence-based intensive care unit early warning system, characterized in that: It includes data acquisition module, data processing and embedding module, modal feature extraction module, multi-channel fusion module, risk prediction module and unplanned self-extubation warning module; The data acquisition module collects the patient's electronic health records, medical imaging data, and clinical notes, and sends the electronic health records, medical imaging data, and clinical notes to the data processing and embedding module; The data processing and embedding module adjusts the size of the medical image, processes the clinical notes using natural language processing technology, embeds the electronic health record and demographic information into the vector using an embedding method to obtain the information of the electronic health record, and sends the adjusted medical image, the processed clinical notes, and the information of the electronic health record to the modality feature extraction module; The modality feature extraction module uses the ResNet model to extract image features from the adjusted medical image to obtain a final medical image feature vector, uses the Doc2Vec model as a feature extractor for the processed clinical notes to obtain a feature vector time series, uses an LSTM network to process the feature vector time series to obtain a single feature vector, uses a bidirectional LSTM network to capture information of the electronic health record to obtain a final feature vector of the electronic health record, and sends the final medical image feature vector, the single feature vector, and the final feature vector of the electronic health record to the multi-channel fusion module; The multi-channel fusion module uses three fusion strategies to integrate the final medical image feature vector, the single feature vector and the final feature vector of the electronic health record to obtain a fusion feature, and sends the fusion feature to the risk prediction module; The risk prediction module inputs the fusion features into the machine learning model for training to obtain a risk score, and divides the patient into different levels according to the risk score. When the patient's risk level is high risk, the unplanned self-extubation warning module is triggered; The unplanned self-extubation warning module uses the YOLO v3 detection method to extract regions of interest, develops a spatiotemporal feature for patient behavior tracking by using the LK optical flow algorithm, and uses a support vector machine for patient behavior classification and warning.

2. The artificial intelligence-based intensive care unit early warning system according to claim 1, characterized in that: The modality feature extraction module uses the ResNet model to extract image features, replaces the original classification head of the ResNet model with a linear layer, generates a feature vector from the output of the convolutional layer, collects medical image features of patients at different timestamps, and performs weighted summation according to the time difference to obtain the final medical image feature vector; The information of the electronic health record is input into the bidirectional LSTM network, features are extracted through forward propagation and backward propagation, and information is extracted through the maximum pooling operation as the final feature vector of the electronic health record.

3. The artificial intelligence-based intensive care unit early warning system according to claim 1, characterized in that: The unplanned self-extubation warning module includes a data acquisition and preprocessing module, a region of interest detection module, a feature extraction module and a classification module; The data acquisition and preprocessing module acquires ICU monitoring video data, clips the ICU monitoring video data into sequence data of fixed length, and processes the sequence data using data enhancement technology; The region of interest detection module uses the YOLO v3 model to detect the area where the patient is located. The YOLO v3 model is pre-trained on the ImageNet dataset and then fine-tuned using the ICU monitoring video data to obtain the area where the patient is located; The feature extraction module uses the LK optical flow algorithm to analyze the ICU monitoring video data, measure and track the changes in the number and distance of corner points in each video sequence, and construct three types of features: optical flow corner point count feature, trajectory distance feature, and wavelet transform feature; The classification module constructs a spatiotemporal feature vector for patient behavior classification, inputs the spatiotemporal feature vector into a support vector machine to classify patient behavior, and when it is detected that the patient has an unplanned tendency to self-extubate, an alarm is issued to notify medical staff to check on the patient's condition.

4. The artificial intelligence-based intensive care unit early warning system according to claim 3, characterized in that: The feature extraction module uses the LK optical flow algorithm to analyze the ICU monitoring video data, measure and track the changes in the number and distance of corner points in each video sequence, and construct three types of features. The steps are as follows: Step A1: After using the Shi-Tomasi corner detection algorithm on the first frame of the video sequence, use the LK optical flow algorithm to track the detected corner points starting from the first frame, determine whether the corner points are moving corner points based on the position changes of the corner points between adjacent frames, calculate the corner point quantity feature of each frame, and select the corner point quantity feature of the last frame as the optical flow corner point count feature; Step A2: Use the corner point quantity feature of the last frame of the video sequence as a reference to calculate the trajectory distance feature, calculate the trajectory length of the corner point between adjacent frames, and average them to obtain the trajectory distance feature of the current video sequence; Step A3: Use wavelet transform to process the optical flow corner point count features to obtain wavelet transform features, and add the time dimension to obtain the spatiotemporal frequency features.

Images