System and method for achieving ICU patient pain scoring, processor and computer readable storage medium thereof

Abstract

The invention relates to a system for realizing pain scoring of an ICU (intensive care unit) patient. The system comprises a data acquisition module for acquiring a facial micro-expression video of the ICU patient in real time by using a high-definition camera; the micro-expression recognition module analyzes the collected micro-expression video image; the vital sign analysis module analyzes the collected vital sign data in real time; the pain scoring model is obtained by training through a machine learning algorithm, and the pain degree of the patient is quantitatively scored; and the result output module outputs the pain scoring result to the medical staff in real time. According to the system and method for achieving ICU patient pain scoring, the processor and the computer readable storage medium of the system and method, the objectivity and accuracy of pain scoring are improved through the artificial intelligence technology. By analyzing the combination of micro-expressions and vital signs, the sensitivity and specificity of pain assessment are improved. Real-time monitoring and continuous evaluation of pain scores are realized, and timely adjustment of a treatment scheme is facilitated.

Description

Technical Field

[0001] The present invention relates to the field of medical systems, in particular to the field of pain management, and specifically refers to a system, method, processor and computer-readable storage medium thereof for implementing pain scoring for ICU patients. Background Art

[0002] In the intensive care unit (ICU), pain management is essential for the overall treatment and recovery of patients. Traditional pain assessment methods rely on patient self-reporting, but many ICU patients are unable to communicate effectively due to their condition or treatment (such as mechanical ventilation), which makes pain assessment difficult. In addition, pain is a subjective experience, and different patients may have different pain perceptions and expressions for the same stimulus. Therefore, there is an urgent need for an objective and accurate pain assessment tool to improve pain management in ICU patients.

[0003] Currently, commonly used pain assessment tools in ICU include the Numerical Pain Rating (NRS), Critical Care Pain Observation Tool (CPOT), etc. Although these tools are helpful for pain assessment to a certain extent, they still have limitations. For example, NRS requires patients to be able to understand and express numbers, while CPOT relies on the observer's subjective interpretation of the patient's behavior, which may affect the accuracy and consistency of the assessment. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a system, method, processor and computer-readable storage medium thereof for implementing ICU patient pain scoring with high accuracy, good consistency and a wide range of applications.

[0005] In order to achieve the above objectives, the system, method, processor and computer-readable storage medium for implementing ICU patient pain scoring of the present invention are as follows:

[0006] The main features of the system for implementing pain scoring for ICU patients are that the system comprises:

[0007] The data acquisition module is used to use a high-definition camera to collect real-time facial micro-expression videos of ICU patients and simultaneously collect the patients' main vital signs data;

[0008] A micro-expression recognition module, connected to the data acquisition module, for analyzing the collected micro-expression video images using a convolutional neural network to identify micro-expression features related to pain;

[0009] A vital sign analysis module, connected to the data acquisition module, for real-time analysis of the collected vital sign data and extraction of changes in physiological parameters related to pain;

[0010] A pain scoring model is connected to the micro-expression recognition module and the vital signs analysis module, and is used to quantitatively score the patient's pain level by combining the micro-expression recognition results and the vital signs analysis results with a pain scoring model trained by a machine learning algorithm;

[0011] A result output module, connected to the pain scoring model, is used to output the pain scoring results to medical staff in real time;

[0012] A clinical validation feedback module, connected to the pain scoring model and result output module, for clinically testing the system in an ICU environment, collecting feedback from medical staff and patients, and adjusting model parameters and user interface design based on clinical feedback;

[0013] The medical record interaction module is connected to the data acquisition module and is used to interact with the electronic medical record system of critically ill patients to obtain the patient's condition.

[0014] Preferably, the micro-expression recognition module uses a convolutional neural network to analyze micro-expression video images and recognize micro-expression features, specifically comprising the following steps:

[0015] Perform video preprocessing; use face detection algorithm to locate faces in images; locate feature points; extract local features and deep features; train CNN model and classify micro-expressions.

[0016] Preferably, the vital signs analysis module performs real-time analysis on the collected vital signs data to extract changes in physiological parameters related to pain; preprocesses the data to remove outliers and noise, fill in missing values, and standardize all vital signs data; calculates statistics from the vital signs time series; evaluates the importance of features through machine learning algorithms, and selects the features with the greatest contribution and patterns related to pain.

[0017] Preferably, in the process of training the pain scoring model, the intensive care pain observation tool CPOT is used to quantitatively score the patient's pain level, which specifically includes the following steps:

[0018] Patients were scored with CPOT and synchronized; data were preprocessed; pre-trained models were fine-tuned based on CPOT scores; pain scores predicted by the model were compared with CPOT scores to assess consistency;

[0019] The pain scoring model also processes facial landmark data through a graph neural network, specifically including the following steps:

[0020] Prepare data; extract node features and edge features, and construct a graph; use a graph attention network model to learn edge features; apply a graph convolutional layer to update the feature representation of the node and apply a nonlinear activation function; use graph pooling technology to aggregate the information of the entire graph to obtain a global feature representation; identify pain features; optimize model parameters; and evaluate and adjust the model.

[0021] The method for implementing ICU patient pain scoring based on the system described in claim 1 is mainly characterized in that the method comprises the following steps:

[0022] (1) Perform patient identification, use a high-definition camera to collect real-time facial micro-expression videos of ICU patients, and proceed to step (2); at the same time, collect the patient's main vital signs data, and proceed to step (3);

[0023] (2) using a convolutional neural network to analyze the collected micro-expression video images and identify micro-expression features related to pain; if the vital sign data is not processed, proceed to step (3); otherwise, proceed to step (4);

[0024] (3) Analyze the collected vital sign data in real time to extract changes in physiological parameters related to pain; if the facial micro-expression video is not processed, continue with step (2); otherwise, continue with step (4);

[0025] (4) Combining the micro-expression recognition results and vital sign analysis results, a pain scoring model is trained using a machine learning algorithm to quantitatively score the patient's pain level;

[0026] (5) The pain score results are output to medical staff in real time.

[0027] Preferably, the step (2) specifically comprises the following steps:

[0028] (2.1) Perform video preprocessing;

[0029] (2.2) Use face detection algorithm to locate the face in the image;

[0030] (2.3) Positioning feature points;

[0031] (2.4) Extract local features and deep features;

[0032] (2.5) Train the CNN model and classify micro-expressions.

[0033] Preferably, the step (3) specifically comprises the following steps:

[0034] (3.1) Analyze the collected vital signs data in real time and extract changes in physiological parameters related to pain;

[0035] (3.2) Preprocess the data to remove outliers and noise, fill in missing values, and standardize all vital sign data;

[0036] (3.3) Calculate statistics from vital sign time series;

[0037] (3.4) Evaluate the importance of features through machine learning algorithms and select the most important features and pain-related patterns;

[0038] The step (3) further includes the step of processing the facial landmark data by means of a graph neural network, which specifically includes the following processing steps:

[0039] (1-3.1) Prepare data;

[0040] (1-3.2) Extract node features and edge features and construct a graph;

[0041] (1-3.3) Use the graph attention network model to learn edge features;

[0042] (1-3.4) Apply graph convolutional layers to update the feature representation of nodes and apply nonlinear activation functions;

[0043] (1-3.5) Use graph pooling technology to aggregate the information of the entire graph and obtain a global feature representation;

[0044] (1-3.6) Identify pain characteristics;

[0045] (1-3.7) Optimize model parameters;

[0046] (1-3.8) Perform model evaluation and adjustment.

[0047] The main feature of the device for implementing pain scoring of ICU patients is that the device comprises:

[0048] a processor configured to execute computer-executable instructions;

[0049] The memory stores one or more computer executable instructions. When the computer executable instructions are executed by the processor, the various steps of the above-mentioned method for implementing the pain score of ICU patients are implemented.

[0050] The main feature of the processor for implementing ICU patient pain scoring is that the processor is configured to execute computer executable instructions, and when the computer executable instructions are executed by the processor, the various steps of the above-mentioned method for implementing ICU patient pain scoring are implemented.

[0051] The main feature of the computer-readable storage medium is that a computer program is stored thereon, and the computer program can be executed by a processor to implement the various steps of the above-mentioned method for achieving ICU patient pain scoring.

[0052] The system, method, processor and computer-readable storage medium for implementing ICU patient pain scoring of the present invention are adopted, without the need for patient self-reporting, and are applicable to all ICU patients, including intubated patients who cannot communicate. Artificial intelligence technology is used to improve the objectivity and accuracy of pain scoring. The sensitivity and specificity of pain assessment are improved by analyzing the combination of micro-expressions and vital signs. Real-time monitoring and continuous evaluation of pain scores are achieved, which helps to adjust treatment plans in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a schematic diagram of the structure of the system for implementing pain scoring for ICU patients according to the present invention.

[0054] Figure 2 This is a schematic diagram of a data acquisition module of a system for implementing pain scoring for ICU patients according to the present invention.

[0055] Figure 3 The present invention is a schematic diagram of a micro-expression recognition module of a system for implementing pain scoring for ICU patients.

[0056] Figure 4 This is a schematic diagram of a vital sign analysis module of a system for implementing pain scoring for ICU patients according to the present invention.

[0057] Figure 5 The present invention is a schematic diagram of a pain scoring model of a system for implementing pain scoring for ICU patients.

[0058] Figure 6 This is a schematic diagram of a result output module of the system for implementing pain scoring for ICU patients of the present invention. DETAILED DESCRIPTION

[0059] In order to more clearly describe the technical content of the present invention, further description is given below in conjunction with specific embodiments.

[0060] The system for implementing ICU patient pain scoring of the present invention comprises:

[0061] The data acquisition module is used to use a high-definition camera to collect real-time facial micro-expression videos of ICU patients and simultaneously collect the patients' main vital signs data;

[0062] A micro-expression recognition module, connected to the data acquisition module, for analyzing the collected micro-expression video images using a convolutional neural network to identify micro-expression features related to pain;

[0063] A vital sign analysis module, connected to the data acquisition module, for real-time analysis of the collected vital sign data and extraction of changes in physiological parameters related to pain;

[0064] A pain scoring model is connected to the micro-expression recognition module and the vital signs analysis module, and is used to quantitatively score the patient's pain level by combining the micro-expression recognition results and the vital signs analysis results with a pain scoring model trained by a machine learning algorithm;

[0065] A result output module, connected to the pain scoring model, is used to output the pain scoring results to medical staff in real time;

[0066] A clinical validation feedback module, connected to the pain scoring model and result output module, for clinically testing the system in an ICU environment, collecting feedback from medical staff and patients, and adjusting model parameters and user interface design based on clinical feedback;

[0067] The medical record interaction module is connected to the data acquisition module and is used to interact with the electronic medical record system of critically ill patients to obtain the patient's condition.

[0068] As a preferred embodiment of the present invention, the micro-expression recognition module uses a convolutional neural network to analyze micro-expression video images and recognize micro-expression features, specifically comprising the following steps:

[0069] Perform video preprocessing; use face detection algorithm to locate faces in images; locate feature points; extract local features and deep features; train CNN model and classify micro-expressions.

[0070] As a preferred embodiment of the present invention, the vital signs analysis module performs real-time analysis on the collected vital signs data, extracts changes in physiological parameters related to pain; preprocesses the data, removes outliers and noise, fills in missing values, and standardizes all vital signs data; calculates statistics from the vital signs time series; evaluates the importance of features through machine learning algorithms, and selects the features with the greatest contribution and patterns related to pain.

[0071] As a preferred embodiment of the present invention, the pain scoring model uses the intensive care pain observation tool CPOT to quantitatively score the patient's pain level during the training of the pain scoring model, which specifically includes the following steps:

[0072] Patients were scored with CPOT and synchronized; data were preprocessed; pre-trained models were fine-tuned based on CPOT scores; pain scores predicted by the model were compared with CPOT scores to assess consistency;

[0073] The pain scoring model also processes facial landmark data through a graph neural network, specifically including the following steps:

[0074] Prepare data; extract node features and edge features, and construct a graph; use a graph attention network model to learn edge features; apply a graph convolutional layer to update the feature representation of the node and apply a nonlinear activation function; use graph pooling technology to aggregate the information of the entire graph to obtain a global feature representation; identify pain features; optimize model parameters; and evaluate and adjust the model.

[0075] The method of implementing pain scoring of ICU patients based on the system of claim 1 of the present invention, wherein the method comprises the following steps:

[0076] (1) Perform patient identification, use a high-definition camera to collect real-time facial micro-expression videos of ICU patients, and proceed to step (2); at the same time, collect the patient's main vital signs data, and proceed to step (3);

[0077] (2) using a convolutional neural network to analyze the collected micro-expression video images and identify micro-expression features related to pain; if the vital sign data is not processed, proceed to step (3); otherwise, proceed to step (4);

[0078] (3) Analyze the collected vital sign data in real time to extract changes in physiological parameters related to pain; if the facial micro-expression video is not processed, continue with step (2); otherwise, continue with step (4);

[0079] (4) Combining the micro-expression recognition results and vital sign analysis results, a pain scoring model is trained using a machine learning algorithm to quantitatively score the patient's pain level;

[0080] (5) The pain score results are output to medical staff in real time.

[0081] As a preferred embodiment of the present invention, the step (2) specifically comprises the following steps:

[0082] (2.1) Perform video preprocessing;

[0083] (2.2) Use face detection algorithm to locate the face in the image;

[0084] (2.3) Positioning feature points;

[0085] (2.4) Extract local features and deep features;

[0086] (2.5) Train the CNN model and classify micro-expressions.

[0087] As a preferred embodiment of the present invention, the step (3) specifically comprises the following steps:

[0088] (3.1) Analyze the collected vital signs data in real time and extract changes in physiological parameters related to pain;

[0089] (3.2) Preprocess the data to remove outliers and noise, fill in missing values, and standardize all vital sign data;

[0090] (3.3) Calculate statistics from vital sign time series;

[0091] (3.4) Evaluate the importance of features through machine learning algorithms and select the most important features and pain-related patterns;

[0092] The step (3) further includes the step of processing the facial landmark data by means of a graph neural network, which specifically includes the following processing steps:

[0093] (1-3.1) Prepare data;

[0094] (1-3.2) Extract node features and edge features and construct a graph;

[0095] (1-3.3) Use the graph attention network model to learn edge features;

[0096] (1-3.4) Apply graph convolutional layers to update the feature representation of nodes and apply nonlinear activation functions;

[0097] (1-3.5) Use graph pooling technology to aggregate the information of the entire graph and obtain a global feature representation;

[0098] (1-3.6) Identify pain characteristics;

[0099] (1-3.7) Optimize model parameters;

[0100] (1-3.8) Perform model evaluation and adjustment.

[0101] The device for implementing ICU patient pain scoring of the present invention, wherein the device comprises:

[0102] a processor configured to execute computer-executable instructions;

[0103] The memory stores one or more computer executable instructions. When the computer executable instructions are executed by the processor, the various steps of the above-mentioned method for implementing the pain score of ICU patients are implemented.

[0104] The processor for implementing ICU patient pain scoring of the present invention, wherein the processor is configured to execute computer executable instructions, and when the computer executable instructions are executed by the processor, the various steps of the above-mentioned method for implementing ICU patient pain scoring are implemented.

[0105] The computer-readable storage medium of the present invention stores a computer program thereon, and the computer program can be executed by a processor to implement the various steps of the above-mentioned method for achieving ICU patient pain scoring.

[0106] The data acquisition module is used to use a high-definition camera to collect real-time facial micro-expression videos of ICU patients and simultaneously collect the patients' main vital signs data;

[0107] A micro-expression recognition module, connected to the data acquisition module, for analyzing the collected micro-expression video images using a convolutional neural network to identify micro-expression features related to pain;

[0108] A vital sign analysis module, connected to the data acquisition module, for real-time analysis of the collected vital sign data and extraction of changes in physiological parameters related to pain;

[0109] A pain scoring model is connected to the micro-expression recognition module and the vital signs analysis module, and is used to quantitatively score the patient's pain level by combining the micro-expression recognition results and the vital signs analysis results with a pain scoring model trained by a machine learning algorithm;

[0110] The result output module is connected to the pain scoring model and is used to output the pain scoring results to medical staff in real time.

[0111] As a preferred embodiment of the present invention, the system also includes a clinical verification feedback module, which is connected to the pain scoring model and result output module, and is used to perform clinical testing on the system in an ICU environment, collect feedback from medical staff and patients, and adjust model parameters and user interface design based on clinical feedback.

[0112] As a preferred embodiment of the present invention, the micro-expression recognition module uses a pre-trained model to fine-tune the specific pain expressions of ICU patients; a machine learning algorithm is used to identify pain-related micro-expression features;

[0113] As a preferred embodiment of the present invention, the vital signs analysis module uses a machine learning algorithm to perform feature selection and pattern recognition on vital signs data to discover the relationship between pain and physiological parameters.

[0114] As a preferred embodiment of the present invention, the pain scoring model adopts cross-validation and hyperparameter optimization technology to perform model training; uses a graph neural network to process facial landmark data and capture dynamic changes in facial expressions.

[0115] As a preferred embodiment of the present invention, the system further includes a medical record interaction module, which is connected to the data acquisition module and is used to interact with the electronic medical record system of critically ill patients to obtain the patient's condition.

[0116] In a specific embodiment of the present invention, the patient's pain level and consciousness level are evaluated in real time, a large-scale clinical facial expression research video data set is constructed, and the automatic acquisition and screening and annotation process and system of the video data set are realized. Based on the latest deep learning technology, the expression features are extracted using the pre-trained model of the large expression data set and input into the time series model. Through the self-attention mechanism, the important video frame features in the time series are learned and the time series features are extracted, and finally the pain and consciousness level scores of the corresponding patients in the video are predicted. At the same time, the system designs an evaluation and optimization mechanism for the prediction effect.

[0117] The present invention comprises the following structure:

[0118] 1. Data acquisition module:

[0119] High-definition cameras are used to capture real-time facial micro-expression videos of ICU patients, ensuring that the image clarity is sufficient to capture subtle muscle movements and changes in facial expressions; the patient's main vital signs data, including heart rate, blood pressure, respiratory rate, and blood oxygen saturation, are simultaneously collected. These data are of great value for assessing pain status.

[0120] 2. Micro-expression recognition module:

[0121] Deep learning technology, especially convolutional neural network (CNN), is used to analyze the collected micro-expression video images to identify micro-expression features related to pain. Through transfer learning technology, a model pre-trained on large-scale face recognition tasks is used to fine-tune the specific pain expressions of ICU patients to improve recognition accuracy.

[0122] The micro-expression recognition module uses a convolutional neural network (CNN) to analyze micro-expression video images and identify micro-expression features. The following are the specific steps of this module:

[0123] 1) Video preprocessing

[0124] Frame extraction: Extract consecutive frames from a video and convert the video into a series of static images.

[0125] Image quality enhancement: Perform noise reduction, contrast enhancement and other processing on the image to improve the accuracy of subsequent feature extraction.

[0126] Image normalization: resize the image to a uniform size to meet the input requirements of the CNN model.

[0127] 2) Face Detection

[0128] Face localization: Use face detection algorithms (such as Haar features or HOG+SVM) to locate faces in images.

[0129] Cropping and rectification: The detected faces are cropped and geometrically rectified to ensure that the faces are located in the center of the image.

[0130] 3) Feature point positioning

[0131] Keypoint detection: Use algorithms (such as dlib or OpenCV) to detect key points of the face, including the locations of eyes, eyebrows, nose, mouth, etc.

[0132] Feature point tracking: These key points are tracked in a sequence of video frames to capture the dynamic changes in facial movements.

[0133] 4) Feature extraction

[0134] Local feature extraction: CNN extracts local features from images, such as edges, corners, and textures.

[0135] Deep feature extraction: Through multiple hidden layers of CNN, local features are combined into more advanced abstract features.

[0136] 5) Micro-expression recognition

[0137] Model training: Use the annotated micro-expression training dataset to train the CNN model to learn the features that distinguish different micro-expressions.

[0138] Micro-expression classification: The extracted deep features are input into the trained CNN model to classify micro-expressions.

[0139] The AffectNet dataset is used for pre-training on large-scale face recognition tasks. AffectNet is a large-scale facial expression dataset that contains rich emotion annotation information, including multiple expression categories and emotion dimensions (such as emotional value and arousal). The following are some key points of using the AffectNet dataset to pre-train the model:

[0140] Emotion recognition: The AffectNet dataset annotates a variety of basic expressions (such as happiness, sadness, anger, etc.) and is suitable for training emotion recognition models.

[0141] Emotional dimension analysis: In addition to basic expressions, AffectNet also provides annotations of emotional value (valence) and arousal (arousal). These dimensions help to understand the emotional state behind the expression.

[0142] Data diversity: AffectNet contains facial images of different ages, genders, and races, which helps train models with better generalization capabilities.

[0143] Transfer learning: Models pre-trained on AffectNet can be applied to specific tasks, such as micro-expression recognition, through transfer learning. Transfer learning usually involves taking the model parameters trained on a large-scale dataset as a starting point and then continuing training on a smaller dataset for a specific task.

[0144] Model fine-tuning: Based on the pre-trained model, fine-tuning can be used to adapt to specific data distributions. For example, if the micro-expression data collected in an ICU environment is distributed differently from the AffectNet dataset, fine-tuning can be used to optimize model performance.

[0145] Feature extraction: The pre-trained models usually have learned rich facial feature representations, which can be used as feature extractors to extract high-level features of micro-expression images.

[0146] The steps for fine-tuning are as follows:

[0147] 1) Data preparation: Collect micro-expression data in the ICU environment, including videos and their annotations. Perform pre-processing operations such as standardization and normalization on the data.

[0148] 2) Load the pre-trained model and freeze the weights of most layers in the model, especially the bottom layers.

[0149] 3) Fine-tuning training:

[0150] Training model: Use the ICU micro-expression dataset to train the model and adjust the model parameters to adapt to the new data.

[0151] Iterative optimization: Gradually optimize model performance through multiple iterative training.

[0152] Partial Unfreezing: Gradually unfreeze more layers in the model, especially when the model performs poorly on new data.

[0153] 4) Model evaluation: Use the validation set to evaluate the performance of the model and monitor overfitting; adjust hyperparameters such as learning rate and batch size based on the evaluation results.

[0154] 5) Model validation: Evaluate the final performance of the model on the test set to ensure the generalization ability of the model.

[0155] 6) Application feedback: Collect feedback from medical staff in actual applications; further adjust and optimize the model based on the feedback.

[0156] 3. Vital signs analysis module:

[0157] The collected vital signs data are analyzed in real time to extract changes in physiological parameters related to pain, such as heart rate variability and blood pressure fluctuations. Machine learning algorithms, such as random forests or support vector machines, are used to perform feature selection and pattern recognition on vital signs data to discover the association between pain and physiological parameters.

[0158] Specific steps for feature selection and pattern recognition of vital signs data:

[0159] 1) Data preprocessing: remove outliers and noise, fill in missing values, and standardize all vital sign data to a uniform range for comparison and analysis.

[0160] 2) Feature extraction

[0161] Calculate statistics: Calculate statistics from the vital signs time series, such as mean, standard deviation, maximum, minimum, etc. For example, calculate heart rate variability (HRV) indicators such as SDNN (standard deviation), RMSSD (root mean square deviation), frequency domain indicators, etc.

[0162] 3) Pattern recognition: Use machine learning algorithms (such as random forests) to evaluate the importance of features and select the features that contribute most to model performance and the patterns associated with pain; different feature combinations can be tried to find the best pattern recognition method.

[0163] 4. Pain scoring model:

[0164] Combining the results of micro-expression recognition and vital sign analysis, the pain scoring model trained by machine learning algorithm is used to quantify the patient's pain level. During the model training process, cross-validation and hyperparameter optimization techniques are used to ensure that the model has good generalization ability and accuracy. A graph neural network (GNN) is used to process facial landmark data to capture the dynamic changes of facial expressions, which is particularly important for assessing the intensity and duration of pain.

[0165] The implementation principle of quantitative scoring is as follows:

[0166] During the entire model training process, the Critical Care Pain Observation Tool (CPOT) scoring tool was combined to ensure that the model can quantify the patient's pain level. CPOT is an assessment tool designed for critically ill patients who are unable to self-report pain. It assesses pain by observing the patient's facial expressions, body movements, muscle tension, and vocalization.

[0167] Here are the steps to incorporate the CPOT scoring tool:

[0168] 1) Data Collection

[0169] Collection of CPOT scores: During the data collection phase, trained medical staff performed CPOT scores on patients and recorded the scoring results.

[0170] Synchronize data: Ensure that the CPOT score is synchronized with the corresponding micro-expression video images and vital signs data.

[0171] 2) Data preprocessing

[0172] Data annotation: CPOT scores are used as labels and associated with the collected micro-expression and vital sign data.

[0173] Data Splitting: Split the dataset into training, validation, and test sets.

[0174] 3) Model training

[0175] Supervised learning: Using CPOT scores as supervisory signals, the model is trained to learn the mapping from features to pain scores.

[0176] Loss Function: Define a loss function to measure the difference between the pain score predicted by the model and the actual CPOT score.

[0177] 4) Fine-tuning of CPOT score

[0178] The pre-trained model was fine-tuned on a dataset containing CPOT scores to adapt it to the specific pain assessment task.

[0179] 5) Model Validation

[0180] Comparison with CPOT scores: The pain scores predicted by the model were compared with the CPOT scores to assess consistency.

[0181] Through these steps, we can ensure that the model fully utilizes the CPOT scoring tool during training, thereby accurately quantifying the patient's pain level. This method helps to improve the objectivity and accuracy of pain assessment, especially when patients are unable to self-report pain.

[0182] The present invention also uses a graph neural network to process facial landmark data as an improved algorithm:

[0183] Graph Neural Network (GNN) is a machine learning model that specializes in processing graph-structured data and is very suitable for analyzing facial landmark data. Facial landmark data can be represented as a graph, where nodes represent key facial landmarks (such as the corners of the eyes, the tip of the nose, the corners of the mouth, etc.), and edges represent the geometric relationships between these landmarks. The following are the specific steps for using GNN to process facial landmark data:

[0184] 1) Data preparation

[0185] Landmark detection: Detect facial landmarks using a facial keypoint detection algorithm (such as dlib or OpenCV).

[0186] Data formatting: The detected landmarks and their relationships are formatted into a graph structure, where each node represents a landmark and the edges represent the spatial relationship between nodes.

[0187] 2) Graph construction

[0188] Node features: Extract features for each node (landmark point), such as local geometric attributes, texture information, etc.

[0189] Edge features: Define the features of an edge. These features can represent the distance, angle, or other geometric relationships between landmarks.

[0190] Graph representation: Construct a representation of the graph, including node features, edge features, and the overall structure of the graph.

[0191] 3) Model selection

[0192] The Graph Attention Network model (GAT) is used. GAT introduces an attention mechanism that allows the model to learn the different importance of different neighbor nodes. This model can better capture complex facial expression changes when processing facial landmarks.

[0193] 4) Feature Learning

[0194] Node feature aggregation: Use the GNN model to aggregate the features of adjacent nodes and update the representation of each node.

[0195] Edge Feature Learning: Learn edge features to capture the relationships between nodes.

[0196] 5) Graph Convolution

[0197] Graph convolutional layer: Graph convolutional layer is applied to update the feature representation of nodes, and each layer takes into account the neighbor information of the node.

[0198] Non-linear activation: Apply a non-linear activation function such as ReLU after the graph convolutional layer.

[0199] 6) Feature Fusion

[0200] Global pooling: Use graph pooling techniques (such as global average pooling or graph readout layer) to aggregate the information of the entire graph to obtain a global feature representation.

[0201] 7) Pain feature recognition

[0202] Classifier design: Add one or more fully connected layers on top of the GNN as a classifier to recognize pain expressions.

[0203] Training GNN: Use labeled pain data to train the GNN model and learn features that distinguish different pain levels.

[0204] 8) Model Optimization

[0205] Loss Function: Define a loss function to measure the difference between the model prediction and the actual pain label.

[0206] Backpropagation: Optimize model parameters through the backpropagation algorithm.

[0207] 9) Model Evaluation

[0208] Performance evaluation: Use the validation set to evaluate the model’s accuracy, recall, and F1 score.

[0209] Model tuning: Adjust the model structure and hyperparameters based on the evaluation results.

[0210] 10) Application Feedback

[0211] Collect feedback: Collect feedback from medical staff during actual application.

[0212] Model iteration: Further adjust and optimize the model based on feedback.

[0213] 11) Deployment and Monitoring

[0214] Model deployment: Deploy the trained model to the actual pain assessment system.

[0215] Performance monitoring: Continuously monitor the performance of the model in actual applications to ensure its stability and accuracy.

[0216] Using GNN to process facial landmark data can effectively capture the complex dynamic changes of facial expressions, thereby improving the accuracy of pain assessment.

[0217] 5. Result output module:

[0218] The pain score results are output to medical staff in real time so that the pain management plan can be adjusted in time; a user-friendly interface is designed to display the dynamic change trend of the pain score and the intervention measures that may be needed.

[0219] 6. Clinical verification and feedback:

[0220] The system was clinically tested in an ICU environment, and feedback from medical staff and patients was collected to evaluate the practicality and accuracy of the system; model parameters and user interface design were adjusted based on clinical feedback to improve the clinical applicability and user satisfaction of the system.

[0221] Through this technical solution, a relatively objective pain scoring tool can be provided for ICU patients, which helps to improve the quality and efficiency of pain management.

[0222] The deep learning algorithm used in the micro-expression recognition module, especially the CNN architecture used to identify pain-related micro-expression features.

[0223] Machine learning algorithm used in the vital signs analysis module to extract changes in pain-related physiological parameters.

[0224] The algorithm used in the pain scoring model to combine micro-expression and vital sign data for pain scoring includes feature fusion technology and model training methods.

[0225] System integration and optimization technology, including measures to ensure the real-time and accuracy of data collection, processing, analysis and output.

[0226] Clinical verification and feedback mechanism ensures the applicability and continuous improvement of the system in actual medical environments.

[0227] In the process of implementing the present invention, the following technical means can also be used: optimize the micro-expression recognition module, such as introducing more advanced machine learning algorithms, adding more physiological signal data features, etc., to improve the accuracy and precision of micro-expression recognition; in addition, a model self-feedback framework can be added to the system to improve the responsiveness and reasoning ability of the model. The present invention can also interact with the electronic medical record system of critically ill patients so that medical staff can fully understand the patient's condition.

[0228] In summary, the pain scoring system of the present invention provides a new pain assessment tool for ICU patients by combining micro-expression recognition and vital sign analysis using artificial intelligence technology, which is expected to significantly improve the quality and efficiency of pain management.

[0229] The specific implementation scheme of this embodiment can refer to the relevant description in the above embodiment, which will not be repeated here.

[0230] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.

[0231] It should be noted that, in the description of the present invention, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, the meaning of "plurality" refers to at least two.

[0232] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0233] The system, method, processor and computer-readable storage medium for implementing ICU patient pain scoring of the present invention are adopted, without the need for patient self-reporting, and are applicable to all ICU patients, including intubated patients who cannot communicate. Artificial intelligence technology is used to improve the objectivity and accuracy of pain scoring. The sensitivity and specificity of pain assessment are improved by analyzing the combination of micro-expressions and vital signs. Real-time monitoring and continuous evaluation of pain scores are achieved, which helps to adjust treatment plans in a timely manner.

[0234] In this specification, the present invention has been described with reference to specific embodiments thereof. However, it is apparent that various modifications and variations may be made without departing from the spirit and scope of the present invention. Therefore, the specification and drawings should be regarded as illustrative rather than restrictive.

Claims

1. A system for implementing pain scoring for ICU patients, characterized in that: The system comprises: The data acquisition module is used to use a high-definition camera to collect real-time facial micro-expression videos of ICU patients and simultaneously collect the patients' main vital signs data; A micro-expression recognition module, connected to the data acquisition module, is used to analyze the collected micro-expression video images using a convolutional neural network to identify micro-expression features related to pain; A vital sign analysis module, connected to the data acquisition module, for real-time analysis of the collected vital sign data and extraction of changes in physiological parameters related to pain; A pain scoring model is connected to the micro-expression recognition module and the vital signs analysis module, and is used to quantitatively score the patient's pain level by combining the micro-expression recognition results and the vital signs analysis results with a pain scoring model trained by a machine learning algorithm; A result output module, connected to the pain scoring model, is used to output the pain scoring results to medical staff in real time; A clinical validation feedback module, connected to the pain scoring model and result output module, for clinically testing the system in an ICU environment, collecting feedback from medical staff and patients, and adjusting model parameters and user interface design based on clinical feedback; The medical record interaction module is connected to the data acquisition module and is used to interact with the electronic medical record system of critically ill patients to obtain the patient's condition.

2. The system for implementing pain scoring of ICU patients according to claim 1, characterized in that: The micro-expression recognition module uses a convolutional neural network to analyze micro-expression video images and recognize micro-expression features, specifically including the following steps: Perform video preprocessing; use face detection algorithm to locate faces in images; locate feature points; extract local features and deep features; train CNN model and classify micro-expressions.

3. The system for implementing pain scoring of ICU patients according to claim 1, characterized in that: The vital signs analysis module performs real-time analysis on the collected vital signs data, extracts changes in physiological parameters related to pain, preprocesses the data, removes outliers and noise, fills in missing values, and standardizes all vital signs data, calculates statistics from the vital signs time series, and evaluates the importance of features through machine learning algorithms, selecting the features with the greatest contribution and patterns related to pain.

4. The system for implementing pain scoring of ICU patients according to claim 1, characterized in that: In the process of training the pain scoring model, the intensive care pain observation tool CPOT is used to quantitatively score the patient's pain level, which specifically includes the following steps: Patients were scored with CPOT and synchronized; data were preprocessed; pre-trained models were fine-tuned based on CPOT scores; pain scores predicted by the model were compared with CPOT scores to assess consistency; The pain scoring model also processes facial landmark data through a graph neural network, specifically including the following steps: Prepare data; extract node features and edge features, and construct a graph; use a graph attention network model to learn edge features; apply a graph convolutional layer to update the feature representation of the node and apply a nonlinear activation function; use graph pooling technology to aggregate the information of the entire graph to obtain a global feature representation; identify pain features; optimize model parameters; and evaluate and adjust the model.

5. A method for implementing pain scoring for ICU patients based on the system of claim 1, characterized in that: The method comprises the following steps: (1) Perform patient identification, use a high-definition camera to collect real-time facial micro-expression videos of ICU patients, and proceed to step (2); at the same time, collect the patient's main vital signs data, and proceed to step (3); (2) using a convolutional neural network to analyze the collected micro-expression video images and identify micro-expression features related to pain; if the vital sign data is not processed, proceed to step (3); otherwise, proceed to step (4); (3) Analyze the collected vital sign data in real time to extract changes in physiological parameters related to pain; if the facial micro-expression video is not processed, proceed to step (2); otherwise, proceed to step (4); (4) Combining the micro-expression recognition results and vital sign analysis results, a pain scoring model is trained using a machine learning algorithm to quantitatively score the patient's pain level; (5) The pain score results are output to medical staff in real time.

6. The method for achieving pain scoring of ICU patients according to claim 5, characterized in that: The step (2) specifically comprises the following steps: (2.1) Perform video preprocessing; (2.2) Use face detection algorithm to locate the face in the image; (2.3) Positioning feature points; (2.4) Extract local features and deep features; (2.5) Train the CNN model and classify micro-expressions.

7. The method for achieving pain scoring of ICU patients according to claim 5, characterized in that: The step (3) specifically comprises the following steps: (3.1) Analyze the collected vital signs data in real time and extract changes in physiological parameters related to pain; (3.2) Preprocess the data to remove outliers and noise, fill in missing values, and standardize all vital sign data; (3.3) Calculate statistics from vital sign time series; (3.4) Evaluate the importance of features through machine learning algorithms and select the most important features and pain-related patterns; The step (3) further includes the step of processing the facial landmark data by means of a graph neural network, which specifically includes the following processing steps: (1-3.1) Prepare data; (1-3.2) Extract node features and edge features and construct a graph; (1-3.3) Use the graph attention network model to learn edge features; (1-3.4) Apply graph convolutional layers to update the feature representation of nodes and apply nonlinear activation functions; (1-3.5) Use graph pooling technology to aggregate the information of the entire graph and obtain a global feature representation; (1-3.6) Identify pain characteristics; (1-3.7) Optimize model parameters; (1-3.8) Perform model evaluation and adjustment.

8. A device for implementing pain scoring of ICU patients, characterized in that: The device comprises: a processor configured to execute computer executable instructions; A memory storing one or more computer executable instructions, wherein when the computer executable instructions are executed by the processor, the steps of the method for achieving pain scoring of ICU patients described in any one of claims 5 to 7 are implemented.

9. A processor for implementing pain scoring of ICU patients, characterized in that: The processor is configured to execute computer executable instructions. When the computer executable instructions are executed by the processor, the various steps of the method for achieving ICU patient pain scoring described in any one of claims 5 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program can be executed by a processor to implement the various steps of the method for achieving ICU patient pain scoring according to any one of claims 5 to 7.

Images