System and method for monitoring data indicators of critically ill patients

Through the combination of facial emotion recognition and monitoring data, the problem of insufficient monitoring of data indicators for critically ill patients is solved, the monitoring efficiency and reliability are improved, and more accurate patient status assessment and care suggestions are provided.

CN118571399BActive Publication Date: 2024-11-12JILIN UNIVERSITY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411059593.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-05
Publication Date
2024-11-12
Estimated Expiration
2044-08-05

AI Technical Summary

Technical Problem

The monitoring of data indicators for critically ill patients is not comprehensive and comprehensive enough, ignoring the patient's emotional factors, affecting the judgment of indicator data.

Method used

The patient's facial images are obtained through the camera device, image enhancement and emotion recognition are performed, mood index is determined, and monitoring data obtained by the sensor device is screened and analyzed to generate index monitoring results and accompanying suggestions.

Benefits of technology

The efficiency and reliability of monitoring data indicators for critically ill patients is improved, and a comprehensive evaluation is carried out through facial emotion recognition combined with monitoring data to provide more accurate patient status assessment and accompanying advice.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118571399B_ABST
    Figure CN118571399B_ABST
Patent Text Reader

Abstract

The present application provides a system and method for monitoring data indicators of critically ill patients, which relates to the field of data monitoring. The method includes: acquiring a facial image of a patient through a camera device; performing image enhancement on the facial image to generate an enhanced image, performing emotion recognition on the enhanced image, and determining the patient's current emotion index; acquiring various monitoring data of the patient through a sensor device; based on the correspondence between the emotion index and the monitoring data, screening the target data corresponding to the emotion index from the monitoring data; based on the emotion index and the target data, performing a comprehensive analysis to generate indicator monitoring results and accompanying suggestions for the patient. This solution uses facial emotion recognition to determine the patient's emotions, and then conducts a comprehensive assessment of the patient's status based on the patient's emotions combined with the corresponding monitoring data, and finally generates relevant indicator monitoring results and accompanying suggestions, thereby improving the efficiency and reliability of data indicator monitoring for critically ill patients.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data monitoring, and specifically, to a system and method for monitoring data indicators of critically ill patients. Background Art

[0002] With the continuous development of technologies such as computers, network communications, and big data science, a data indicator monitoring system for critically ill patients has been constructed and improved. These technologies provide strong support for the system's real-time monitoring, data analysis, prediction and early warning functions. The system can monitor the patient's vital signs, treatment progress, and condition changes in real time, providing timely and accurate data support for medical staff. However, in clinical practice, the emotions of critically ill patients are often an important factor affecting treatment and recovery. At present, the above factors are often ignored in the indicator monitoring of critically ill patients, which results in the problem that the data indicator monitoring of critically ill patients is not comprehensive and integrated, affecting the judgment of indicator data. Summary of the invention

[0003] The embodiments of the present application provide a system and method for monitoring data indicators of critically ill patients, which can at least to a certain extent solve the problem that data indicator monitoring of critically ill patients is not comprehensive and integrated enough.

[0004] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0005] According to one aspect of the present application, a method for monitoring data indicators of critically ill patients is provided, comprising: acquiring a facial image of the patient through a camera device; performing image enhancement on the facial image to generate an enhanced image, performing emotion recognition on the enhanced image, and determining the patient's current emotion index; acquiring various monitoring data of the patient through a sensor device; based on the correspondence between the emotion index and the monitoring data, screening target data corresponding to the emotion index from the monitoring data; performing a comprehensive analysis based on the emotion index and the target data to generate indicator monitoring results and accompanying suggestions for the patient.

[0006] In the present application, based on the aforementioned scheme, the method of obtaining a facial image of a patient through a camera device includes: capturing a target image through a camera device; converting the target image into a grayscale image based on the pixel values ​​corresponding to each pixel point in the target image; detecting whether the grayscale image contains a face based on a face detection algorithm; if so, extracting feature information corresponding to the grayscale image at preset key points; comparing the feature information with pre-stored patient features to determine the identity information corresponding to the face in the target image; if the identity information is patient information, using the target image as the facial image.

[0007] In the present application, based on the above-mentioned solution, it also includes: tracking based on the facial area in the target image, and acquiring the facial image of the patient based on a preset acquisition cycle.

[0008] In the present application, based on the aforementioned scheme, the facial image is enhanced to generate an enhanced image, emotion recognition is performed on the enhanced image, and the patient's current emotion index is determined, including: performing image enhancement on the facial image to generate an enhanced image; detecting a facial area from the enhanced image, performing emotion recognition based on the facial area, and generating an emotion classification result; and determining the emotion index based on the emotion classification result.

[0009] In the present application, based on the aforementioned scheme, a facial area is detected from an enhanced image, emotion recognition is performed based on the facial area, and an emotion classification result is generated, including: locating features in the facial area and determining the image position of key points in the facial area; extracting emotion-related features from the facial area based on the image position; and determining the emotion classification result corresponding to the feature based on the feature and an emotion label classifier.

[0010] In the present application, based on the aforementioned scheme, based on the correspondence between the emotion index and the monitoring data, the target data corresponding to the emotion index is screened from the monitoring data, including: pre-establishing a correspondence between the emotion index and the monitoring data, the monitoring data including at least one of the following: blood pressure data, electrocardiogram data, respiratory data, body temperature data and nervous system data; based on the correspondence, the target data corresponding to the emotion index is screened from the detection data.

[0011] In the present application, based on the aforementioned scheme, after conducting a comprehensive analysis based on the emotional index and the target data to generate indicator monitoring results and accompanying suggestions for the patient, the following steps are also included: generating a monitoring report based on the emotional index, target data, and the generated indicator monitoring results and accompanying suggestions; and sending the monitoring report to the caregiver and the attending physician.

[0012] According to one aspect of the present application, a critically ill patient data indicator monitoring system is provided, comprising:

[0013] An acquisition unit, used for acquiring a facial image of a patient through a camera device;

[0014] An enhancement unit, configured to perform image enhancement on the facial image to generate an enhanced image, perform emotion recognition on the enhanced image, and determine a current emotion index of the patient;

[0015] A monitoring unit, used to obtain various monitoring data of the patient through a sensor device;

[0016] A screening unit, configured to screen target data corresponding to the emotion index from the monitoring data based on the corresponding relationship between the emotion index and the monitoring data;

[0017] The analysis unit is used to perform a comprehensive analysis based on the emotion index and the target data to generate indicator monitoring results and nursing suggestions for the patient.

[0018] According to one aspect of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method for monitoring data indicators of critically ill patients as described in the above embodiments is implemented.

[0019] According to one aspect of the present application, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for monitoring data indicators of critically ill patients as described in the above-mentioned embodiments.

[0020] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the critically ill patient data indicator monitoring method provided in the above-mentioned various optional implementations.

[0021] In the technical solution of the present application, a facial image of a patient is acquired through a camera device; the facial image is enhanced to generate an enhanced image, emotion recognition is performed on the enhanced image to determine the patient's current emotion index; various monitoring data of the patient are acquired through a sensor device; based on the correspondence between the emotion index and the monitoring data, target data corresponding to the emotion index is screened from the monitoring data; based on the emotion index and the target data, a comprehensive analysis is performed to generate indicator monitoring results and accompanying suggestions for the patient. The technical solution of the present application determines the patient's emotions through facial emotion recognition, and then comprehensively evaluates the patient's condition based on the patient's emotions combined with the corresponding monitoring data, and finally generates relevant indicator monitoring results and accompanying suggestions, thereby improving the efficiency and reliability of data indicator monitoring of critically ill patients.

[0022] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the specification are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 A flowchart of a method for monitoring data indicators of critically ill patients in one embodiment of the present application is schematically shown.

[0025] Figure 2 A flowchart for acquiring a facial image of a patient by using a camera device in one embodiment of the present application is schematically shown.

[0026] Figure 3 A schematic diagram of a critically ill patient data indicator monitoring system in one embodiment of the present application is schematically shown.

[0027] Figure 4 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0028] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0029] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0030] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0031] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0032] The implementation details of the technical solution of this application are described in detail below:

[0033] Figure 1 A flowchart of a method for monitoring data indicators of critically ill patients according to an embodiment of the present application is shown. Figure 1 As shown, the critically ill patient data indicator monitoring method includes at least steps S110 to S150, which are described in detail as follows:

[0034] In step S110, a facial image of the patient is acquired by a camera.

[0035] Usually, a video camera or a high-definition camera is used to collect facial images. These devices have high resolution, good light adaptability and stable shooting performance.

[0036] Optionally, in this embodiment, the camera device is installed at a location where the facial image of the patient can be clearly captured, such as a ward, consulting room or examination area of ​​a hospital.

[0037] like Figure 2 As shown, in one embodiment of the present application, acquiring a facial image of a patient by a camera device includes:

[0038] S210: Capturing a target image by a camera device;

[0039] S220: Converting the target image into a grayscale image based on a pixel value corresponding to each pixel point in the target image;

[0040] S230: Detecting whether the grayscale image contains a face based on a face detection algorithm;

[0041] S240: If yes, extracting feature information corresponding to the grayscale image at the preset key point;

[0042] S250: Compare the feature information with pre-stored patient features to determine identity information corresponding to the face in the target image;

[0043] S260: If the identity information is patient information, use the target image as the facial image.

[0044] In one embodiment of the present application, a high-resolution, high-sensitivity camera is selected to ensure that the patient's facial image can be clearly captured. The patient's facial image is captured in real time by a camera device and transmitted to a processing system. The captured color image is converted into a grayscale image to reduce processing complexity.

[0045] In grayscale images, face detection algorithms are used to identify facial regions, such as Haar feature algorithms, SVM algorithms, or algorithms based on deep learning. A detection window is slid across the image, and a classifier is applied to determine whether the window contains a face.

[0046] Optionally, the detected face images can be resized to a uniform size for subsequent processing. Enhancement processing such as histogram equalization and denoising can be performed on the face images to improve image quality. If the face image is tilted or rotated, posture correction is required to make the face face the camera.

[0047] Locate multiple key points (such as corner points of eyes, nose, mouth, etc.) in the face image, and use these key points for subsequent feature extraction. Extract the feature information corresponding to the preset key points in the grayscale image, and calculate the geometric features of the face or the feature vector based on deep learning (such as 128-dimensional, 256-dimensional or 512-dimensional floating-point vector) based on the key point information to describe the uniqueness and difference of the face through these feature vectors.

[0048] In one embodiment of the present application, a database containing facial features of known identities is constructed, and the facial features extracted in real time are compared with the features in the database. Common comparison methods include calculating the cosine similarity or Euclidean distance between feature vectors. Based on the comparison results, it is determined which face in the database is most similar to the real-time facial image, and the identity of the patient is confirmed accordingly. The feature information is compared with the pre-stored patient features to determine the identity information corresponding to the face in the target image; if the identity information is patient information, the target image is used as the facial image.

[0049] In one embodiment of the present application, it also includes: tracking based on the facial area in the target image, and acquiring the facial image of the patient based on a preset acquisition period.

[0050] The facial region is detected from the target image (usually a frame in the video stream) and then continuously tracked in subsequent video frames. This is usually achieved through a tracking algorithm, such as a deep learning tracker. A fixed time interval is set as the acquisition cycle according to actual needs. At the end of each acquisition cycle, the facial image is extracted from the current video frame. At the end of the acquisition cycle, the facial region is cropped from the current video frame and saved as a separate image file.

[0051] In step S120, the facial image is enhanced to generate an enhanced image, and emotion recognition is performed on the enhanced image to determine the patient's current emotion index.

[0052] In one embodiment of the present application, the facial image is enhanced to generate an enhanced image, and emotion recognition is performed on the enhanced image to determine the patient's current emotion index, so as to improve the visual quality of the facial image and make it more suitable for subsequent emotion recognition tasks.

[0053] In one embodiment of the present application, performing image enhancement on the facial image to generate an enhanced image, performing emotion recognition on the enhanced image, and determining the patient's current emotion index includes:

[0054] Performing image enhancement on the facial image to generate an enhanced image;

[0055] Detecting a facial region from the enhanced image, performing emotion recognition based on the facial region, and generating an emotion classification result;

[0056] An emotion index is determined according to the emotion classification result.

[0057] In one embodiment of the present application, after acquiring the facial image, the image quality is first improved to improve the recognizability of key information in the image, such as facial expression details. The image enhancement process may specifically include: adjusting the lighting conditions of the image to make the light in the facial area uniform and reduce shadows and reflections; enhancing the contrast of the image to make subtle changes in facial expressions more obvious; removing noise in the image through filtering and other methods to make the image clearer; sharpening the image to enhance facial contours and texture details, which helps the emotion recognition algorithm capture more information.

[0058] In one embodiment of the present application, a facial region is detected from an enhanced image, emotion recognition is performed based on the facial region, and an emotion classification result is generated, including the following steps:

[0059] Positioning features in the facial region to determine image positions of key points in the facial region;

[0060] extracting emotion-related features from the facial region based on the image location;

[0061] Based on the feature and the emotion label classifier, an emotion classification result corresponding to the feature is determined.

[0062] Specifically, the specific processing methods in the above steps are as follows:

[0063] 1. Use face detection algorithms (such as Haar features + AdaBoost, CNN, etc.) to locate the facial area, that is, place the facial area at a fixed position in the image.

[0064] Assume the input image is I , use the face detection algorithm to locate the face area. The face detection algorithm can be expressed as a function FaceDetection, whose output is the bounding box of the face area R , assuming it is the coordinate of the upper left corner of the rectangular box and the lower right corner coordinates , and its formula is shown in (1).

[0065] (1)

[0066] 2. Perform affine transformation or more complex 3D transformation on the detected facial area to achieve facial alignment, that is, to ensure that the positions of key parts such as eyes, nose, mouth, etc. in the image are relatively fixed. Alignment transformation matrix T The left eye coordinates in the source image can be , right eye coordinates , nose coordinates , left mouth corner coordinates , right mouth corner coordinates , and the left eye coordinates in the transformed target image , right eye coordinates , nose coordinates , left mouth corner coordinates , right mouth corner coordinates The position coordinates of the key points are calculated. Assume that the key point position is and , respectively represent the position coordinate sets of key points in the source image and the target image, and their calculation formula is shown in (2).

[0067] ,

[0068] , (2)

[0069] Alignment Transformation Matrix T It is calculated in the following way, as shown in formula (3).

[0070] (3)

[0071] Then, apply the change matrix To the facial area Above, we get the aligned facial image :

[0072]

[0073] 3. Extract emotion-related features from the aligned facial images. This includes extracting the location information of facial key points, such as the shape and position changes of eyes, eyebrows, and mouth; using feature extraction algorithms to obtain the locations of facial key points. Assume that the key point extraction function is The key point position is , assuming the key point Include points, the position coordinates of each point are ,but As shown in formula (4).

[0074] (4)

[0075] 4. Analyze the texture changes of facial skin, such as wrinkles, skin gloss, etc.; use deep learning models to automatically extract high-level emotion-related features. Assume that the texture analysis function is , The texture feature is , the extraction formula is shown in (5), where Information representing multiple feature points of the skin.

[0076] (5)

[0077] 5. Use a facial image dataset with emotion labels (such as happiness, sadness, anger, surprise, fear, disgust, etc.) to train a classifier (such as SVM, random forest, neural network, etc.), match the extracted features with the trained classifier, and output the emotion classification result, i.e., the emotion label

[0078] ,in Represents the emotion feature vector.

[0079] The extracted different features are combined into a combined feature vector: , combined feature vector As shown in formula (6).

[0080] (6)

[0081] Based on the combined feature vector, emotion classification is performed to generate the emotion classification result: , where C represents the classifier used in training.

[0082] In one embodiment of the present application, after generating the emotion classification result, the emotion index is determined based on the emotion classification result. The emotion index in this embodiment can be used to represent the intensity and type of emotion. By converting the emotion classification result into a specific emotion index, it is easy for medical staff to understand and apply it, and it is also easy to determine the corresponding monitoring data based on the emotion index.

[0083] Specifically, in this embodiment, a quantitative value (such as happiness = 100, 60 = moderate, sadness = 20, etc.) is assigned to each emotion category according to the emotion classification result, or the probability value output by the classifier is directly used as a quantitative indicator of the emotion intensity. If multiple emotions are identified to coexist, a weighted synthesis can be performed based on the quantitative values ​​and importance of different emotions to obtain a comprehensive emotion index.

[0084] Optionally, in this embodiment, a higher emotion index score indicates that the patient's emotion is more optimistic, and a lower emotion index score indicates that the patient's emotion is more pessimistic.

[0085] In step S130, various monitoring data of the patient are acquired through a sensor device.

[0086] In one embodiment of the present application, the monitoring data includes at least one of the following: blood pressure data, electrocardiogram data, respiratory data, body temperature data, and nervous system data.

[0087] The sensing device can include various sensors, which have different functions and characteristics depending on the object being measured. Common sensors used for medical monitoring include:

[0088] Temperature sensor: used to measure the patient's body temperature, which is one of the important indicators for assessing the patient's health status.

[0089] Pressure sensors: can be placed under a hospital bed or mattress to monitor the patient’s skin pressure, helping to prevent complications such as skin ulcers.

[0090] Heart rate sensor: monitors the patient's heart rate changes through electrocardiogram or other non-invasive methods to reflect heart health status.

[0091] Blood oxygen saturation sensor: Assess the patient's respiratory function and oxygenation capacity by measuring the percentage of oxygenated hemoglobin in the blood.

[0092] Respiratory sensor: Monitoring the patient's breathing rate and depth is important for evaluating respiratory system function and diagnosing related diseases.

[0093] In step S140, based on the correspondence between the emotion index and the monitoring data, target data corresponding to the emotion index is filtered from the monitoring data.

[0094] In one embodiment of the present application, based on the correspondence between the emotion index and the monitoring data, filtering the target data corresponding to the emotion index from the monitoring data includes:

[0095] Establish the corresponding relationship between the emotion index and the monitoring data in advance;

[0096] Based on the corresponding relationship, target data corresponding to the emotion index is filtered out from the detection data.

[0097] In this embodiment, the monitoring data includes blood pressure data, electrocardiogram data, respiratory data, body temperature data, sleep data and nervous system data.

[0098] In this embodiment, a correspondence between the emotion index and the monitoring data is preset. That is, the corresponding monitoring data is determined according to the emotion index, so as to comprehensively determine the patient's condition according to the emotion index and the monitoring data. The specific correspondence is as follows:

[0099] Sentiment Index Monitoring data 0~20 Nervous system data, blood pressure data, ECG data, sleep data, body temperature data 21~40 Blood pressure data, nervous system data, sleep data, body temperature data 41~60 Blood pressure data, ECG data 61~80 Blood pressure data, ECG data 81~100 Blood pressure data, ECG data, respiratory data

[0100] Through the above table, the corresponding monitoring data is obtained based on the sentiment index, so as to publish the above monitoring data to the analysis module for comprehensive analysis.

[0101] In step S150, a comprehensive analysis is performed based on the emotion index and the target data to generate indicator monitoring results and nursing recommendations for the patient.

[0102] In one embodiment of the present application, based on the comprehensive analysis of the emotion index and target data, the indicator monitoring results and accompanying suggestions for the patient can be generated. The target data includes the patient's physiological indicators such as heart rate, blood pressure, sleep quality, etc., as well as possible medical records, medication status, etc.

[0103] For example, Mr. Zhang, a 65-year-old patient, has a mood index of 40 (out of 100) today through facial image emotion recognition technology, which is at a lower-middle level, showing a certain negative emotion. Compared with the previous few days, the mood index has dropped, which may indicate that Mr. Zhang's emotional state has deteriorated. Therefore, based on the corresponding relationship, the target data to be monitored include: blood pressure data, nervous system data, and sleep data. Combining Mr. Zhang's mood index and target data, we can conduct the following comprehensive analysis: Mr. Zhang's mood index has dropped, which may be affected by factors such as hypertension or hospitalization environment, leading to depression. The increase in blood pressure may be related to mood changes, poor sleep quality, or poor drug control. The decline in sleep quality may further aggravate the depression and fluctuations in physiological indicators. Based on the above analysis, the accompanying suggestions are generated: strengthen communication with the patient, understand the reasons for his mood changes, provide necessary psychological counseling and support, and encourage him to maintain a positive and optimistic attitude.

[0104] In one embodiment of the present application, a monitoring report is generated based on the emotion index, target data, and the generated indicator monitoring results and accompanying suggestions;

[0105] The monitoring report is sent to the caregiver and the attending physician.

[0106] In one embodiment of the present application, through user portrait and preference analysis, the reading habits and concerns of the caregiver and the attending physician are understood, and a report template that meets their needs is intelligently recommended or generated. At the same time, a report customization function is provided to allow users to adjust and edit according to their needs.

[0107] Develop a monitoring and reporting system with integrated real-time communication functions, allowing caregivers and attending physicians to communicate online through the system's built-in chat tools. At the same time, provide a report sharing function, allowing users to send reports to others in the form of links or files for viewing and discussion. In addition, you can set up a message push function to ensure that important information can be conveyed to users in a timely manner. Combine monitoring reports with real-time communication tools so that caregivers and attending physicians can receive reports instantly and communicate feedback.

[0108] In the technical solution of the present application, a facial image of a patient is acquired through a camera device; the facial image is enhanced to generate an enhanced image, emotion recognition is performed on the enhanced image to determine the patient's current emotion index; various monitoring data of the patient are acquired through a sensor device; based on the correspondence between the emotion index and the monitoring data, target data corresponding to the emotion index is screened from the monitoring data; based on the emotion index and the target data, a comprehensive analysis is performed to generate indicator monitoring results and accompanying suggestions for the patient. The technical solution of the present application determines the patient's emotions through facial emotion recognition, and then comprehensively evaluates the patient's condition based on the patient's emotions combined with the corresponding monitoring data, and finally generates relevant indicator monitoring results and accompanying suggestions, thereby improving the efficiency and reliability of data indicator monitoring of critically ill patients.

[0109] The following describes an embodiment of the device of the present application, which can be used to execute the method for monitoring data indicators of critically ill patients in the above-mentioned embodiment of the present application. It is understandable that the device can be a computer program (including program code) running in a computer device, for example, the device is an application software; the device can be used to execute the corresponding steps in the method provided in the embodiment of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method for monitoring data indicators of critically ill patients described above in the present application.

[0110] Figure 3 A block diagram of a critically ill patient data indicator monitoring system according to an embodiment of the present application is shown.

[0111] Reference Figure 3 As shown, a critically ill patient data indicator monitoring system according to one embodiment of the present application includes:

[0112] An acquisition unit 310 is used to acquire a facial image of a patient through a camera device;

[0113] The enhancement unit 320 is used to perform image enhancement on the facial image to generate an enhanced image, perform emotion recognition on the enhanced image, and determine the patient's current emotion index;

[0114] Monitoring unit 330, used to obtain various monitoring data of the patient through a sensor device;

[0115] A screening unit 340, configured to screen target data corresponding to the emotion index from the monitoring data based on the corresponding relationship between the emotion index and the monitoring data;

[0116] The analysis unit 350 is used to perform a comprehensive analysis based on the emotion index and the target data to generate indicator monitoring results and nursing recommendations for the patient.

[0117] In the present application, based on the aforementioned scheme, the method of obtaining a facial image of a patient through a camera device includes: capturing a target image through a camera device; converting the target image into a grayscale image based on the pixel values ​​corresponding to each pixel point in the target image; detecting whether the grayscale image contains a face based on a face detection algorithm; if so, extracting feature information corresponding to the grayscale image at preset key points; comparing the feature information with pre-stored patient features to determine the identity information corresponding to the face in the target image; if the identity information is patient information, using the target image as the facial image.

[0118] In the present application, based on the above-mentioned solution, it also includes: tracking based on the facial area in the target image, and acquiring the facial image of the patient based on a preset acquisition cycle.

[0119] In the present application, based on the aforementioned scheme, the facial image is enhanced to generate an enhanced image, emotion recognition is performed on the enhanced image, and the patient's current emotion index is determined, including: performing image enhancement on the facial image to generate an enhanced image; detecting a facial area from the enhanced image, performing emotion recognition based on the facial area, and generating an emotion classification result; and determining the emotion index based on the emotion classification result.

[0120] In the present application, based on the aforementioned scheme, a facial area is detected from an enhanced image, emotion recognition is performed based on the facial area, and an emotion classification result is generated, including: locating features in the facial area and determining the image position of key points in the facial area; extracting emotion-related features from the facial area based on the image position; and determining the emotion classification result corresponding to the feature based on the feature and an emotion label classifier.

[0121] In the present application, based on the aforementioned scheme, based on the correspondence between the emotion index and the monitoring data, the target data corresponding to the emotion index is screened from the monitoring data, including: pre-establishing a correspondence between the emotion index and the monitoring data, the monitoring data including at least one of the following: blood pressure data, electrocardiogram data, respiratory data, body temperature data and nervous system data; based on the correspondence, the target data corresponding to the emotion index is screened from the detection data.

[0122] In the present application, based on the aforementioned scheme, after conducting a comprehensive analysis based on the emotional index and the target data to generate indicator monitoring results and accompanying suggestions for the patient, the following steps are also included: generating a monitoring report based on the emotional index, target data, and the generated indicator monitoring results and accompanying suggestions; and sending the monitoring report to the caregiver and the attending physician.

[0123] In the technical solution of the present application, a facial image of a patient is acquired through a camera device; the facial image is enhanced to generate an enhanced image, emotion recognition is performed on the enhanced image to determine the patient's current emotion index; various monitoring data of the patient are acquired through a sensor device; based on the correspondence between the emotion index and the monitoring data, target data corresponding to the emotion index is screened from the monitoring data; based on the emotion index and the target data, a comprehensive analysis is performed to generate indicator monitoring results and accompanying suggestions for the patient. The technical solution of the present application determines the patient's emotions through facial emotion recognition, and then comprehensively evaluates the patient's condition based on the patient's emotions combined with the corresponding monitoring data, and finally generates relevant indicator monitoring results and accompanying suggestions, thereby improving the efficiency and reliability of data indicator monitoring of critically ill patients.

[0124] Figure 4 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown.

[0125] It should be noted that the computer system 400 of the electronic device shown in the figure is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0126] The computer system 400 includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 402 or the program loaded from the storage part 408 to the random access memory (RAM) 403, such as executing the method described in the above embodiment. Various programs and data required for system operation are also stored in the RAM 403. The CPU 401, the ROM 402 and the RAM 403 are connected to each other through the bus 404. The input / output (I / O) interface 405 is also connected to the bus 404.

[0127] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed so that a computer program read therefrom is installed into the storage section 408 as needed.

[0128] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 409, and / or installed from a removable medium 411. When the computer program is executed by the central processing unit (CPU) 401, various functions defined in the system of the present application are executed.

[0129] It should be noted that the computer-readable medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable computer program. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0130] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0131] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0132] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above-mentioned various optional implementations.

[0133] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

[0134] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0135] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software, or by combining software with necessary hardware. Therefore, the technical solution according to the implementation method of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation method of the present application.

[0136] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0137] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for monitoring data indicators of critically ill patients, characterized in that: include: Acquiring a facial image of the patient by means of a camera device; Performing image enhancement on the facial image to generate an enhanced image, and performing emotion recognition on the enhanced image to determine the patient's current emotion index; Obtain various monitoring data of patients through sensor devices; Based on the corresponding relationship between the emotion index and the monitoring data, selecting target data corresponding to the emotion index from the monitoring data; Based on the emotion index and the target data, a comprehensive analysis is performed to generate indicator monitoring results and accompanying care suggestions for the patient; The step of performing image enhancement on the facial image to generate an enhanced image, and performing emotion recognition on the enhanced image to determine the patient's current emotion index includes: Performing image enhancement on the facial image to generate an enhanced image; Detecting a facial region from the enhanced image, performing emotion recognition based on the facial region, and generating an emotion classification result; Determining an emotion index according to the emotion classification result; The method of detecting a facial region from the enhanced image, performing emotion recognition based on the facial region, and generating an emotion classification result includes: The features in the facial region are located to determine the image position of the key points in the facial region; wherein the output is a bounding box of the facial region R , assuming it is the coordinate of the upper left corner of the rectangular box and the lower right corner coordinates , the formula is shown in (1): (1); Perform an affine transformation or a more complex 3D transformation on the detected facial area, aligning the transformation matrix T with the left eye coordinates in the source image , right eye coordinates , nose coordinates , left mouth corner coordinates And the coordinates of the right corner of the mouth The position coordinates are calculated; assuming that the key point position is and , respectively represent the position coordinate sets of key points in the source image and the target image, and their calculation formula is shown in (2): ; ; ; (2); The alignment transformation matrix T is calculated as follows, as shown in formula (3): (3); Then, apply the change matrix To the facial area Above, we get the aligned facial image for: ; Extracting emotion-related features from the facial region based on the image position; extracting the position information of facial key points, using a feature extraction algorithm to obtain the position of facial key points, assuming that the key point extraction function is The key point position is , assuming the key point Include points, the position coordinates of each point are ,but As shown in formula (4): (4) Analyze the texture changes of facial skin. Assume that the texture analysis function is TA and the texture feature is , the extraction formula is shown in (5), where Information representing multiple feature points of the skin: (5); Based on the features and the emotion label classifier, the emotion classification result corresponding to the features is determined; wherein the classifier is trained using a facial image dataset annotated with emotion labels, the extracted features are matched with the trained classifier, and the emotion classification result, i.e., the emotion label, is output. ,in Represents the emotional feature vector; the extracted different features are combined into a combined feature vector , combined feature vector As shown in formula (6): (6); Based on the combined feature vector, emotion classification is performed to generate the emotion classification result: , where C represents the classifier used in training.

2. The method for monitoring data indicators of critically ill patients according to claim 1, characterized in that: Acquiring a facial image of the patient through a camera device, including: capturing a target image by a camera device; Based on the pixel value corresponding to each pixel point in the target image, converting the target image into a grayscale image; Detecting whether the grayscale image contains a face based on a face detection algorithm; If yes, extracting the feature information corresponding to the grayscale image at the preset key point; Comparing the feature information with pre-stored patient features to determine identity information corresponding to the face in the target image; If the identity information is patient information, the target image is used as the facial image.

3. The method for monitoring data indicators of critically ill patients according to claim 2, characterized in that: Also includes: Tracking is performed based on the facial area in the target image, and the facial image of the patient is acquired based on a preset acquisition cycle.

4. The method for monitoring data indicators of critically ill patients according to claim 1, characterized in that: Based on the correspondence between the emotion index and the monitoring data, target data corresponding to the emotion index is filtered from the monitoring data, including: Pre-establishing a correspondence between the emotion index and the monitoring data, wherein the monitoring data includes at least one of the following: blood pressure data, electrocardiogram data, respiratory data, body temperature data, and nervous system data; Based on the corresponding relationship, target data corresponding to the emotion index is filtered out from the detection data.

5. The method for monitoring data indicators of critically ill patients according to claim 1, characterized in that: After a comprehensive analysis is performed based on the emotion index and the target data to generate the indicator monitoring results and accompanying suggestions for the patient, the following steps are also included: Generate a monitoring report based on the emotion index, target data, and generated indicator monitoring results and accompanying care suggestions; The monitoring report is sent to the caregiver and the attending physician.

6. A critically ill patient data indicator monitoring system, characterized in that: include: An acquisition unit, used for acquiring a facial image of a patient through a camera device; an enhancement unit, configured to perform image enhancement on the facial image to generate an enhanced image, perform emotion recognition on the enhanced image, and determine a current emotion index of the patient; A monitoring unit, used to obtain various monitoring data of the patient through a sensor device; A screening unit, configured to screen target data corresponding to the emotion index from the monitoring data based on the corresponding relationship between the emotion index and the monitoring data; An analysis unit, configured to perform a comprehensive analysis based on the emotion index and the target data to generate an indicator monitoring result and accompanying care suggestions for the patient; The step of performing image enhancement on the facial image to generate an enhanced image, and performing emotion recognition on the enhanced image to determine the patient's current emotion index includes: Performing image enhancement on the facial image to generate an enhanced image; Detecting a facial region from the enhanced image, performing emotion recognition based on the facial region, and generating an emotion classification result; Determining an emotion index according to the emotion classification result; The method of detecting a facial region from the enhanced image, performing emotion recognition based on the facial region, and generating an emotion classification result includes: The features in the facial region are located to determine the image position of the key points in the facial region; wherein the output is a bounding box of the facial region R , assuming it is the coordinate of the upper left corner of the rectangular box and the lower right corner coordinates , the formula is shown in (1): (1); Perform an affine transformation or a more complex 3D transformation on the detected facial area, aligning the transformation matrix T with the left eye coordinates in the source image , right eye coordinates , nose coordinates , left mouth corner coordinates And the coordinates of the right corner of the mouth The position coordinates are calculated; assuming that the key point position is and , respectively represent the position coordinate sets of key points in the source image and the target image, and their calculation formula is shown in (2): ; ; ; (2); The alignment transformation matrix T is calculated as follows, as shown in formula (3): (3); Then, apply the change matrix To the facial area Above, we get the aligned facial image for: ; Extracting emotion-related features from the facial region based on the image position; extracting the position information of facial key points, using a feature extraction algorithm to obtain the position of facial key points, assuming that the key point extraction function is The key point position is , assuming the key point Include points, the position coordinates of each point are ,but As shown in formula (4): (4) Analyze the texture changes of facial skin. Assume that the texture analysis function is TA and the texture feature is , the extraction formula is shown in (5), where Information representing multiple feature points of the skin: (5); Based on the features and the emotion label classifier, the emotion classification result corresponding to the features is determined; wherein the classifier is trained using a facial image dataset annotated with emotion labels, the extracted features are matched with the trained classifier, and the emotion classification result, i.e., the emotion label, is output. ,in Represents the emotional feature vector; the extracted different features are combined into a combined feature vector , combined feature vector As shown in formula (6): (6); Based on the combined feature vector, emotion classification is performed to generate the emotion classification result: , where C represents the classifier used in training.

7. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for monitoring data indicators of critically ill patients according to any one of claims 1 to 5 is implemented.

8. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the method for monitoring data indicators of critically ill patients as described in any one of claims 1 to 5.