An artificial intelligence anesthesia recovery monitoring system and method

Abstract

The application relates to an artificial intelligence anesthesia recovery monitoring system and method, characterized in that the system comprises a camera, an upper computer and a loudspeaker; the camera is used for collecting real-time facial and eye images of a patient; the upper computer is used for recognizing, calculating and judging the facial and eye images collected by the camera, determining the eye aspect ratio of the patient, determining whether the patient is awake according to a pre-set patient awakening threshold and a patient eye closing threshold, sending an alarm signal to the loudspeaker, and displaying the processed facial and eye images, the eye aspect ratio and a picture alarm; and the loudspeaker is used for emitting an alarm sound according to the alarm signal sent by the upper computer. The application can be applied to an artificial intelligence anesthesia recovery monitoring system in various scenes such as operating rooms, anesthesia recovery rooms and ICUs, and can be widely applied to the anesthesia recovery technical field.

Description

Technical Field

[0001] This invention relates to the field of anesthesia recovery technology, and in particular to an artificial intelligence anesthesia recovery monitoring system and method. Background Technology

[0002] The American Society of Anesthesiologists (ASA) considers avoiding intraoperative awareness a primary goal of anesthesia. Although the incidence of intraoperative awareness is low, the psychological sequelae (including psychological and behavioral abnormalities such as sleep disturbances, anxiety, and mental disorders) can last for months or years. Patients with intraoperative awareness often require psychological treatment, and in severe cases, it can develop into post-traumatic stress disorder (PTSD). PTSD is a syndrome characterized by the involuntary recurrence of traumatic situations in the patient's thoughts and memories after experiencing a traumatic event, avoidance, and physiological overreaction. Recent statistics show that approximately 22% of patients with intraoperative awareness develop this sequela. In a study, researchers followed up patients who experienced intraoperative awareness, and the incidence of PTSD was 71%, significantly higher than the control group (12%), with symptoms persisting for an average of 4.7 years. This indicates that patients with intraoperative awareness are more prone to later psychological disorders, and the incidence and duration of PTSD are high. Intraoperative awareness can cause an unpleasant anesthesia experience for patients and lead to serious sequelae. Therefore, intraoperative awareness is not only a concern for patients and a serious cause of anesthetic complications, but also one of the causes of medical disputes for anesthesiologists.

[0003] In recent years, bispectral index (BIS), entropy index, Narcotrend anesthesia / EEG depth of consciousness monitoring index (NI), and auditory evoked potential index (AEPI) have emerged to monitor the depth of anesthesia during surgery. However, studies have shown that these indicators can only partially reduce the incidence of intraoperative awareness and cannot accurately detect intraoperative awareness.

[0004] Currently, the most widely used BIS (Bipolar Injection System) requires electrode pads to be attached to the patient's forehead, which can affect the exposure of part of the surgical field and is not suitable for monitoring all surgeries. Furthermore, the device is expensive and bulky, posing a challenge for some small and medium-sized hospitals or hospitals in remote areas to purchase it. In addition, BIS EEG monitoring can be inaccurate, such as EEG artifacts or poor signal quality. Considerations must also be made when using and interpreting BIS data for patients with confirmed mental disorders, patients taking psychotropic medications, and children under one year old; therefore, BIS EEG monitoring alone is not feasible. At present, whether in the operating room, post-anesthesia recovery room, or intensive care unit (ICU), the most common method for determining patient awakening is still calling the patient's name, without the application of anesthesia awakening monitoring equipment. However, this method requires doctors or nurses to be at the patient's bedside, which consumes a lot of medical staff's energy if the patient's awakening is delayed, resulting in low efficiency as one medical staff member can only care for one patient. Summary of the Invention

[0005] To address the aforementioned problems, the purpose of this invention is to provide an artificial intelligence anesthesia recovery monitoring system and method that can monitor the patient's recovery status in real time and remotely.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: On the one hand, it provides an artificial intelligence anesthesia recovery monitoring system, characterized in that it includes a camera, a host computer, and a speaker;

[0007] The camera is used to capture images of the patient's face and eyes in real time;

[0008] The host computer is used to recognize, calculate and judge the face and eye images captured by the camera, determine the aspect ratio of the patient's eyes, determine whether the patient has woken up according to the preset patient awakening threshold and patient eye closing threshold, send an alarm signal to the speaker, and display the processed face and eye images, the aspect ratio of the eyes and the screen alarm.

[0009] The speaker is used to emit an alarm sound according to the alarm signal sent by the host computer.

[0010] Furthermore, the artificial intelligence anesthesia recovery monitoring system also includes an adjustable fixing device, which is installed on the headboard of the operating table or the headboard of the operating cart for placing the camera.

[0011] Furthermore, the artificial intelligence anesthesia recovery monitoring system also includes a local server or cloud, which is connected to the host computer via a wired or wireless network. The local server or cloud is used to record and store the analysis results of the host computer, individualized patient recovery thresholds and patient eye-closing thresholds, patient basic information, preoperative monitoring entry thresholds, intraoperative monitoring information and images, and information on the relationship between surgical time and the frequency of eye changes.

[0012] Furthermore, the host computer includes a running host and a display;

[0013] The host computer is used to identify, calculate and judge the face and eye images and the aspect ratio of the eyes captured by the camera, determine the aspect ratio of the patient's eyes, and determine whether the patient has woken up according to the preset patient awakening threshold and patient eye closing threshold, and send an alarm signal to the display and speaker.

[0014] The display is used to show processed facial and eye images and the aspect ratio of the eyes, as well as to provide a visual alert when the aspect ratio of the eyes exceeds a set patient awakening threshold.

[0015] Furthermore, the host computer is equipped with:

[0016] The patient information entry module is used to enter the patient's basic information, the default patient awakening threshold and the patient eye-closing threshold, and to update the default patient awakening threshold and the patient eye-closing threshold through an AI self-learning library;

[0017] The awakening monitoring module is used to recognize and process the images of the patient's face and eyes captured by the camera, calculate the aspect ratio of the patient's eyes, and monitor whether the patient has awakened.

[0018] The individualized patient threshold setting module is used to set individualized patient awakening threshold and patient eye-closing threshold based on the average of the eye aspect ratio of several eye-opening and eye-closing actions of the patient acquired by the camera.

[0019] The awakening monitoring and alarm module is used to send an alarm signal to the display and speaker when the aspect ratio of the eye exceeds the set patient awakening threshold;

[0020] The AI ​​self-learning module is used to analyze, compare, and learn the data entered and monitored by the patient information entry module, the individualized patient threshold setting module, and the awakening monitoring module, and generate different awakening threshold models and eye-closing threshold models, which are then stored in the local server or the cloud.

[0021] Furthermore, the awakening monitoring module is equipped with:

[0022] An image acquisition unit is used to acquire face and eye images captured by the camera;

[0023] The image setting unit is used to set the resolution and grayscale of each acquired image frame;

[0024] The facial feature point localization unit is used to locate facial feature points in each frame of the image after image setting, based on the feature point detection library of facial recognition.

[0025] The eye feature point localization unit is used to locate eye feature points based on each frame of the image after facial feature point localization using a facial feature position detector, and obtain the coordinates corresponding to each eye feature point.

[0026] The EAR value calculation unit is used to calculate the EAR value of the patient's left eye and right eye based on each frame of the image after localization of eye feature points;

[0027] The awakening status assessment unit is used to determine the patient's awakening status based on the calculated EAR values ​​of the left and right eyes, as well as the set patient awakening threshold and patient eye-closing threshold.

[0028] Furthermore, the judgment process of the awakening status judgment unit is as follows:

[0029] When the average EAR value of the left eye and the right eye is greater than or equal to the patient's awakening threshold, the patient is determined to have opened their eyes, and an alarm signal is sent to the display and speaker.

[0030] When the patient's eye-closing threshold is less than the average of the left and right eye EAR values ​​and less than the patient's awakening threshold, it is counted as one squinting state. If the squinting state count exceeds the squinting count threshold, it is determined as rapid eye twitching, and an alarm signal is sent to the display and speaker.

[0031] When the average EAR value of the left eye and the right eye is less than or equal to the patient's eye-closing threshold, the count is reset to zero, the display prompt disappears, and the speaker stops alarming.

[0032] Furthermore, the AI ​​self-learning module is equipped with:

[0033] The threshold model generation unit is used to analyze, compare and learn the data entered and monitored by the patient information entry module, the individualized patient threshold setting module and the awakening monitoring module, generate different awakening threshold models and closed-eye threshold models and store them in the local server or cloud.

[0034] The threshold suggestion unit is used to query the awakening threshold model and the eye-closing threshold model that match the patient's basic information in the local server or the cloud, and give the patient awakening threshold and patient eye-closing threshold suggestions as the default patient awakening threshold and patient eye-closing threshold.

[0035] On the other hand, an artificial intelligence-based method for monitoring anesthesia recovery is provided, including:

[0036] The camera captures images of the patient's face and eyes in real time;

[0037] The host computer recognizes, calculates, and judges the face and eye images captured by the camera, determines the aspect ratio of the patient's eyes, and determines whether the patient has woken up based on the preset patient awakening threshold and patient eye closing threshold. It also sends an alarm signal to the speaker and displays the processed face and eye images, the aspect ratio of the eyes, and the screen alarm.

[0038] The speaker emits an alarm sound based on the alarm signal sent by the host computer.

[0039] Furthermore, the host computer performs recognition, calculation, and judgment processing on the face and eye images captured by the camera, determines the aspect ratio of the patient's eyes, and determines whether the patient has awakened based on preset patient awakening thresholds and patient eye-closing thresholds. It then sends an alarm signal to the speaker and displays the processed face and eye images, eye aspect ratio, and screen alarm, including:

[0040] The host computer enters the patient's basic information;

[0041] The host machine determines the patient awakening threshold and the patient eye-closing threshold as either the default patient awakening threshold and the patient eye-closing threshold or an individualized patient awakening threshold and the patient eye-closing threshold.

[0042] The host computer processes and identifies the images of the patient's face and eyes captured by the camera, and calculates the aspect ratio of the patient's eyes in order to monitor whether the patient has regained consciousness.

[0043] When the aspect ratio of the eye exceeds the set patient awakening threshold, the host computer sends an alarm signal to the monitor and speaker, and the monitor displays a visual reminder.

[0044] The host analyzes, compares, and learns from the patient's basic information, awakening threshold, eye-closing threshold, and facial and eye images captured by the camera. It generates different awakening threshold models and eye-closing threshold models and stores them on a local server or in the cloud. The host can then query the local server or cloud to find the awakening threshold model and eye-closing threshold model that match the patient and provide suggestions for the patient's awakening threshold and eye-closing threshold.

[0045] The present invention has the following advantages due to the adoption of the above technical solutions:

[0046] 1. This invention allows patients to input basic information through a patient information input module. It uses artificial intelligence algorithms to provide suggestions for default patient awakening thresholds and patient eye-closing thresholds based on machine learning results. Individualized patient awakening thresholds and patient eye-closing thresholds can also be set based on these suggestions.

[0047] 2. After the threshold is set, the present invention can acquire images of the patient's face and eyes through a monitoring camera, and perform recognition processing on the acquired images through the host computer to calculate the aspect ratio of the eyes. When the aspect ratio of the eyes exceeds the set patient awakening threshold, an alarm signal will be sent to the monitor for displaying monitoring information and the speaker for alarming monitoring information to alert the anesthesiologist.

[0048] 3. All monitoring information in this invention is stored on a local server or in the cloud for the learning and training of the AI ​​self-learning module, so as to provide more accurate suggestions on the patient awakening threshold and the patient eye-closing threshold.

[0049] 4. This invention is an AI-powered anesthesia recovery monitoring system applicable to various scenarios such as operating rooms, post-anesthesia recovery rooms, and ICUs. It can monitor the patient's recovery status in real time and remotely, and provide alarm prompts. It helps anesthesiologists to promptly detect intraoperative arousal during surgery, and frees anesthesiologists from waking patients in post-anesthesia recovery rooms and ICUs. The anesthesia recovery monitoring system can continuously improve its data and analysis, filling the gap in anesthesia recovery monitoring equipment.

[0050] 5. This invention is a non-invasive monitoring device that is lightweight and portable, has no disposable consumables, and is reusable. It is cheaper than the anesthesia depth monitoring devices used in clinical practice, which can reduce anesthesia-related costs and facilitate clinical promotion, especially for medical institutions that cannot purchase expensive anesthesia depth monitoring devices.

[0051] In summary, this invention can be widely applied in the field of anesthesia recovery technology. Attached Figure Description

[0052] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. In the drawings:

[0053] Figure 1 This is a system front view provided in an embodiment of the present invention;

[0054] Figure 2 This is a right view of a system provided in an embodiment of the present invention;

[0055] Figure 3This is a top view of a system provided in an embodiment of the present invention;

[0056] Figure 4 This is a schematic diagram of the system connection structure provided in an embodiment of the present invention;

[0057] Figure 5 This is a schematic diagram of the structure of a host computer provided in an embodiment of the present invention;

[0058] Figure 6 This is a schematic diagram illustrating the specific location of facial and eye feature points according to an embodiment of the present invention;

[0059] Figure 7 This is a schematic diagram illustrating the specific location of eye feature points according to an embodiment of the present invention, wherein, Figure 7 (a) is a schematic diagram showing the specific location of eye feature points when the eyes are open. Figure 7 (b) is a schematic diagram showing the specific location of eye feature points when the eyes are closed;

[0060] Figure 8 This is a flowchart illustrating the awakening monitoring module provided in an embodiment of the present invention;

[0061] Figure 9 This is a flowchart illustrating the AI ​​self-learning module provided in an embodiment of the present invention. Detailed Implementation

[0062] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the invention and to fully convey the scope of the invention to those skilled in the art.

[0063] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0064] Although terms such as first, second, third, etc., may be used in this document to describe multiple elements, components, regions, layers, and / or segments, these elements, components, regions, layers, and / or segments should not be limited by these terms. These terms may be used only to distinguish one element, component, region, layer, or segment from another. Unless the context clearly indicates otherwise, terms such as "first," "second," and other numerical terms used herein do not imply order or sequence. Therefore, the first element, component, region, layer, or segment discussed below may be referred to as the second element, component, region, layer, or segment without departing from the teachings of the exemplary embodiments.

[0065] The artificial intelligence anesthesia recovery monitoring system and method provided in this invention can be applied to various scenarios such as operating rooms, anesthesia recovery rooms, and ICUs. It can monitor the patient's recovery status in real time and remotely, and provide alarm prompts. It helps anesthesiologists to promptly detect intraoperative arousal, and frees anesthesiologists from waking patients in the anesthesia recovery room and ICU. Furthermore, the device can continuously and autonomously improve the anesthesia recovery monitoring system based on data analysis and judgment, filling the gap in anesthesia recovery monitoring equipment.

[0066] Example 1

[0067] like Figures 1 to 3 As shown, this embodiment provides an artificial intelligence anesthesia recovery monitoring system, including a camera 1, a host computer 2, and a speaker 3.

[0068] Camera 1 is used to capture images of the patient's face and eyes in real time.

[0069] The host computer 2 is used to recognize, calculate and judge the face and eye images captured by the camera 1, determine the patient's eye aspect ratio (EAR), and determine whether the patient has woken up according to the preset patient awakening threshold and patient eye closing threshold. It also sends an alarm signal to the speaker 3 and displays the processed face and eye images, eye aspect ratio and screen alarm, so that the anesthesiologist can monitor the patient in real time and analyze the patient's awakening status.

[0070] The speaker 3 is used to emit an alarm sound according to the alarm signal sent by the host computer 2, so as to remind the anesthesiologist to pay attention to the patient's awakening status.

[0071] In a preferred embodiment, such as Figure 2 and Figure 3 As shown, the artificial intelligence anesthesia recovery monitoring system also includes an adjustable fixing device 4, which is fixed on the headboard of the operating table or the headboard of the operating cart by rotating screws, for placing the camera 1 so that the camera 1 is located directly above the patient's head to capture images of the patient's face and eyes.

[0072] Specifically, the adjustable fixing device 4 is fixedly set on the headboard of the operating table or the headboard of the operating cart so that the camera 1 is located directly above the patient's face, about 30cm away from the patient's face, ensuring that the monitor 22 can see a complete image of the patient's face, especially the eye image is clear enough.

[0073] In a preferred embodiment, such as Figure 4 As shown, the AI-powered anesthesia recovery monitoring system also includes a local server or cloud server. The local server or cloud server is connected to the host computer 2 via wired or wireless means. The local server or cloud server is used to record and store the analysis results of the host computer 2, individualized patient recovery thresholds and patient eye-closing thresholds, patient basic information, preoperative monitoring entry thresholds, intraoperative monitoring information and images, and information on the relationship between surgical time and the frequency of eye changes. This data is used to further strengthen the training of the AI ​​self-learning module and provide more accurate suggested thresholds and intraoperative monitoring focus.

[0074] In a preferred embodiment, such as Figure 4 As shown, the host computer 2 includes a main unit 21 and a display 22. The display 22 is connected to the main unit 21 via a data cable. The main unit 21 is used to recognize, calculate, and process the face and eye images captured by the camera 1, determine the aspect ratio of the patient's eyes, and determine whether the patient has awakened according to a preset patient awakening threshold, sending an alarm signal to the display 22 and the speaker 3. The display 22 is used to display the processed face and eye images and the aspect ratio of the eyes, and to provide a visual alert when the aspect ratio of the eyes exceeds the preset patient awakening threshold.

[0075] Specifically, such as Figure 5 As shown, the host computer 21 is equipped with a patient information entry module, a resuscitation monitoring module, an individualized patient threshold setting module, a resuscitation monitoring display and alarm module, an AI self-learning module, and a data transmission module.

[0076] The patient information entry module is used to enter the patient's basic information, the default patient awakening threshold and the patient eye-closing threshold, and to update the default patient awakening threshold and the patient eye-closing threshold through an AI self-learning library. The patient's basic information includes the patient's age, gender, height, and weight.

[0077] The awakening monitoring module is used to recognize and process the images of the patient's face and eyes captured by camera 1, and calculate the aspect ratio of the patient's eyes in order to monitor whether the patient has awakened.

[0078] The individualized patient threshold setting module is used to set individualized patient awakening thresholds and patient eye-closing thresholds based on the average of the eye aspect ratio of the patient's natural eye-opening and eye-closing movements acquired by camera 1 (generally 3 to 5 times, but more times can also be collected. Although more times will increase the workload, it can reduce the error, so a trade-off needs to be made). The individualized patient awakening threshold is the average of the eye aspect ratio of the patient's eyes in several natural eye-opening movements, and the individualized patient eye-closing threshold is the average of the eye aspect ratio of the patient's eyes in several natural eye-closing movements.

[0079] The awakening monitoring and alarm module is used to send an alarm signal to the display 22 and the speaker 3 when the aspect ratio of the eye exceeds the set patient awakening threshold. The display 22 provides a visual reminder, and the speaker 3 emits an alarm sound.

[0080] The AI ​​self-learning module analyzes, compares, and learns from the data entered and monitored by the patient information entry module, the individualized patient threshold setting module, and the awakening monitoring module. It generates different awakening threshold models and eye-closing threshold models and stores them on a local server or in the cloud via the data transmission module. This allows anesthesiologists to query the awakening threshold model that matches the patient's basic information on the local server or in the cloud, and provide more accurate suggestions for the patient's awakening threshold and eye-closing threshold.

[0081] Specifically, the wake-up monitoring module includes an image acquisition unit, an image setting unit, a facial feature point positioning unit, an eye feature point positioning unit, an EAR value calculation unit, a wake-up status judgment unit, a convex hull point calculation unit, and a facial region drawing unit.

[0082] The image acquisition unit is used to acquire face and eye images captured by camera 1 using the VideoCapture() function in the OpenCV library.

[0083] The image setting unit is used to set the resolution and grayscale of each acquired image frame.

[0084] The facial feature point localization unit uses the `shape_predictor()` function from the dlib library, based on a 68-feature point detection library for face recognition, to perform facial feature point localization on each frame of the image after image definition, such as... Figure 6 As shown.

[0085] The eye feature point localization unit uses a facial feature position detector to locate eye feature points in each frame of the image after facial feature point localization, and obtains the coordinates of each eye feature point. Each eye includes 6 feature points, with coordinates from the outside to the inside represented by p1 to p6. p1 is the outer canthus, p2 is the junction of the outer iris and upper eyelid, p3 is the junction of the inner iris and upper eyelid, p4 is the inner canthus, p5 is the junction of the inner iris and lower eyelid, and p6 is the junction of the outer iris and upper eyelid. Figure 7 As shown.

[0086] The EAR value calculation unit is used to calculate the EAR values ​​of the patient's left and right eyes for each frame of the image after localization of eye feature points.

[0087]

[0088] The EAR value differs significantly when the eyes are open and closed.

[0089] The awakening status judgment unit is used to determine the patient's awakening status based on the calculated EAR values ​​of the left and right eyes, as well as the set patient awakening threshold and patient eye-closing threshold. More specifically, the judgment process is as follows: ① When the average of the left and right eye EAR values ​​is greater than or equal to the patient awakening threshold, the patient is determined to be awake, and an alarm signal is sent to the display 22 and the speaker 3. ② When the patient eye-closing threshold is less than the average of the left and right eye EAR values ​​and less than the patient awakening threshold, it is counted as one instance of squinting. If the squinting count exceeds 5 times, it is determined to be rapid eye twitching, and an alarm signal is sent to the display 22 and the speaker 3. ③ When the average of the left and right eye EAR values ​​is less than or equal to the patient eye-closing threshold, the count is reset to zero, the display 22 indicator disappears, and the speaker 3 stops alarming.

[0090] The convex hull point calculation unit is used to calculate the convex hull points of the patient's left and right eyes and draw curves to display on the monitor 22.

[0091] The face region rendering unit is used to render the face region curve for each frame of image after locating the face feature points and display it on the display 22.

[0092] Specifically, the AI ​​self-learning module includes a threshold model generation unit and a threshold suggestion unit.

[0093] The threshold model generation unit uses artificial intelligence algorithms to analyze, compare, and learn the data entered and monitored by the patient information entry module, the individualized patient threshold setting module, and the awakening monitoring module, and generates different awakening threshold models and eye-closing threshold models, which are then stored on a local server or in the cloud.

[0094] The threshold suggestion unit is used to query the awakening threshold model and the eye-closing threshold model that match the patient's basic information from the local server or the cloud, and provide suggestions for the patient's awakening threshold and the patient's eye-closing threshold as an AI self-learning library to update the default patient awakening threshold and the patient's eye-closing threshold. The given threshold may be a single value or multiple values, and the multiple values ​​may be set as a range.

[0095] Specifically, the specific model construction process for the threshold model generation unit is as follows:

[0096] Artificial intelligence algorithms are used to compare the average EAR values ​​of the patient's left and right eyes with the set individualized patient awakening and eye-closing thresholds or the default patient awakening and eye-closing thresholds, and then perform big data correction and updates. The results are then uploaded to a local server or the cloud.

[0097] More specifically, if the patient awakening threshold or the patient eye-closing threshold is suitable: the host computer 2 does not alarm and the anesthesiologist does not perform any operational intervention (the host computer 2 is not operated), no model comparison and correction update is required; wherein, the suitable patient awakening threshold or patient eye-closing threshold can be defined as: the patient awakening threshold and patient eye-closing threshold set when the patient is monitored by the artificial intelligence anesthesia awakening monitoring system of the present invention, and there is no situation where the anesthesiologist believes that the patient has not awakened or has no obvious signs of awakening, but the host computer 2 alarms (i.e., false positive); and also when the anesthesiologist believes that the patient has awakened or has obvious signs of awakening, but the host computer 2 does not alarm (i.e., false negative).

[0098] If the patient's awakening threshold or eye-closing threshold is too low (false positive): The host computer 2 will issue an alarm, but if the anesthesiologist believes that the patient has not awakened, the alarm will be cleared, and the patient's awakening threshold or eye-closing threshold will be actively increased. This value will be used as the new threshold for continued monitoring of the patient. Using artificial intelligence algorithms, the adjusted patient awakening threshold or eye-closing threshold will be compared with the set individualized patient awakening threshold and patient eye-closing threshold or the suggested patient awakening threshold and patient eye-closing threshold, and big data will be used for model correction and update. The results will be uploaded to the local server or the cloud.

[0099] If the patient's awakening threshold or eye-closing threshold is too high (false negative): The host computer 2 does not alarm, but the anesthesiologist believes that the patient has awakened and actively lowers the patient's awakening threshold or eye-closing threshold, then this value is used as the new threshold for continued monitoring of the patient; using artificial intelligence algorithms, the adjusted patient awakening threshold or patient eye-closing threshold is compared with the set individualized patient awakening threshold and patient eye-closing threshold or the suggested patient awakening threshold and patient eye-closing threshold for model correction and update, and the results are uploaded to the local server or cloud.

[0100] Specifically, the host machine 21 can use a Linux-based Raspberry Pi operating system, a Windows system, or a Mac system.

[0101] In a preferred embodiment, the camera 1 can be a CSI camera 1, a USB camera 1, or a wirelessly connected camera 1, and can be connected to the host 21 via a data cable or wireless Wi-Fi. When the camera 1 is connected via wireless Wi-Fi, remote monitoring and image acquisition can be achieved.

[0102] In a preferred embodiment, the speaker 3 may be a 3.5mm headphone jack speaker 3 or an HDMI-enabled speaker 3, and the speaker 3 is connected to the host computer 21 via a data cable.

[0103] Example 2

[0104] This embodiment provides an artificial intelligence-based method for monitoring anesthesia recovery, including the following steps:

[0105] 1) Camera 1 captures real-time images of the patient's face and eyes, specifically:

[0106] 1.1) Fix the camera 1 on the adjustable fixing device 4.

[0107] 1.2) The adjustable fixing device 4 is fixed on the headboard of the operating table or the headboard of the operating cart by rotating screws, so that the camera 1 is about 30cm away from the patient's face, ensuring that the patient's complete facial image can be seen on the monitor 22, especially the eye image is clear enough.

[0108] 1.3) Camera 1 captures images of the patient's face and eyes in real time.

[0109] 2) The host computer 2 recognizes, calculates, and processes the facial and eye images captured by the camera 1 to determine the aspect ratio of the patient's eyes. Based on pre-set patient awakening and eye-closing thresholds, it determines whether the patient has awakened, sends an alarm signal to the speaker 3, and displays the processed facial and eye images, eye aspect ratio, and alarm information. Specifically:

[0110] 2.1) The host computer 21 enters the patient's basic information, including the patient's age, gender, height and weight.

[0111] 2.2) The host machine 21 determines the patient awakening threshold and the patient eye-closing threshold as either the default patient awakening threshold and the patient eye-closing threshold or an individualized patient awakening threshold and the patient eye-closing threshold. Regardless of whether the default patient awakening threshold and the patient eye-closing threshold are set, as long as one is set, the monitoring of the patient's awakening status can begin.

[0112] 2.2.1) The data transmission module sends the patient's basic information to the local server or the cloud. The AI ​​self-learning module provides the default patient awakening threshold and patient eye-closing threshold based on the patient's basic information and displays them to the anesthesiologist on the monitor 22.

[0113] 2.2.2) The individualized patient threshold setting module sets individualized patient awakening threshold and patient eye-closing threshold based on the 3 to 5 natural eye-opening and eye-closing actions captured by the camera 1.

[0114] 2.3) As Figure 8 As shown, the host computer 21 performs recognition processing on the patient's face and eye images captured by the camera 1, calculates the aspect ratio of the patient's eyes, and monitors whether the patient has regained consciousness.

[0115] 2.3.1) The image acquisition unit of the wake-up monitoring module uses the VideoCapture() function in the OpenCV library to acquire face and eye images captured by camera 1.

[0116] 2.3.2) The image setting unit of the wake-up monitoring module sets the resolution and grayscale of the acquired image frame.

[0117] 2.3.3) The facial feature point localization unit of the wake-up monitoring module uses the shape_predictor() function in the dlib library, based on the 68 feature point detection library for face recognition, to locate facial feature points in the frame image after image setting.

[0118] 2.3.4) The eye feature point localization unit of the wake-up monitoring module adopts a facial feature position detector. Based on the frame image after facial feature point localization, the eye feature points are located and the coordinates corresponding to each eye feature point are obtained. Each eye includes 6 feature points, and the coordinates from the outside to the inside are represented by p1 to p6 respectively.

[0119] 2.3.5) The EAR value calculation unit of the awakening monitoring module calculates the EAR value of the patient's left eye and right eye based on the frame image after the eye feature points are located.

[0120] 2.3.6) The awakening status judgment unit of the awakening monitoring module determines the patient's awakening status based on the calculated left eye EAR value and right eye EAR value, as well as the set patient awakening threshold and patient eye-closing threshold:

[0121] ① When the average value of the EAR value of the left eye and the right eye is greater than or equal to the patient's awakening threshold, the patient is determined to have opened their eyes, and an alarm signal is sent to the display 22 and the speaker 3.

[0122] ② When the patient's eye-closing threshold is less than the average of the left and right eye EAR values ​​and less than the patient's awakening threshold, it is counted as 1 squinting state. If the squinting state count exceeds 5 times, it is determined as rapid eye twitching, and an alarm signal is sent to the display 22 and the speaker 3.

[0123] ③ When the average value of the left eye EAR value and the right eye EAR value is less than or equal to the patient's eye-closing threshold, the count is reset to zero, the display 22 disappears, and the speaker 3 stops alarming.

[0124] 2.3.7) The convex point calculation unit of the awakening monitoring module calculates the convex points of the patient's left and right eyes and plots the curves displayed on the display 22.

[0125] 2.3.8) The face region drawing unit of the wake-up monitoring module draws the face region curve based on each frame of the image after the facial feature points are located and displays it on the display 22.

[0126] 2.3.9) Once the host computer 21 completes the calculation and analysis in one go, proceed to step 2.3.2) to continue the next analysis process, performing cyclical analysis frame by frame.

[0127] 2.4) When the aspect ratio of the eye exceeds the set patient awakening threshold, the host 21 sends an alarm signal to the display 22 and the speaker 3, and the display 22 provides a visual reminder.

[0128] 2.5) such as Figure 9 As shown, the host computer 21 analyzes, compares, and learns from the patient's basic information, awakening threshold, eye-closing threshold, and facial and eye images captured by camera 1. It generates different awakening threshold models and eye-closing threshold models, storing them on a local server or in the cloud. This allows anesthesiologists to retrieve the appropriate awakening and eye-closing threshold models based on the patient's basic information, providing more accurate recommendations for patient awakening and eye-closing thresholds.

[0129] 2.5.1) The threshold model generation unit of the AI ​​self-learning module uses artificial intelligence algorithms to analyze, compare and learn the data entered and monitored by the patient information entry module, the individualized patient threshold setting module and the awakening monitoring module, and generate different awakening threshold models and closed-eye threshold models, which are then stored on the local server or in the cloud.

[0130] 2.5.2) The threshold suggestion unit of the AI ​​self-learning module queries the awakening threshold model and the eye-closing threshold model that match the patient's basic information in the local server or cloud, and gives the patient awakening threshold and patient eye-closing threshold suggestions as the AI ​​self-learning library to update the default patient awakening threshold and patient eye-closing threshold.

[0131] 3) The speaker 3 emits an alarm sound according to the alarm signal sent by the host computer 2.

[0132] The above embodiments are only used to illustrate the present invention. The structure, connection method and manufacturing process of each component can be varied. All equivalent transformations and improvements made on the basis of the technical solution of the present invention should not be excluded from the protection scope of the present invention.

Claims

1. An artificial intelligence-based intraoperative anesthesia recovery monitoring system, applied to intraoperative awareness monitoring, characterized in that, Includes camera, host computer, and speakers; The camera is used to capture images of the patient's face and eyes in real time; The host computer is used to recognize, calculate and judge the face and eye images captured by the camera, determine the aspect ratio of the patient's eyes, determine whether the patient has woken up according to the preset patient awakening threshold and patient eye closing threshold, send an alarm signal to the speaker, and display the processed face and eye images, the aspect ratio of the eyes and the screen alarm. The speaker is used to emit an alarm sound according to the alarm signal sent by the host computer; The host computer includes a running host, and the running host is equipped with: The patient information entry module is used to enter the patient's basic information, the default patient awakening threshold and the patient eye-closing threshold, and to update the default patient awakening threshold and the patient eye-closing threshold through an AI self-learning library; The awakening monitoring module is used to recognize and process the images of the patient's face and eyes captured by the camera, calculate the aspect ratio of the patient's eyes, and monitor whether the patient has awakened. The individualized patient threshold setting module is used to set individualized patient awakening threshold and patient eye-closing threshold based on the average of the eye aspect ratio of several eye-opening and eye-closing actions of the patient acquired by the camera. The awakening monitoring and alarm module is used to send an alarm signal to the speaker when the aspect ratio of the eye exceeds the set patient awakening threshold; The AI ​​self-learning module includes: The threshold model generation unit is used to analyze, compare and learn the data entered and monitored by the patient information entry module, the individualized patient threshold setting module and the awakening monitoring module, generate different awakening threshold models and closed-eye threshold models and store them in the local server or cloud. The threshold suggestion unit is used to query the awakening threshold model and the eye-closing threshold model that match the patient's basic information in the local server or cloud, and give the patient awakening threshold and patient eye-closing threshold suggestions as the default patient awakening threshold and patient eye-closing threshold. The awakening monitoring module is equipped with an awakening status judgment unit, and the judgment process of the awakening status judgment unit is as follows: When the average of the left eye EAR value and the right eye EAR value When the patient reaches the awakening threshold, the patient is determined to have opened their eyes, and an alarm signal is sent to the speaker. When the patient's eye-closing threshold is less than the average of the left and right eye EAR values ​​and less than the patient's awakening threshold, it is counted as one squinting state. If the squinting state count exceeds the squinting count threshold, it is determined as rapid eye twitching, and an alarm signal is sent to the speaker. When the average of the left eye EAR value and the right eye EAR value When the patient closes their eyes to the threshold, the count is reset to zero, and the speaker stops alarming; The model construction process of the threshold model generation unit is as follows: If the patient’s awakening threshold or the patient’s eye-closing threshold is appropriate, that is, the host computer does not alarm and the anesthesiologist does not perform any operation intervention, there is no need to perform model comparison and correction updates. If the patient’s awakening threshold or the patient’s eye-closing threshold is too low, i.e. the host computer alarms, but the anesthesiologist believes that the patient has not awakened, he will actively raise the patient’s awakening threshold or the patient’s eye-closing threshold and use the value as the new threshold to continue monitoring the patient. Artificial intelligence algorithms are used to compare the adjusted patient awakening threshold or patient eye-closing threshold with the set individualized patient awakening threshold and patient eye-closing threshold or the default patient awakening threshold and patient eye-closing threshold, and to make model corrections and updates based on big data. If the patient’s awakening threshold or the patient’s eye-closing threshold is too high, that is, the host computer does not alarm, but the anesthesiologist believes that the patient has awakened and actively lowers the patient’s awakening threshold or the patient’s eye-closing threshold, then the value is used as the new threshold to continue monitoring the patient. Artificial intelligence algorithms are used to compare and update the adjusted patient awakening threshold or patient eye-closing threshold with the set individualized patient awakening threshold and patient eye-closing threshold or the default patient awakening threshold and patient eye-closing threshold using a model and big data correction.

2. The artificial intelligence intraoperative anesthesia recovery monitoring system as described in claim 1, characterized in that, The AI-powered intraoperative anesthesia recovery monitoring system also includes an adjustable fixation device, which is mounted on the headboard of the operating table or the headboard of the operating cart to house the camera.

3. The artificial intelligence intraoperative anesthesia recovery monitoring system as described in claim 1, characterized in that, The artificial intelligence intraoperative anesthesia recovery monitoring system also includes a local server or cloud, which is connected to the host computer via a wired or wireless network. The local server or cloud is used to record and store the analysis results of the host computer, individualized patient recovery thresholds and patient eye closure thresholds, patient basic information, preoperative monitoring entry thresholds, intraoperative monitoring information and images, and information on the relationship between surgical time and the frequency of eye changes.

4. The artificial intelligence intraoperative anesthesia recovery monitoring system as described in claim 3, characterized in that, The host computer also includes a display; The host computer is used to identify, calculate and judge the face and eye images and the aspect ratio of the eyes captured by the camera, determine the aspect ratio of the patient's eyes, and determine whether the patient has woken up according to the preset patient awakening threshold and patient eye closing threshold, and send an alarm signal to the display and speaker. The display is used to show processed facial and eye images and the aspect ratio of the eyes, as well as to provide a visual alert when the aspect ratio of the eyes exceeds a set patient awakening threshold.

5. The artificial intelligence intraoperative anesthesia recovery monitoring system as described in claim 1, characterized in that, The awakening monitoring module is equipped with: An image acquisition unit is used to acquire face and eye images captured by the camera; The image setting unit is used to set the resolution and grayscale of each acquired image frame; The facial feature point localization unit is used to locate facial feature points in each frame of the image after image setting, based on the feature point detection library for facial recognition. The eye feature point localization unit is used to locate eye feature points based on each frame of the image after facial feature point localization using a facial feature position detector, and obtain the coordinates corresponding to each eye feature point. The EAR value calculation unit is used to calculate the EAR value of the patient's left eye and right eye based on each frame of the image after localization of eye feature points; The awakening status assessment unit is used to determine the patient's awakening status based on the calculated EAR values ​​of the left and right eyes, as well as the set patient awakening threshold and patient eye-closing threshold.

Images