Anesthesia depth intelligent monitoring system based on facial features

Abstract

The application provides a facial feature-based anesthesia depth intelligent monitoring system, and relates to the technical fields of medical monitoring and anesthesia depth evaluation.The system comprises a video acquisition module, which continuously acquires a patient facial video stream; a key point detection module, which locates facial key point coordinates and inter-frame displacement change quantity for each frame of image; a feature extraction module, which extracts eyelid movement features based on the coordinates and counts facial expression muscle effective movement frequency features based on the displacement change quantity within a preset time window; a depth evaluation module, which uses a pre-trained target neural network to calculate an anesthesia depth index by taking the eyelid movement features and the effective movement frequency features as multi-modal inputs; and an anesthesia depth determination module, which determines a patient anesthesia depth grade based on the anesthesia depth index. The system uses non-contact facial video to real-time extract multi-modal feature inputs into a neural network, reduces device cost and operation interference, and improves clinical practicability.

Description

Technical Field

[0001] This invention relates to the field of medical monitoring and anesthesia depth assessment technology, and in particular to an intelligent anesthesia depth monitoring system based on facial features. Background Technology

[0002] Currently, monitoring the depth of anesthesia is a crucial step in ensuring surgical safety and preventing intraoperative awareness or excessive anesthesia. Traditional methods for monitoring the depth of anesthesia mainly rely on electrophysiological parameters such as bispectral index (BIS) and entropy index. These methods require special electrodes to be attached to the patient's forehead and are contact-based measurements. This method has problems such as high electrode consumption costs, allergies in patients with sensitive skin, and unstable signals in electromagnetic interference environments. In addition, some patients cannot have electrodes attached due to traumatic brain injury, forehead bandage, or special body position, which limits the application of this type of method. In recent years, researchers have attempted to use facial video analysis to assess the state of anesthesia, for example, by detecting single indicators such as blinking frequency and pupil changes to assess the level of consciousness. However, simple eye features are easily affected by surgical lighting, facial obstruction, or passive eyelid closure by the patient, and cannot fully reflect the overall activity inhibition pattern of facial expression muscles during anesthesia induction and maintenance. In addition, existing methods lack the ability to dynamically capture the trend of motion decay, making it difficult to distinguish between effective movement and minor tremors. Moreover, most of them use fixed rules or traditional machine learning models, which are not adaptable to individual differences and rapid changes in anesthesia state, making it difficult to accurately and reliably monitor the depth of anesthesia. Therefore, in order to overcome the above-mentioned defects, the present invention provides an intelligent anesthesia depth monitoring system based on facial features. Summary of the Invention

[0003] Therefore, the purpose of this invention is to propose a remote monitoring and interactive control method for intelligent pet carriers, which improves the stability and accuracy of depth index and provides intuitive and quantitative decision-making basis for clinical practice.

[0004] To achieve the above objectives, the first aspect of this invention proposes an intelligent anesthesia depth monitoring system based on facial features, comprising: The video acquisition module is used to continuously acquire video stream data of the patient's facial area during anesthesia; The key point detection module is used to locate facial key points in each frame of the video stream data, and obtain the spatial coordinates of each key point in the image coordinate system and the displacement change between adjacent frames. The feature extraction module is used to extract eyelid movement features in the eye region based on spatial location coordinates, and to count the effective movement frequency features of facial expression muscles based on the displacement change within a preset time window. The depth assessment module is used to calculate the anesthesia depth index based on the eyelid movement features and the effective movement frequency features of facial expression muscles as multimodal input variables using a pre-trained target neural network. The anesthesia depth determination module is used to determine the current level of anesthesia for the patient based on the anesthesia depth index.

[0005] To improve the stability and accuracy of the depth index, the system will ultimately output a continuous index and convert it into anesthesia depth levels, providing clinicians with an intuitive and quantitative basis for decision-making.

[0006] In one possible implementation, the video acquisition module includes: A light sensing unit is used to detect the light intensity of the surgical environment in real time. The frame quality assessment unit is used to assess the quality of the acquired video frames in real time and calculate the frame quality score. The preprocessing unit is used to perform adaptive enhancement processing on the video stream when the frame quality score is less than a preset quality threshold and the illumination intensity is less than a preset illumination threshold. The enhancement intensity of the adaptive enhancement processing is negatively correlated with the frame quality score and the illumination intensity value, respectively.

[0007] While avoiding unnecessary computational overhead, it significantly improves the clarity and stability of facial videos, providing reliable input for subsequent key point detection and anesthesia depth assessment.

[0008] In one possible implementation, the key point detection module includes: The face detection unit is used to detect the face region in each frame of the image and output the face region location and detection confidence. The key point localization unit is used to locate key points of the eyes and key points of facial expression muscles within the detected facial area, and outputs the spatial coordinates and localization reliability of each key point. The key point quality assessment unit is used to compare the location confidence of each key point in the current frame with the average location confidence of the key point in a preset historical time window, and calculate the key point quality deviation rate. The key point stability evaluation unit is used to calculate the positional offset between each key point in the current frame and the corresponding key point in the previous frame, and compare the positional offset with a preset jitter threshold. The displacement calculation unit is used to calculate the Euclidean distance between corresponding key points in adjacent frames and obtain the displacement change when the key point quality deviation rate is less than a preset quality deviation threshold and the detection confidence is greater than a preset detection confidence threshold.

[0009] It effectively filters out false displacement noise caused by sudden changes in lighting, facial occlusion, or slight shaking, significantly improving the continuity of key point trajectories and the reliability of displacement calculation, providing a high-quality data foundation for subsequent accurate statistics of the effective movement frequency of facial expression muscles.

[0010] In one possible implementation, the eyelid movement features include at least one of eyelid closure degree, blink frequency, and blink duration.

[0011] In one possible implementation, the feature extraction module includes: The acquisition unit is used to acquire the displacement changes of the same facial key points between adjacent frames. The comparison unit is used to obtain a preset displacement threshold and compare the displacement change of the same facial key point between adjacent frames with the preset displacement threshold to determine whether the facial expression muscles to which the same facial key point belongs have undergone effective movement; when the displacement change of the same facial key point between adjacent frames is equal to or greater than the preset displacement threshold, it is determined that the facial expression muscles to which the same facial key point belongs have undergone effective movement; otherwise, it is determined that the facial expression muscles to which the same facial key point belongs have not undergone effective movement. The statistics unit is used to count the number of effective movements within the preset time window to obtain the frequency characteristics of effective movements. The attenuation trend monitoring unit is used to count the effective motion frequency in multiple adjacent preset time windows, calculate the frequency change rate between adjacent windows, and obtain the motion attenuation trend characteristics. The depth assessment module uses the eyelid movement characteristics, the effective movement frequency characteristics, and the movement decay trend characteristics as multimodal input variables to calculate the anesthesia depth index.

[0012] It effectively distinguishes between voluntary effective movement and non-specific interference, thereby significantly improving the sensitivity and accuracy of the anesthesia depth index in detecting changes in consciousness level.

[0013] One possible implementation also includes: The motion trend prediction unit is used to construct a linear prediction model based on the frequency change rate, predict the estimated value of the effective motion frequency at the end of the current preset time window based on the linear prediction model, and calculate the prediction deviation rate between the estimated value and the preset effective motion threshold. The historical baseline analysis unit is used to obtain the frequency of the patient's historical effective motion baseline during the stable phase of anesthesia induction and to calculate the attenuation deviation of the estimated value relative to the frequency of the historical effective motion baseline. The motion decay warning unit is used to determine that facial expression muscle activity has entered a rapid inhibition phase and output an indication of deepening anesthesia trend only when the estimated deviation rate is equal to or greater than a preset deviation threshold and the decay deviation is equal to or greater than a preset deviation threshold. The frequency acquisition adaptive unit is used for: When the anesthesia deepening trend is received, the weighted fusion value of the estimated deviation rate and the attenuation deviation is determined, and the target sampling resource budget is determined by querying the preset resource budget mapping table according to the weighted fusion value. The target sampling resource budget increases monotonically according to the weighted fusion value. Under the target sampling resource budget constraint, the combination of target enhancement frequency and target enhancement duration is solved to maximize the target enhancement frequency, and the product of the target enhancement frequency and the target enhancement duration is equal to the target sampling resource budget. Wherein, the target enhancement frequency is less than or equal to the maximum supported frequency of video acquisition, and the target enhancement duration is less than or equal to the minimum inference duration required by the depth evaluation module; Before the current preset time window ends, the acquisition frequency is pre-increased at the target enhancement frequency, and the duration of the next preset time window is updated to the target enhancement duration.

[0014] It significantly improves the temporal resolution of the rapid changes in anesthesia depth, providing a denser and more targeted sequence of motion features for the subsequent depth assessment module, and enhancing the system's ability to track and respond to inflection points in anesthesia depth.

[0015] In one possible implementation, the target neural network in the deep evaluation module includes: The first coding unit is used to perform temporal coding on eyelid movement features to obtain a temporal representation of the eyelid; The second coding unit is used to perform temporal coding on the effective motion frequency features to obtain a temporal representation of facial motion. The third coding unit is used to perform temporal coding on the motion decay trend features to obtain a temporal representation of the decay trend. A timing alignment evaluation unit is used to calculate the timing alignment deviation between the eyelid timing representation and the facial motion timing representation; The attenuation state assessment unit is used to calculate the attenuation acceleration based on the motion attenuation trend characteristics. The fusion unit is used to interactively fuse the eyelid temporal representation, the facial motion temporal representation, and the decay trend temporal representation to obtain a fused representation; The fusion unit includes a preset fusion weight mapping table. The preset fusion weight mapping table uses the temporal alignment deviation and the decay acceleration as indexes to record the fusion weight ratio of the eyelid temporal representation, the facial motion temporal representation, and the decay trend temporal representation. When the temporal alignment deviation is less than a preset alignment threshold and the absolute value of the decay acceleration is less than a preset smoothing threshold, the preset fusion weight mapping table is queried to obtain a first weight ratio. In the first weight ratio, the sum of the weights of the eyelid temporal representation and the facial motion temporal representation is greater than the weight of the decay trend temporal representation. Otherwise, the second weight ratio is obtained by querying the preset fusion weight mapping table, in which the weight of the decay trend time sequence representation is greater than the sum of the weights of the eyelid time sequence representation and the facial movement time sequence representation; The regression output subunit is used to input the fused representation into the fully connected regression network, which maps it to the regression feature space through two fully connected hidden layers, and outputs a continuous value as the anesthesia depth index through a single neuron linear output layer. The anesthesia depth index ranges from (0 to 100), and the larger the value, the deeper the anesthesia. The network parameters of the first coding unit, the second coding unit, the third coding unit, the temporal alignment evaluation unit, the decay state evaluation unit, the fusion unit, and the regression output unit are synchronously updated through end-to-end training.

[0016] It significantly improved the robustness and sensitivity of the anesthesia depth index to changes in consciousness level, while also enhancing the network's ability to generalize to complex anesthetic states.

[0017] In one possible implementation, the joint loss function for end-to-end training includes: The prediction error loss term is used to calculate the mean square error between the predicted value of the anesthesia depth index and the clinically labeled value. The mapping table smoothing loss term is used to calculate the rate of change between the fusion weight ratios corresponding to adjacent index positions in the preset fusion weight mapping table. When the rate of change exceeds a preset smoothing threshold, a penalty is applied. The joint loss function is a weighted sum of the prediction error loss term and the mapping table smoothing loss term.

[0018] By combining prediction error and mapping table smoothing loss for end-to-end training, we can ensure a high degree of consistency between the anesthesia depth index and the clinically labeled value, while also constraining the gradual change in the adjacent ratio of the weight mapping table, thereby improving the generalization stability and clinical interpretability of the model under different anesthesia states.

[0019] In one possible implementation, the anesthesia depth determination module includes: A grading unit is used to map the anesthesia depth index to discrete anesthesia depth levels, wherein the discrete anesthesia depth levels include: conscious state, light anesthesia state, adequate anesthesia state, and deep anesthesia state. The alarm output unit is used to output graded alarm signals according to discrete anesthesia depth levels, specifically: When the anesthesia depth level is at the level of awake state, a level one prompt signal is output; When the anesthesia depth level is light anesthesia, a level 2 attention signal is output. When the anesthesia depth level is deep anesthesia, a level three danger signal is output. The appropriate anesthesia state does not trigger an alarm signal.

[0020] The tiered alarm mechanism enables medical staff to intuitively and quickly identify a patient's level of consciousness and risk, thereby improving the safety of anesthesia management and the efficiency of clinical response.

[0021] In one possible implementation, the system further includes an edge deployment module for quantizing the target neural network and deploying it to the anesthesia monitoring device for real-time inference.

[0022] By quantizing neural networks and deploying them on anesthesia monitoring equipment, real-time inference with low latency and high privacy protection is achieved, avoiding network latency and data security risks caused by uploading video data to the cloud, and significantly improving clinical usability and response efficiency.

[0023] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in this application.

[0024] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0025] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a structural diagram of an intelligent anesthesia depth monitoring system based on facial features, as described in an embodiment of the present invention. Figure 2 This is a structural diagram of the video acquisition module in an intelligent anesthesia depth monitoring system based on facial features, as described in an embodiment of the present invention. Figure 3 This is a structural diagram of a key point detection module in an intelligent anesthesia depth monitoring system based on facial features, as described in an embodiment of the present invention. Figure 4 This is a structural diagram of the feature extraction module in an intelligent anesthesia depth monitoring system based on facial features, according to an embodiment of the present invention. Figure 5 This is a structural diagram of the anesthesia depth determination module in an intelligent anesthesia depth monitoring system based on facial features, as described in an embodiment of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of this application, unless otherwise stated, " / " indicates that the objects before and after are in an "or" relationship; for example, A / B can represent A or B. "And / or" in this application merely describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. Additionally, to facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first," "second," etc., are used in the embodiments of this application to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or order of execution, and that "first," "second," etc., are not necessarily different. Furthermore, in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate that something is being used as an example, illustration, or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner for ease of understanding.

[0027] Furthermore, the business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0028] The application of facial feature-based intelligent monitoring of anesthesia depth involves multiple scenarios, including routine anesthesia monitoring in the operating room, sedation assessment in the ICU, monitoring of painless treatment in outpatient clinics, monitoring during emergency transport, and simulation exercises for teaching and training. In these scenarios, existing technologies have shortcomings such as the independence of physiological signal monitoring and facial video analysis, the inability to dynamically respond based on the combination of video features and motion trends, the inability to proactively warn when physiological signals are normal but facial movements are abnormal, and the lack of dynamic allocation of acquisition resources based on real-time anesthesia status. These shortcomings result in poor monitoring sensitivity, high false alarm rate, poor anesthesia safety, and low utilization of equipment resources.

[0029] like Figure 1 As shown, a first aspect of the present invention proposes an intelligent anesthesia depth monitoring system based on facial features, comprising: The video acquisition module is used to continuously acquire video stream data of the patient's facial area during anesthesia; By using non-contact optical sensors (such as conventional USB cameras or in-operating room cameras) to capture real-time images of the patient's face as the original data source for all subsequent analyses, the skin damage, electromagnetic interference, and cable entanglement caused by traditional contact sensors such as EEG electrodes are avoided. This reduces equipment costs and clinical operation complexity while ensuring patient comfort, minimizing the impact of the monitoring process on the surgical procedure itself.

[0030] The key point detection module is used to locate facial key points in each frame of the video stream data, and obtain the spatial coordinates of each key point in the image coordinate system and the displacement change between adjacent frames. In the key point detection module, facial key points such as the corners of the eyes, eyebrows, and corners of the mouth are accurately located in each frame of the video stream. The two-dimensional coordinates of each key point in the image coordinate system and the displacement changes between adjacent frames are calculated. Based on feature point regression models in computer vision, such as convolutional neural networks, this method tracks minute muscle movements by modeling facial structures. It transforms raw visual data into quantified motion trajectory information, providing high-precision spatiotemporal foundational data for subsequent feature extraction.

[0031] The feature extraction module is used to extract eyelid movement features in the eye region based on spatial location coordinates, and to count the effective movement frequency features of facial expression muscles based on the displacement change within a preset time window. The eyelid movement features include eyelid closure degree, blinking frequency, and blinking duration. The effective movement frequency feature is the cumulative number of times the displacement of facial key points exceeds a threshold within a preset time window. The sliding statistical mechanism of the preset time window enables the features to have temporal locality, which can reflect the instantaneous anesthesia state and retain historical context information through window overlap, providing data slices in the time dimension for attenuation trend analysis.

[0032] The depth assessment module is used to calculate the anesthesia depth index based on the eyelid movement features and the effective movement frequency features of facial expression muscles as multimodal input variables using a pre-trained target neural network. By utilizing a pre-trained target neural network, eyelid movement features and effective movement frequency features of facial expression muscles are used as multimodal input variables. The anesthesia depth index is a continuous value in the range of 0 to 100. Through end-to-end learning, a nonlinear mapping relationship between facial movement patterns and the concentration effect of anesthetic drugs is established, realizing deep fusion and intelligent reasoning of multi-source heterogeneous features. The pre-training mechanism enables the model to have cross-patient generalization ability, reduces the dependence on individualized calibration of new patients, and shortens the system preparation time.

[0033] The anesthesia depth determination module is used to determine the current level of anesthesia for the patient based on the anesthesia depth index.

[0034] The anesthesia depth index determines the patient's current anesthesia depth level, which includes conscious state, light anesthesia state, adequate anesthesia state, and deep anesthesia state. This converts continuous values ​​into discrete clinical states, making it easier for medical staff to quickly identify the patient's current anesthesia stage and trigger the corresponding clinical intervention process. The four-level classification is aligned with the clinical anesthesia staging standards, allowing the output results to be directly embedded into existing anesthesia records without additional conversion or interpretation.

[0035] The beneficial effects of the above technical solution are as follows: By continuously acquiring facial videos of patients, eyelid movement features and effective facial expression muscle movement frequency features are extracted. A neural network is then used to calculate the anesthesia depth index using both as multimodal inputs. Compared to methods that rely solely on a single feature, such as blinking frequency or facial movement, this invention simultaneously uses eyelid parameters reflecting brainstem arousal activity and facial movement parameters reflecting the degree of motor cortex inhibition, enabling the neural network to comprehensively assess the anesthesia state from two dimensions. When one feature changes due to individual differences or external factors, the other feature can still provide complementary information, thereby improving the stability and accuracy of the depth index. Finally, a continuous index is output and converted into anesthesia depth levels, providing clinicians with an intuitive and quantitative basis for decision-making.

[0036] In one possible implementation, such as Figure 2 As shown, the video acquisition module includes: A light sensing unit is used to detect the light intensity of the surgical environment in real time. By integrating ambient light sensors into the camera module or deploying them independently, the light intensity of the surgical environment can be detected in real time, thereby providing an objective ambient light benchmark. When the angle of the shadowless lamp shifts or the indoor light fluctuates, it can actively sense the trend of light deterioration and avoid the failure of subsequent feature extraction due to changes in light.

[0037] The frame quality assessment unit is used to assess the quality of the acquired video frames in real time and calculate the frame quality score. In one feasible approach, for example, quality sub-indicators in three dimensions are calculated for each frame of the image: image gradient energy: reflecting edge sharpness, calculated by the mean of gradient magnitude using the Sobel operator; Laplacian variance: reflecting overall image sharpness, calculated by the variance of pixel values ​​after convolution using the Laplacian operator; and noise estimation: extracting the standard deviation of high-frequency components using wavelet transform or median filtering. These three sub-indicators are linearly normalized (mapped to the 0-1 interval, with 1 being optimal), and then a weighted sum is calculated according to preset weighting coefficients (e.g., gradient energy weight 0.35, Laplacian variance weight 0.45, noise estimation weight 0.20). This sum is then multiplied by 100 to obtain a comprehensive quality score from 0 to 100. A higher score indicates that the image quality is more suitable for keypoint detection. When the score is lower than a preset quality threshold, such as 60, it indicates that the image is excessively blurred, too dark, or has strong noise interference. This method can actively filter out blurry, too dark, or too noisy invalid frames, avoiding invalid frames occupying subsequent computing resources. At the same time, it provides a precise trigger threshold for the preprocessing unit, reducing error propagation in the feature extraction stage.

[0038] The preprocessing unit is used to perform adaptive enhancement processing on the video stream when the frame quality score is less than a preset quality threshold and the illumination intensity is less than a preset illumination threshold. The enhancement intensity of the adaptive enhancement processing is negatively correlated with the frame quality score and the illumination intensity value, respectively.

[0039] By combining frame quality score and illumination intensity value, a dual-condition logic judgment is performed. Adaptive enhancement processing of the video stream is triggered only when both the frame quality score and illumination intensity are below a preset quality threshold and a preset illumination threshold. The enhancement intensity is determined by the following formula: ;in, Indicates increased strength. Indicates the factor and the mass decay coefficient; This represents the preset light intensity attenuation coefficient, where, , Calibration was performed through offline experiments. and M represents the normalized light intensity value, which is the ratio of the current light intensity to the preset standard light intensity, and the value ranges from 0 to 1. Indicates the frame quality score; In one feasible approach, for example, the enhancement process includes: brightness enhancement and contrast enhancement, wherein the brightness enhancement is based on gamma correction. , This represents the gamma correction coefficient; the greater the enhancement intensity, the smaller the gamma correction coefficient and the greater the brightness increase; the contrast enhancement is achieved based on contrast stretching, S Where S represents the stretching range; This indicates the basic range; the greater the enhancement strength, the wider the stretching range and the greater the contrast improvement. The preset quality threshold was determined through offline calibration experiments. The experiment involved collecting 1000 frames of surgical facial video samples covering different lighting conditions (50-500 lux) and different camera focal lengths (clear / out of focus / motion blur). The Dlib library was used to detect 68 facial key points, and the correlation between the detection success rate and the frame quality score was statistically analyzed. The experimental results showed that when the frame quality score was greater than a certain threshold (e.g., 0.3), the key point detection success rate reached over 95%; however, when the frame quality score was less than this threshold (0.3), the key point detection success rate was far less than 80%. Therefore, based on the above experiments, the preset weight threshold can be set to 0.3.

[0040] The preset illumination threshold was determined through the following clinical validation: In a simulated operating room environment, the illumination intensity of the surgical area was measured and adjusted using a standard lux meter. Three senior anesthesiologists independently scored the identifiability of the facial video (1-5 points, where 5 points is completely clear and 1 point is modular). For example, when the illumination intensity was equal to or greater than 50 lux, the average score of the doctors was equal to or greater than 3 points; conversely, the average score of the doctors was much less than 3 points and the failure rate of key point detection increased significantly. Therefore, the preset illumination threshold was set to 50 lux.

[0041] The beneficial effects of the above technical solution are: by detecting ambient light and video frame quality in real time, adaptive enhancement processing is triggered only under low light and low quality conditions, and the enhancement intensity automatically increases as the quality decreases or the light weakens. This significantly improves the clarity and stability of facial videos while avoiding unnecessary computational overhead, providing reliable input for subsequent key point detection and anesthesia depth assessment.

[0042] In one possible implementation, such as Figure 3 As shown, the key point detection module includes: The face detection unit is used to detect the face region in each frame of the image and output the face region location and detection confidence. By employing deep learning-based object detection algorithms such as MTCNN, RetinaFace, or YOLO-Face, forward inference is performed on the input video frames to search for and locate candidate regions containing faces in the image. The bounding box coordinates of the face region are output (column x, row y, width w, height h). For example, (100, 80, 200, 180) indicates that the top-left corner is located at column 100, row 80 of the image, with a rectangle width of 200 pixels and height of 180 pixels. A detection confidence score is also provided, which is a probability value between 0 and 1, representing the model's certainty that the region indeed contains a face. When the detection confidence score is greater than a preset detection confidence threshold, the face region is determined to be valid and sent to the keypoint localization subunit. When the detection confidence score is less than or equal to the preset detection confidence threshold, the region is discarded. This provides an accurate search range for subsequent keypoint localization, reduces the computational cost and risk of mislocalization, and provides a reliable quality judgment basis through the confidence score output.

[0043] The key point localization unit is used to locate key points of the eyes and key points of facial expression muscles within the detected facial area, and outputs the spatial coordinates and localization reliability of each key point. Within the output bounding box region, keypoint coordinates are predicted based on a cascaded regression network or heatmap regression network (e.g., Dlib's 68-point model, MediaPipe's 468-point model, or a custom lightweight network). The two-dimensional pixel coordinates of keypoints in the eyes (including upper eyelid contour points, lower eyelid contour points, inner corner of the eye, and outer corner of the eye) and keypoints of facial expression muscles (including brow bone points, nasal alar points, corner of the mouth points, and jaw contour points) are output. Simultaneously, the location confidence score for each keypoint is output, reflecting the accuracy of the keypoint coordinate prediction. This achieves refined digital labeling of facial anatomy, decomposing the facial region into a set of semantically meaningful keypoints. This allows micro-expression changes such as eyelid movement, mouth twitching, and brow bone elevation to be converted into quantifiable coordinate displacements, providing high-granularity, low-redundancy input data for the feature extraction module.

[0044] The key point quality assessment unit is used to compare the location confidence of each key point in the current frame with the average location confidence of the key point in a preset historical time window, and calculate the key point quality deviation rate. The individualized quality baseline for each keypoint is calculated by using the moving average of the location confidence of each keypoint within a preset historical time window (e.g., 30 frames). The location confidence of each keypoint in the current frame is compared with the corresponding baseline to calculate the keypoint quality deviation rate. ; If the keypoint quality deviation rate is less than the preset quality deviation threshold, the keypoint is considered to be of reliable quality in the current frame. If the keypoint quality deviation rate is equal to or greater than the preset quality deviation threshold, the keypoint is considered to be of abnormal quality. This effectively achieves dynamic and individualized assessment of keypoint quality, adapting to differences in facial features among different patients. For example, the loose skin of elderly people makes keypoint localization naturally difficult, as well as differences in the viewing angles of different cameras. At the same time, determining the average localization reliability within the historical time window can smooth out random noise and avoid misjudgments caused by single-frame anomalies.

[0045] The key point stability evaluation unit is used to calculate the positional offset between each key point in the current frame and the corresponding key point in the previous frame, and compare the positional offset with a preset jitter threshold. The position offset is obtained by calculating the Euclidean distance between the coordinates of each key point in the current frame and the corresponding key point coordinates in the previous frame. The position offset is compared with a preset jitter threshold (e.g., 5 pixels). When the position offset is greater than the preset jitter threshold, the key point is determined to be in an unstable tracking state (possibly caused by rapid head movement, occlusion, or positioning failure). When the position offset is less than or equal to the preset jitter threshold, it is determined to be in a stable tracking state. This enables real-time monitoring of the key point's motion state, distinguishing between normal facial muscle movements (small and continuous offset) and abnormal tracking jitter (abrupt offset). By comparing with the preset jitter threshold, it can effectively avoid calculating incorrect displacement changes when the head moves significantly or is briefly occluded, thus improving the accuracy of subsequent effective motion frequency statistics.

[0046] The displacement calculation unit is used to calculate the Euclidean distance between corresponding key points in adjacent frames and obtain the displacement change when the key point quality deviation rate is less than a preset quality deviation threshold and the detection confidence is greater than a preset detection confidence threshold.

[0047] By using the keypoint quality deviation rate and detection confidence level, a dual-condition logic judgment is executed. Only when the keypoint quality deviation rate is less than a preset quality deviation threshold and the detection confidence level is greater than a preset detection confidence threshold, is the Euclidean distance between corresponding keypoints in adjacent frames calculated. The Euclidean distance calculation formula is as follows: ,in, This represents the Euclidean distance between corresponding keypoints in adjacent frames; This represents the x-coordinate value of the key point in the current frame; The ordinate value of the key point in the current frame; This represents the x-coordinate value of key points in adjacent frames; The ordinate values ​​of key points in adjacent frames are represented; the displacement change is obtained, which reflects the actual movement amplitude of the key points between adjacent frames; on-demand triggering and quality control of displacement calculation are realized to avoid outputting incorrect displacement data when facial region detection fails or key point positioning is unreliable, thereby reducing noise and false triggering in subsequent effective motion frequency statistics.

[0048] The beneficial effects of the above technical solution are as follows: by performing facial region detection and key point localization on each frame of image, a quality deviation rate assessment based on historical positioning confidence and a stability judgment based on inter-frame position offset are introduced. The displacement change is calculated only when the key point quality is stable and the detection confidence is high enough. This effectively filters out false displacement noise caused by sudden changes in illumination, facial occlusion, or slight shaking, significantly improves the continuity of key point trajectory and the reliability of displacement calculation, and provides a high-quality data foundation for subsequent accurate statistics of the effective movement frequency of facial expression muscles.

[0049] In one possible implementation, the eyelid movement characteristics include at least one of eyelid closure degree, blink frequency, and blink duration.

[0050] In one possible implementation, such as Figure 4 As shown, the feature extraction module includes: The acquisition unit is used to acquire the displacement changes of the same facial key points between adjacent frames. By receiving Euclidean distance data of corresponding key points between adjacent frames output by the key point detection module, the displacement change sequence of the same facial key point at continuous time intervals is collected according to a preset time interval or frame interval (such as every frame or every 5 frames). The displacement change is a scalar distance value in the two-dimensional image coordinate system, reflecting the actual movement amplitude of the key point between adjacent sampling times. This realizes the temporal acquisition and structured storage of facial key point motion data, organizing discrete frame-by-frame displacement data into a time series that can be used for subsequent analysis, which is convenient for the statistical unit to perform windowed aggregation calculation. At the same time, the fixed interval sampling reduces data redundancy and reduces the computational overhead of the subsequent comparison unit, providing continuous and uniform data input for effective motion determination.

[0051] The comparison unit is used to obtain a preset displacement threshold and compare the displacement change of the same facial key point between adjacent frames with the preset displacement threshold to determine whether the facial expression muscles to which the same facial key point belongs have undergone effective movement; when the displacement change of the same facial key point between adjacent frames is equal to or greater than the preset displacement threshold, it is determined that the facial expression muscles to which the same facial key point belongs have undergone effective movement; otherwise, it is determined that the facial expression muscles to which the same facial key point belongs have not undergone effective movement. A preset displacement threshold (e.g., 2 pixels or 0.5% of the facial region size) is used as the effective motion judgment benchmark. The displacement change output by the acquisition unit is compared point by point with this threshold. When the displacement change is equal to or greater than the preset displacement threshold, it is determined that the facial expression muscle to which the key point belongs has undergone effective movement (i.e., a recognizable facial movement caused by muscle contraction or relaxation). Otherwise, it is determined as invalid movement (e.g., slight tremors, measurement noise, or passive displacement caused by breathing). This achieves binary discrimination of facial muscle movement, converting continuous displacement data into discrete sequences of effective / invalid movement events, filtering out physiological noise and measurement errors, and highlighting clinically significant facial movements. The preset displacement threshold is determined through offline calibration experiments. Based on the displacement distribution statistics of manually annotated effective and invalid movement frames in a large number of surgical videos, the threshold with the lowest misjudgment rate is selected as the system default value. At the same time, it supports individual fine-tuning to adapt to the differences in the degree of facial muscle development of different patients. This provides the statistical unit with high-quality, low-noise binary movement event input, ensuring the accuracy of effective movement frequency statistics and avoiding falsely high frequencies caused by slight tremors or positioning noise.

[0052] The statistics unit is used to count the number of effective movements within the preset time window to obtain the frequency characteristics of effective movements. By counting effective motion events within a preset time window, the number of effective movements occurring within the preset time window is determined, resulting in an effective motion frequency feature. This effective motion frequency feature is a scalar value that reflects the activity level of facial muscles within the time window. This achieves temporal dimension aggregation of facial muscle movements, compressing discrete binary motion events into comparable frequency indicators. This facilitates the quantitative assessment of the patient's muscle activity level at different anesthesia stages. Simultaneously, the sliding mechanism of the preset time window enables a smooth transition of frequency, avoiding frequency jumps caused by fixed window boundaries. This provides the deep assessment module with temporally localized feature data, which can reflect instantaneous changes in anesthesia status while retaining historical context information. The attenuation trend monitoring unit is used to count the effective motion frequency in multiple adjacent preset time windows, calculate the frequency change rate between adjacent windows, and obtain the motion attenuation trend characteristics. The effective motion frequency is obtained by calling the statistical unit in multiple adjacent preset time windows (e.g., window 1: 0-30 seconds, window 2: 15-45 seconds, window 3: 30-60 seconds), and the frequency change rate between adjacent windows is calculated. ,in, Indicates the rate of change of frequency between adjacent windows; Indicates the effective frequency of motion within the current window; This indicates the sequence number of a preset time window; This indicates the effective movement frequency within the next window after the current window; it also obtains the movement decay trend feature, which is a signed scalar value. A positive value indicates increased facial muscle activity (possibly during anesthesia recovery or stimulus response), while a negative value indicates decreased facial muscle activity (possibly during the deepening of anesthesia). The larger the absolute value, the faster the change. This enables the quantitative capture of the dynamic trend of facial muscle activity, allowing for early warning of rapid changes in the anesthetic state. The movement decay trend feature, along with the effective movement frequency feature and eyelid movement feature, constitutes a trimodal input, providing complementary information for the depth assessment module. The frequency feature reflects the current activity level, the eyelid feature reflects the degree of central inhibition, and the trend feature reflects the direction of dynamic change. The fusion of these three features enables the anesthesia depth assessment to possess multi-dimensional perception capabilities, including static and dynamic, instantaneous and trend-based assessments.

[0053] The depth assessment module uses the eyelid movement characteristics, the effective movement frequency characteristics, and the movement decay trend characteristics as multimodal input variables to calculate the anesthesia depth index.

[0054] The beneficial effects of the above technical solution are as follows: by comparing the inter-frame displacement change with a preset threshold, effective facial movements rather than minor tremors are accurately screened out, and the frequency of effective movements within a preset time window is counted. At the same time, the frequency change rate between adjacent windows is further calculated to obtain the motion decay trend characteristics. Combined with eyelid movement characteristics as a multimodal input, it can dynamically capture the inhibition evolution law of facial expression muscle activity during anesthesia induction or deepening, effectively distinguish between random effective movements and non-specific interference, thereby significantly improving the sensitivity and discrimination accuracy of the anesthesia depth index to changes in consciousness level.

[0055] One possible implementation also includes: The motion trend prediction unit is used to construct a linear prediction model based on the frequency change rate, predict the estimated value of the effective motion frequency at the end of the current preset time window based on the linear prediction model, and calculate the prediction deviation rate between the estimated value and the preset effective motion threshold. By using the time series of frequency change rates between adjacent windows, a linear prediction model (such as first-order linear extrapolation or least squares fitting) is constructed to predict the estimated effective motion frequency at the end of the current preset time window. Then, the prediction deviation rate between the estimated value and the preset effective motion threshold is calculated. The linear prediction model is constructed as follows: The effective motion frequency change rate corresponding to the N most recent complete time windows (e.g., N=5) is taken, and the window number is used as the independent variable. The least squares method is used to linearly fit these change rates, resulting in a straight line reflecting the trend of the change rate over time. The slope of this line represents the rate of increase or decrease, and the intercept represents the initial level. Extending this line outward by the length of a window allows us to extrapolate the predicted change rate at the end of the current window. Finally, based on the effective motion frequency already counted in the current window and the remaining time of the window, combined with the predicted change rate, the estimated value of the effective motion frequency at the end of the current window is calculated. The estimated effective exercise frequency is compared with a preset effective exercise threshold (such as 20% of the normal awake frequency or the lower limit of clinical safety), and the prediction deviation rate is calculated. ,in, Indicates the predicted deviation rate; This indicates the preset effective motion threshold; This represents an estimated value of the effective frequency of movement. Based on the above formula, it can be seen that when the estimated value of effective exercise frequency is less than the preset effective exercise threshold, the prediction deviation rate is positive, indicating that the trend is deteriorating; when the estimated value of effective exercise frequency is equal to or greater than the preset effective exercise threshold, the prediction deviation is negative, indicating that the trend is not deteriorating. The preset effective exercise threshold is determined according to the average effective exercise frequency of the patient in the conscious state according to a preset proportion (such as 20%) or the lower limit of clinical safety. The above method can effectively predict and quantify the trend of facial muscle activity in the short term, extrapolate the historical rate of change to the future state, predict the trend of deepening anesthesia in advance rather than just report the current facts, and provide early warning information in the time dimension for subsequent dual-condition triggering.

[0056] The historical baseline analysis unit is used to obtain the frequency of the patient's historical effective motion baseline during the stable phase of anesthesia induction and to calculate the attenuation deviation of the estimated value relative to the frequency of the historical effective motion baseline. Effective motor frequencies were collected and recorded during the anesthesia induction period (the transition phase from wakefulness to anesthesia) when the patient was in a stable anesthetic state (e.g., an appropriate anesthetic state maintained for more than 5 minutes). The average of multiple windows of frequencies during the stable period was calculated as the current individualized historical effective motor baseline frequency for the patient. The historical effective motor baseline frequency reflects the level of facial muscle activity in the patient under normal anesthesia. The estimated value was compared with the historical effective motor baseline frequency to calculate the attenuation deviation. The attenuation deviation is calculated as (historical effective motion baseline frequency - estimated value) / historical effective motion baseline frequency, with a value ranging from 0 to 1. The larger the value, the more severe the deviation of the estimated value from the patient's normal state. This enables the establishment of an individualized benchmark for anesthetic status assessment, eliminating absolute frequency differences caused by physiological differences such as facial muscle development, age, and gender among different patients, and improving the accuracy and adaptability of the assessment.

[0057] The motion decay warning unit is used to determine that facial expression muscle activity has entered a rapid inhibition phase and output an indication of deepening anesthesia trend only when the estimated deviation rate is equal to or greater than a preset deviation threshold and the decay deviation is equal to or greater than a preset deviation threshold. By calculating the predicted deviation rate and the attenuation deviation, a dual-condition logic judgment is performed. Only when the predicted deviation rate is equal to or greater than a preset deviation threshold (e.g., 0.3, indicating that the predicted value is lower than 70% of the threshold) and the attenuation deviation is equal to or greater than a preset deviation threshold (e.g., 0.5, indicating that the predicted value is lower than 50% of the baseline), it is determined that facial expression muscle activity has entered a rapid inhibition phase, and an anesthesia deepening trend prompt is output. The trend prompt is a Boolean signal or a graded code, used to trigger the response of the subsequent acquisition frequency adaptive unit. This achieves a reliable early warning of rapid anesthesia deepening. Through the dual verification of the trend prediction deviation from the safety threshold and the individual baseline deviation, false alarms caused by fluctuations in a single indicator (such as a sudden drop in frequency caused by the patient's brief breath-holding or equipment noise) are avoided, providing clinicians with a high-confidence basis for early intervention.

[0058] The frequency acquisition adaptive unit is used for: When the anesthesia deepening trend is received, the weighted fusion value of the estimated deviation rate and the attenuation deviation is determined, and the target sampling resource budget is determined by querying the preset resource budget mapping table according to the weighted fusion value. The target sampling resource budget increases monotonically according to the weighted fusion value. Under the target sampling resource budget constraint, the combination of target enhancement frequency and target enhancement duration is solved to maximize the target enhancement frequency, and the product of the target enhancement frequency and the target enhancement duration is equal to the target sampling resource budget. Wherein, the target enhancement frequency is less than or equal to the maximum supported frequency of video acquisition, and the target enhancement duration is less than or equal to the minimum inference duration required by the depth evaluation module; Before the current preset time window ends, the acquisition frequency is pre-increased at the target enhancement frequency, and the duration of the next preset time window is updated to the target enhancement duration.

[0059] Based on the anesthesia deepening trend indication, an adaptive resource allocation process is initiated. First, the estimated deviation rate and attenuation deviation are weighted and fused according to a preset weighting coefficient (e.g., 0.6:0.4, calibrated through offline experiments). The larger the weighted fusion value, the higher the risk. A preset resource budget mapping table is queried based on the weighted fusion value. The preset resource budget mapping table records the total sampling point budget corresponding to different risk levels. The higher the risk level, the larger the budget, realizing on-demand allocation. Under the constraint of the target sampling resource budget, the combination of target enhancement frequency and target enhancement duration is solved to maximize the target enhancement frequency and satisfy that the product of the target enhancement frequency and target enhancement duration equals the target sampling resource budget. Among them, the target enhancement frequency is less than or equal to the maximum supported frequency of the video acquisition module, and the target enhancement duration is equal to or greater than the minimum inference duration required by the depth evaluation module.

[0060] The solution is as follows: First, set the target enhancement frequency equal to the maximum supported frequency of the video acquisition module. Calculate the target enhancement duration by dividing the target sampling resource budget by the frequency. If the target enhancement duration is greater than or equal to the minimum inference duration, retain both the frequency and duration. If the target enhancement duration is less than the minimum inference duration, set the target enhancement duration to the minimum inference duration, and then calculate the target enhancement frequency again by dividing the target sampling resource budget by the target enhancement duration. In this case, the target enhancement frequency will automatically be less than the maximum supported frequency. For example, based on the weighted fusion value, the target sampling resource budget is 1800 sampling points, the maximum supported frequency of the video acquisition module is 60fps, and the minimum inference duration required by the depth evaluation module is 10 seconds. Under the budget conservation constraint, prioritize maximizing the frequency. If the frequency is 60fps, the required duration is 1800 ÷ 60 = 30 seconds. This duration is not less than the minimum inference duration (i.e., 30 seconds is greater than 10 seconds). Therefore, the target enhancement frequency is determined to be 60fps, and the target enhancement duration is 30 seconds. For example, if the budget is 500 sampling points, the corresponding duration at 60fps is approximately 8.33 seconds, which is less than 10 seconds. In this case, the frequency needs to be reduced to 500÷10=50fps, which corresponds to a duration of 10 seconds, satisfying all constraints. Therefore, the target enhancement frequency is determined to be 50fps and the target enhancement duration is 10 seconds.

[0061] Before the current preset time window ends, the acquisition frequency is immediately pre-increased at the target enhancement frequency (e.g., from 30fps to 60fps) to achieve real-time capture of rapid changes in anesthesia. At the same time, the duration of the next preset time window is updated to the target enhancement duration (e.g., from 30 seconds to 15 seconds) to make the window aggregation of subsequent statistical units more sensitive to short-term changes; thus achieving dynamic optimization of computing resources.

[0062] The beneficial effects of the above technical solution are as follows: By constructing a linear prediction model based on the frequency change rate, the effective motion frequency at the end of the current time window is estimated in advance. Combined with the historical baseline of the stable phase of the patient's anesthesia induction period, the prediction deviation rate and attenuation deviation are calculated. The early warning of the rapid inhibition phase is triggered only when both reach the threshold, thereby avoiding misjudgment caused by single fluctuations or individual differences, and achieving highly specific indication of the trend of deepening anesthesia. At the same time, after the early warning is triggered, the sampling resource budget is dynamically allocated according to the weighted fusion value, and the acquisition frequency is increased in advance within the current window. Meanwhile, the duration of the next window is adaptively shortened to ensure that resources are concentrated on capturing facial motion details in the key inhibition phase without exceeding the computing power and maximum acquisition frequency. The joint optimization of frequency and duration under the resource budget constraint significantly improves the temporal resolution of the rapid change phase of anesthesia depth, providing a denser and more targeted motion feature sequence for the subsequent depth assessment module, thereby enhancing the system's ability to track the turning point of anesthesia depth and its response speed.

[0063] In one possible implementation, the target neural network in the deep evaluation module includes: The first coding unit is used to perform temporal coding on eyelid movement features to obtain a temporal representation of the eyelid; The algorithm acquires eyelid closure degree, blink frequency, and blink duration corresponding to multiple consecutive frames within a preset time window. These three parameters are then aligned frame-by-frame and concatenated to form the eyelid feature vector for each frame. The eyelid feature vectors of each frame are arranged in chronological order of acquisition time to form an eyelid temporal feature sequence. This sequence is then input into a single-layer long short-term memory (LSTM) network. At each time step, the LSM network receives the eyelid feature vector of the current frame and updates its internal cell and hidden states. After processing the input of the last time step, the hidden state vector output at that time step is used as the temporal representation of the eyelid. By updating and compressing the temporal feature sequence composed of eyelid closure degree, blink frequency, and blink duration using the LSM network, the temporal dependence of eyelid movement can be effectively captured, providing a compact representation rich in dynamic information for subsequent multimodal fusion.

[0064] The second coding unit is used to perform temporal coding on the effective motion frequency features to obtain a temporal representation of facial motion. By acquiring the effective motion frequency feature sequence aligned frame by frame within a preset time window, the effective motion frequency feature sequence is statistically obtained by thresholding the displacement changes of facial key points corresponding to each frame image. The values ​​in the effective motion frequency feature sequence are arranged in chronological order to form a one-dimensional temporal vector. The one-dimensional temporal vector is input into a gating loop unit. At each time step, the gating loop unit receives the effective motion frequency value at the current moment and updates the state of the reset gate and the update gate. When the input of the last time step is processed, the hidden state vector output by the time step is used as the temporal representation of facial motion. By updating the temporal state of the effective motion frequency feature sequence through the gating loop unit, the dynamic change pattern of facial muscle activity can be extracted, providing a temporal representation reflecting the degree of motor cortex inhibition for anesthesia depth assessment.

[0065] The third coding unit is used to perform temporal coding on the motion decay trend features to obtain a temporal representation of the decay trend. The frequency change rate for each of multiple adjacent preset time windows is obtained. The frequency change rate is the difference in effective motion frequency between adjacent windows divided by the window duration. The frequency change rates of each window are arranged in chronological order to form a temporal sequence of frequency change rates. This temporal sequence of frequency change rates is input into a single-layer long short-term memory (LSTM) network. At each time step, the LSM network receives the frequency change rate of the current window and updates its cell state and hidden state. After the input of the last time step is processed, the hidden state vector output at that time step is used as a temporal representation of the decay trend. By using the LSM network to perform temporal modeling of the frequency change rate sequence, the acceleration or deceleration trend of facial muscle motor inhibition under the action of anesthetic drugs can be extracted, providing a compact representation of the dynamic change rate for in-depth assessment.

[0066] A timing alignment evaluation unit is used to calculate the timing alignment deviation between the eyelid timing representation and the facial motion timing representation; The temporal representations of eyelid movement and facial motion are input into two independent temporal alignment prediction branches, each consisting of a fully connected layer that outputs alignment feature vectors of the same dimension. The cosine similarity between the two alignment feature vectors is calculated, yielding a similarity value between -1 and 1. Subtracting the cosine similarity value from 1 yields the temporal alignment deviation. A smaller temporal alignment deviation indicates greater temporal synchronization between eyelid and facial motion, while a larger deviation indicates temporal misalignment. By calculating the temporal alignment deviation between the temporal representations of eyelid and facial motion, the degree of synchronization between the two on the time axis can be quantified.

[0067] The attenuation state assessment unit is used to calculate the attenuation acceleration based on the motion attenuation trend characteristics. By analyzing the motion decay trend characteristics (time series of frequency change rates), the difference between adjacent change rates is calculated and then divided by the time interval to obtain the decay acceleration. The decay acceleration is a second derivative index; a positive value indicates an accelerated decay rate (rapid deepening of anesthesia), a negative value indicates a slowed decay rate (anesthesia tends to stabilize), and a zero value indicates linear decay. By calculating the second difference of the frequency change rate, the decay acceleration can be obtained, which can quantify the changing trend of facial muscle movement inhibition speed and provide a dynamic basis for distinguishing between the rapid deepening of anesthesia stage and the stable stage.

[0068] The fusion unit is used to interactively fuse the eyelid temporal representation, the facial motion temporal representation, and the decay trend temporal representation to obtain a fused representation; The fusion unit includes a preset fusion weight mapping table. The preset fusion weight mapping table uses the temporal alignment deviation and the decay acceleration as indexes to record the fusion weight ratio of the eyelid temporal representation, the facial motion temporal representation, and the decay trend temporal representation. When the temporal alignment deviation is less than a preset alignment threshold and the absolute value of the decay acceleration is less than a preset smoothing threshold, the preset fusion weight mapping table is queried to obtain a first weight ratio. In the first weight ratio, the sum of the weights of the eyelid temporal representation and the facial motion temporal representation is greater than the weight of the decay trend temporal representation. Otherwise, the second weight ratio is obtained by querying the preset fusion weight mapping table, in which the weight of the decay trend time sequence representation is greater than the sum of the weights of the eyelid time sequence representation and the facial movement time sequence representation; By receiving the eyelid timing representation, facial motion timing representation, and decay trend timing representation output by the first, second, and third encoding units respectively, as well as the timing alignment deviation output by the timing alignment evaluation unit and the decay acceleration output by the decay state evaluation unit, a three-modal interactive fusion is performed using a preset fusion weight mapping table. This preset fusion weight mapping table uses timing alignment deviation and decay acceleration as a joint index, pre-stores the fusion weight ratios corresponding to different index combinations, and quickly determines the weight allocation of the three modalities through a lookup table. When the timing alignment deviation is less than a preset alignment threshold (e.g., 0.2, indicating good time synchronization between the two modalities) and the absolute value of the decay acceleration is less than a preset smoothing threshold (e.g., 0.05, indicating stable decay speed), a first weight ratio is obtained. In this first weight ratio, the sum of the weights of the eyelid timing representation and the facial motion timing representation is greater than the weight of the decay trend timing representation (e.g., the sum of the weights of the eyelid timing representation and the facial motion timing representation is greater than the weight of the facial motion timing representation). The current static level indicator is prioritized (40% for eyelids, 40% for face, and 20% for attenuation). Otherwise, a second weighting ratio is obtained, in which the weight of the attenuation trend temporal representation is greater than the sum of the weights of the eyelid temporal representation and the facial movement temporal representation (e.g., 50% attenuation, 25% for eyelids, and 25% for face). The dynamic trend indicator is prioritized. The fused representation is a comprehensive latent space vector after weighted summation of the three modalities, which encodes comprehensive information on the current anesthesia depth, facial activity level, and change trend. Dynamic adaptive fusion of three modal features is achieved. The fusion strategy is automatically switched according to data quality (temporal alignment deviation) and state dynamics (attenuation acceleration), avoiding the failure of fixed weights during data misalignment or rapid change periods. At the same time, the computational complexity of the lookup table mechanism is much lower than that of real-time optimization, making it suitable for real-time inference scenarios of edge devices. Furthermore, the preset weights are determined through offline training optimization to ensure the clinical rationality of the fusion strategy.

[0069] The regression output subunit is used to input the fused representation into the fully connected regression network, which maps it to the regression feature space through two fully connected hidden layers, and outputs a continuous value as the anesthesia depth index through a single neuron linear output layer. The anesthesia depth index ranges from (0 to 100), and the larger the value, the deeper the anesthesia. The fusion representation is input into a fully connected regression network, which includes two fully connected hidden layers (128 neurons per layer, with ReLU activation function) to map the fusion representation from the hidden space to the regression feature space. A single-neuron linear output layer (without activation function) then outputs a continuous value as the anesthesia depth index. The anesthesia depth index ranges from 0 to 100, with higher values ​​indicating deeper anesthesia and higher levels of consciousness inhibition. Specifically, 0-40 corresponds to a conscious state, 40-60 to a light anesthesia state, 60-80 to a suitable anesthesia state, and 80-100 to a deep anesthesia state. This achieves a continuous mapping from multimodal fusion features to a clinically interpretable anesthesia depth index, facilitating direct understanding and use by medical staff without additional conversion.

[0070] The network parameters of the first coding unit, the second coding unit, the third coding unit, the temporal alignment evaluation unit, the decay state evaluation unit, the fusion unit, and the regression output unit are synchronously updated through end-to-end training.

[0071] The beneficial effects of the above technical solution are as follows: by temporally encoding eyelid movements, effective movement frequency and their decay trend, and introducing temporal alignment deviation and decay acceleration as dynamic fusion basis, the neural network can adaptively adjust feature weights according to the consistency of facial movements and inhibition speed. When the movements are smooth and well aligned, it focuses on eyelid and facial movement features, and highlights decay trend features when the movements are rapidly inhibited or the temporal sequence is disordered. The condition-dependent multimodal fusion strategy combined with end-to-end training significantly improves the robustness and sensitivity of the anesthesia depth index to changes in consciousness level, and enhances the network's generalization ability to complex anesthetic states.

[0072] In one possible implementation, the joint loss function for the end-to-end training includes: The prediction error loss term is used to calculate the mean square error between the predicted value of the anesthesia depth index and the clinically labeled value. The predicted value of the anesthesia depth index is compared with the clinically labeled value synchronously collected by standard equipment (such as a BIS monitor). The mean squared difference between the two values ​​is calculated as the prediction error loss term. The mean squared error formula is as follows: ;in, The mean squared error is represented by N, which is the number of samples in the batch. The prediction error loss term is a positive scalar value, and the smaller the value, the closer the model prediction is to the clinical reality. By minimizing the mean squared error between the predicted value and the clinically labeled value, the neural network output can approximate the true value of the anesthesia depth labeled by the experts, thereby improving the clinical credibility of the model evaluation results.

[0073] The mapping table smoothing loss term is used to calculate the rate of change between the fusion weight ratios corresponding to adjacent index positions in the preset fusion weight mapping table. When the rate of change exceeds a preset smoothing threshold, a penalty is applied. The system retrieves all entries in a pre-defined fusion weight mapping table, arranged in index order. Each entry contains a set of fusion weight ratios, composed of the weight values ​​for eyelid temporal representation, facial movement temporal representation, and decay trend temporal representation. It iterates through adjacent index positions in the pre-defined fusion weight mapping table, extracting the differences between corresponding weight components in each set of fusion weight ratios. The difference between corresponding weight components in each set of fusion weight ratios is divided by the index interval to obtain the rate of change of each weight component between adjacent index positions. The sum of the absolute values ​​of all rates of change is calculated to obtain the total rate of change. A pre-defined smoothing threshold is obtained, and the total rate of change is compared with the pre-defined smoothing threshold. If the total rate of change is greater than the pre-defined smoothing threshold, the difference between the total rate of change and the pre-defined smoothing threshold is calculated, and the square of the difference is used as the value of the smoothing loss term in the mapping table. If the total rate of change is less than or equal to the pre-defined smoothing threshold, the value of the smoothing loss term in the mapping table is zero. By applying a smoothing constraint to the rate of change of weight ratios at adjacent index positions in the mapping table, drastic jumps in fusion weights between different risk levels are avoided, improving the temporal stability and clinical interpretability of weight allocation.

[0074] The joint loss function is a weighted sum of the prediction error loss term and the mapping table smoothing loss term.

[0075] The beneficial effects of the above technical solution are: by jointly using prediction error and mapping table smoothing loss for end-to-end training, it can ensure a high degree of consistency between the anesthesia depth index and the clinical labeled value, and constrain the adjacent ratio of the weight mapping table to change gradually, thereby avoiding drastic changes in the feature fusion strategy and improving the generalization stability and clinical interpretability of the model under different anesthesia states.

[0076] In one possible implementation, such as Figure 5 As shown, the anesthesia depth determination module includes: A grading unit is used to map the anesthesia depth index to discrete anesthesia depth levels, wherein the discrete anesthesia depth levels include: conscious state, light anesthesia state, adequate anesthesia state, and deep anesthesia state. The anesthesia depth index is obtained, with a value ranging from 0 to 100; a higher value indicates a deeper anesthesia. A first, second, and third threshold are also obtained, where the first threshold is less than the second, and the second threshold is less than the third. For example, the first threshold might be 40, the second 60, and the third 80. The anesthesia depth index is compared to the first threshold. If the index is less than the first threshold (e.g., 35), the patient is considered conscious. If the index is equal to or greater than the first threshold, it is compared to the second threshold. If the index is less than the second threshold (e.g., 50), the patient is considered lightly anesthetized. If the index is equal to or greater than the second threshold, it is compared to the third threshold. If the index is less than the third threshold (e.g., 70), the patient is considered adequately anesthetized. If the index is equal to or greater than the third threshold (e.g., 85), the patient is considered deeply anesthetized. The continuous anesthesia depth index is mapped to four discrete levels: awake, light anesthesia, appropriate anesthesia, and deep anesthesia. This achieves an intuitive correspondence between clinical staging and quantitative indices, making it easier for medical staff to quickly identify the current anesthesia status and implement graded interventions.

[0077] The alarm output unit is used to output graded alarm signals according to discrete anesthesia depth levels, specifically: When the anesthesia depth level is at the level of awake state, a level one prompt signal is output; When the anesthesia depth level is light anesthesia, a level 2 attention signal is output. When the anesthesia depth level is deep anesthesia, a level three danger signal is output. The appropriate anesthesia state does not trigger an alarm signal.

[0078] By using discrete anesthesia depth levels, a graded alarm system is implemented. When the level is "awake," a Level 1 alert signal is output. This Level 1 alert signal, for example, can be a low-frequency flashing green indicator light, indicating that the anesthesia is too shallow and the patient may be aware of the surgical procedure. When the level is "light anesthesia," a Level 2 attention signal is output. This Level 2 attention signal can be a medium-frequency flashing yellow indicator light accompanied by a low-frequency alert sound, indicating that the anesthesia depth is approaching the limit and medication dosage needs to be adjusted. Medication dosage can be adjusted according to the actual situation. When the level is "deep anesthesia," a Level 3 danger signal is output. This Level 3 danger signal, for example, can be a high-frequency flashing red indicator light accompanied by a high-frequency alarm sound, indicating that the anesthesia is too deep and medication needs to be reduced immediately or an antagonist should be administered. When the level is "appropriate anesthesia," no alarm signal is triggered. This system achieves differentiated audible and visual alarms for anesthesia status, realizes graded risk warning, and avoids interference from invalid alarms.

[0079] The beneficial effects of the above technical solution are as follows: by mapping the continuous anesthesia depth index into four discrete levels—awake, light anesthesia, appropriate anesthesia, and deep anesthesia—and outputting graded alarm signals according to different levels, the appropriate anesthesia state does not trigger alarms, thus avoiding unnecessary interference. The graded alarm mechanism enables medical staff to intuitively and quickly identify the patient's state of consciousness and risk level, pay attention during light anesthesia, and intervene urgently during deep anesthesia, thereby significantly improving the safety of anesthesia management and the efficiency of clinical response.

[0080] In one possible implementation, the system further includes an edge deployment module for quantizing the target neural network and deploying it to an anesthesia monitoring device for real-time inference.

[0081] The target neural network, which has undergone end-to-end training, is obtained. This target neural network includes a first encoding unit, a second encoding unit, a third encoding unit, a temporal alignment evaluation unit, a decay state evaluation unit, a fusion unit, and a regression output unit. The floating-point weight parameters and bias parameters of each network layer in the target neural network are converted into low-bit integer representations to obtain a quantized neural network. The quantized neural network is then loaded into the edge computing unit of the anesthesia monitoring device. The edge computing unit includes an embedded processor or a neural network acceleration chip. When the video acquisition module acquires real-time video stream data of the patient's face, the key point detection module, feature extraction module, and depth evaluation module sequentially call the quantized neural network deployed in the edge computing unit for forward inference to generate an anesthesia depth index. By quantizing the neural network parameters and deploying them to the edge computing unit of the anesthesia monitoring device, model storage and computational overhead are reduced, enabling local real-time inference without network transmission, thus improving system response speed and clinical applicability.

[0082] The beneficial effects of the above technical solution are: by quantizing the neural network and deploying it to the anesthesia monitoring equipment, it enables real-time inference with low latency and high privacy protection, avoids network latency and data security risks caused by uploading video data to the cloud, and significantly improves clinical applicability and response efficiency.

[0083] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. An intelligent anesthesia depth monitoring system based on facial features, characterized in that, include: The video acquisition module is used to continuously acquire video stream data of the patient's facial area during anesthesia; The key point detection module is used to locate facial key points in each frame of the video stream data, and obtain the spatial coordinates of each key point in the image coordinate system and the displacement change between adjacent frames. The feature extraction module is used to extract eyelid movement features in the eye region based on spatial location coordinates, and to count the effective movement frequency features of facial expression muscles based on the displacement change within a preset time window. The depth assessment module is used to calculate the anesthesia depth index based on the eyelid movement features and the effective movement frequency features of facial expression muscles as multimodal input variables using a pre-trained target neural network. The anesthesia depth determination module is used to determine the current level of anesthesia for the patient based on the anesthesia depth index.

2. The intelligent anesthesia depth monitoring system based on facial features according to claim 1, characterized in that, The video acquisition module includes: A light sensing unit is used to detect the light intensity of the surgical environment in real time. The frame quality assessment unit is used to assess the quality of the acquired video frames in real time and calculate the frame quality score. The preprocessing unit is used to perform adaptive enhancement processing on the video stream when the frame quality score is less than a preset quality threshold and the illumination intensity is less than a preset illumination threshold. The enhancement intensity of the adaptive enhancement processing is negatively correlated with the frame quality score and the illumination intensity value, respectively.

3. The intelligent anesthesia depth monitoring system based on facial features according to claim 1, characterized in that, The key point detection module includes: The face detection unit is used to detect the face region in each frame of the image and output the face region location and detection confidence. The key point localization unit is used to locate key points of the eyes and key points of facial expression muscles within the detected facial area, and outputs the spatial coordinates and localization reliability of each key point. The key point quality assessment unit is used to compare the location confidence of each key point in the current frame with the average location confidence of the key point in a preset historical time window, and calculate the key point quality deviation rate. The key point stability evaluation unit is used to calculate the positional offset between each key point in the current frame and the corresponding key point in the previous frame, and compare the positional offset with a preset jitter threshold. The displacement calculation unit is used to calculate the Euclidean distance between corresponding key points in adjacent frames and obtain the displacement change when the key point quality deviation rate is less than a preset quality deviation threshold and the detection confidence is greater than a preset detection confidence threshold.

4. The intelligent anesthesia depth monitoring system based on facial features according to claim 1, characterized in that, The eyelid movement characteristics include at least one of the following: eyelid closure degree, blinking frequency, and blinking duration.

5. The intelligent anesthesia depth monitoring system based on facial features according to claim 1, characterized in that, The feature extraction module includes: The acquisition unit is used to acquire the displacement changes of the same facial key points between adjacent frames. The comparison unit is used to obtain a preset displacement threshold and compare the displacement change of the same facial key point between adjacent frames with the preset displacement threshold to determine whether the facial expression muscles to which the same facial key point belongs have undergone effective movement; when the displacement change of the same facial key point between adjacent frames is equal to or greater than the preset displacement threshold, it is determined that the facial expression muscles to which the same facial key point belongs have undergone effective movement; otherwise, it is determined that the facial expression muscles to which the same facial key point belongs have not undergone effective movement. The statistics unit is used to count the number of effective movements within the preset time window to obtain the frequency characteristics of effective movements. The attenuation trend monitoring unit is used to count the effective motion frequency in multiple adjacent preset time windows, calculate the frequency change rate between adjacent windows, and obtain the motion attenuation trend characteristics. The depth assessment module uses the eyelid movement characteristics, the effective movement frequency characteristics, and the movement decay trend characteristics as multimodal input variables to calculate the anesthesia depth index.

6. The intelligent anesthesia depth monitoring system based on facial features according to claim 5, characterized in that, Also includes: The motion trend prediction unit is used to construct a linear prediction model based on the frequency change rate, predict the estimated value of the effective motion frequency at the end of the current preset time window based on the linear prediction model, and calculate the prediction deviation rate between the estimated value and the preset effective motion threshold. The historical baseline analysis unit is used to obtain the frequency of the patient's historical effective motion baseline during the stable phase of anesthesia induction and to calculate the attenuation deviation of the estimated value relative to the frequency of the historical effective motion baseline. The motion decay warning unit is used to determine that facial expression muscle activity has entered a rapid inhibition phase and output an indication of deepening anesthesia trend only when the estimated deviation rate is equal to or greater than a preset deviation threshold and the decay deviation is equal to or greater than a preset deviation threshold. The frequency acquisition adaptive unit is used for: When the anesthesia deepening trend is received, the weighted fusion value of the estimated deviation rate and the attenuation deviation is determined, and the target sampling resource budget is determined by querying the preset resource budget mapping table according to the weighted fusion value. The target sampling resource budget increases monotonically according to the weighted fusion value. Under the target sampling resource budget constraint, the combination of target enhancement frequency and target enhancement duration is solved to maximize the target enhancement frequency, and the product of the target enhancement frequency and the target enhancement duration is equal to the target sampling resource budget. Wherein, the target enhancement frequency is less than or equal to the maximum supported frequency of video acquisition, and the target enhancement duration is less than or equal to the minimum inference duration required by the depth evaluation module; Before the current preset time window ends, the acquisition frequency is pre-increased at the target enhancement frequency, and the duration of the next preset time window is updated to the target enhancement duration.

7. The intelligent anesthesia depth monitoring system based on facial features according to claim 5, characterized in that, The target neural network in the deep evaluation module includes: The first coding unit is used to perform temporal coding on eyelid movement features to obtain a temporal representation of the eyelid; The second coding unit is used to perform temporal coding on the effective motion frequency features to obtain a temporal representation of facial motion. The third coding unit is used to perform temporal coding on the motion decay trend features to obtain a temporal representation of the decay trend. A timing alignment evaluation unit is used to calculate the timing alignment deviation between the eyelid timing representation and the facial motion timing representation; The attenuation state assessment unit is used to calculate the attenuation acceleration based on the motion attenuation trend characteristics. The fusion unit is used to interactively fuse the eyelid temporal representation, the facial motion temporal representation, and the decay trend temporal representation to obtain a fused representation; The fusion unit includes a preset fusion weight mapping table. The preset fusion weight mapping table uses the temporal alignment deviation and the decay acceleration as indexes to record the fusion weight ratio of the eyelid temporal representation, the facial motion temporal representation, and the decay trend temporal representation. When the temporal alignment deviation is less than a preset alignment threshold and the absolute value of the decay acceleration is less than a preset smoothing threshold, the preset fusion weight mapping table is queried to obtain a first weight ratio. In the first weight ratio, the sum of the weights of the eyelid temporal representation and the facial motion temporal representation is greater than the weight of the decay trend temporal representation. Otherwise, the second weight ratio is obtained by querying the preset fusion weight mapping table, in which the weight of the decay trend time sequence representation is greater than the sum of the weights of the eyelid time sequence representation and the facial movement time sequence representation; The regression output subunit is used to input the fused representation into the fully connected regression network, which maps it to the regression feature space through two fully connected hidden layers, and outputs a continuous value as the anesthesia depth index through a single neuron linear output layer. The anesthesia depth index ranges from (0 to 100), and the larger the value, the deeper the anesthesia. The network parameters of the first coding unit, the second coding unit, the third coding unit, the temporal alignment evaluation unit, the decay state evaluation unit, the fusion unit, and the regression output unit are synchronously updated through end-to-end training.

8. The intelligent anesthesia depth monitoring system based on facial features according to claim 7, characterized in that, The joint loss function for end-to-end training includes: The prediction error loss term is used to calculate the mean square error between the predicted value of the anesthesia depth index and the clinically labeled value. The mapping table smoothing loss term is used to calculate the rate of change between the fusion weight ratios corresponding to adjacent index positions in the preset fusion weight mapping table. When the rate of change exceeds a preset smoothing threshold, a penalty is applied. The joint loss function is a weighted sum of the prediction error loss term and the mapping table smoothing loss term.

9. The intelligent anesthesia depth monitoring system based on facial features according to claim 1, characterized in that, The anesthesia depth determination module includes: A grading unit is used to map the anesthesia depth index to discrete anesthesia depth levels, wherein the discrete anesthesia depth levels include: conscious state, light anesthesia state, adequate anesthesia state, and deep anesthesia state. The alarm output unit is used to output graded alarm signals according to discrete anesthesia depth levels, specifically: When the anesthesia depth level is at the level of awake state, a level one prompt signal is output; When the anesthesia depth level is light anesthesia, a level 2 attention signal is output. When the anesthesia depth level is deep anesthesia, a level three danger signal is output. The appropriate anesthesia state does not trigger an alarm signal.

10. A facial feature-based intelligent anesthesia depth monitoring system according to any one of claims 1 to 9, characterized in that, The system also includes an edge deployment module, which is used to quantize the target neural network and deploy it to the anesthesia monitoring equipment for real-time inference.

Images