Monitoring method and system applied to post-anesthesia patient recovery
By combining facial images and physiological data with a deep convolutional neural network scoring method, the inadequacy of monitoring during the post-anesthesia recovery period is addressed, providing automated and comprehensive status assessment and improving the comprehensiveness and safety of monitoring.
Patent Information
- Application Number
- CN202511960641.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-24
- Publication Date
- 2026-01-23
AI Technical Summary
Existing technologies for monitoring patients during the recovery period after anesthesia have limitations, including the inability of medical staff to conduct continuous, round-the-clock observation, a lack of comprehensive intelligent assessment, and the failure to quantify facial expressions, leading to potential safety hazards.
By acquiring patients' facial images and physiological time-series data, deep convolutional neural networks are used to extract facial expression features and physiological features, which are then combined with a pre-set model to perform a comprehensive score, providing automated status assessment and prompts.
It enables a comprehensive and objective assessment of the patient's condition after anesthesia, reduces subjective errors, and improves nursing efficiency and safety.
Smart Images
Figure CN121370080A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of medical monitoring, in particular to a monitoring method and system applied to post-anesthesia patient recovery. BACKGROUND
[0002] The post-anesthesia recovery period is a critical stage for patients to transition from an anesthetized state to a conscious state. During this period, the patient's vital signs and conscious state are in dynamic change. If not properly monitored, various complications such as respiratory depression, circulatory instability, and delayed awakening may occur, which seriously threatens patient safety. Currently, clinical monitoring of post-anesthesia patients mainly relies on regular observation by medical staff and alarm of multi-parameter monitors. However, this approach has obvious limitations: first, medical staff cannot achieve around-the-clock, uninterrupted bedside observation, which may miss critical information; second, existing monitors mainly set thresholds for single physiological parameters to alarm, lacking comprehensive and intelligent assessment of the patient's overall state; third, the patient's facial expression, an important behavioral indicator reflecting their pain, comfort, and conscious state, has not been effectively quantified and integrated into the assessment system in existing technology. Therefore, there is an urgent need for a method that can automatically, continuously, and comprehensively assess the recovery status of post-anesthesia patients to address the shortcomings of manual monitoring and improve medical safety and quality. SUMMARY
[0003] The purpose of the present application is to provide a monitoring method and system applied to post-anesthesia patient recovery to improve the above problems.
[0004] To achieve the above purpose, the embodiments of the present application provide the following technical solutions: On the one hand, the embodiments of the present application provide a monitoring method applied to post-anesthesia patient recovery, which comprises: Obtaining multiple facial images and physiological time series data of the patient within a preset period, the physiological time series data including heart rate time series data, respiratory rate time series data, blood pressure time series data, and blood oxygen saturation time series data; Calculating the patient's expression score within the preset period according to the facial images; pre-processing the physiological time series data, and extracting features after pre-processing to obtain a physiological feature vector; Inputting the physiological feature vector into a preset physiological scoring model to obtain a physiological score, fusing the physiological score and the expression score to obtain a comprehensive score, determining the patient's current state information according to the comprehensive score, and sending different prompt information to medical staff according to the state information.
[0005] Secondly, the embodiments of the present application provide a monitoring system applied to post-anesthesia patient recovery, which comprises: The acquisition module is configured to acquire a plurality of facial images and physiological time-series data of the patient within a preset time period, wherein the physiological time-series data comprises heart rate time-series data, respiration rate time-series data, blood pressure time-series data and blood oxygen saturation time-series data; The extraction module is configured to calculate an expression score of the patient within the preset time period according to the facial images, and to pre-process the physiological time-series data and then extract features to obtain a physiological feature vector; The scoring module is configured to input the physiological feature vector into a preset physiological scoring model to obtain a physiological score, to fuse the physiological score and the expression score to obtain a comprehensive score, to determine the current state information of the patient according to the comprehensive score, and to send different prompt information to medical staff according to the state information.
[0006] In a third aspect, an embodiment of the present application provides a monitoring device applied to patient recovery after anesthesia, the device comprising a memory and a processor. The memory is configured to store a computer program; and the processor is configured to execute the computer program to implement the steps of the monitoring method applied to patient recovery after anesthesia.
[0007] In a fourth aspect, an embodiment of the present application provides a readable storage medium, and the readable storage medium stores a computer program. The computer program is executed by a processor to implement the steps of the monitoring method applied to patient recovery after anesthesia.
[0008] The present application has the following beneficial effects: 1. The present application synchronously collects and analyzes the physiological signs (heart rate, blood pressure, blood oxygen, etc.) and facial expression information of the patient. The expression features are automatically extracted and quantified as a score by a deep convolutional neural network, and at the same time, a multi-dimensional feature vector is extracted from the physiological time-series data. Finally, the two are fused into a comprehensive score by a specific model. This multi-modal fusion method overcomes the one-sidedness of single dependence on physiological parameters or subjective observation, provides more comprehensive and objective data support for patient state evaluation, and greatly reduces subjective judgment errors.
[0009] 2. The present application can automatically complete the whole process from data acquisition, processing, analysis to state classification and alarm, and free medical staff from the heavy task of continuous observation. It enables them to invest their limited energy in the patients who need the most care, thereby reasonably allocating medical resources and improving the overall nursing efficiency and ward safety management level.
[0010] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the application as described in the written description and claims. The objects and other advantages of the present application will be realized and attained by the structure particularly pointed out in the written description and claims. BRIEF DESCRIPTION OF DRAWINGS
[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0012] Figure 1 This is a schematic diagram of the monitoring method for post-anesthesia patient recovery as described in an embodiment of the present invention; Figure 2 This is a schematic diagram of the monitoring system for post-anesthesia patient recovery as described in an embodiment of the present invention; Figure 3 This is a schematic diagram of the monitoring device for post-anesthesia patient recovery as described in an embodiment of the present invention. Detailed Implementation
[0013] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.
[0014] It should be noted that similar reference numerals or letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0015] Example 1 like Figure 1 As shown, this embodiment provides a monitoring method for the recovery of patients after anesthesia, which includes steps S1, S2 and S3.
[0016] Step S1: Acquire multiple facial images and physiological time-series data of the patient within a preset time period. The physiological time-series data includes heart rate time-series data, respiratory rate time-series data, blood pressure time-series data, and blood oxygen saturation time-series data. In this step, the data collected is in the recovery period after the patient is anesthetized, and the preset time period is also in the recovery period, which can be customized by the user, for example, the length of the time period can be 30 seconds, 1 minute, etc. The recovery period of the present application can be from the time when the patient enters the post-anesthesia care unit (PACU) to the time when the patient meets the discharge standard. The discharge standard can be objectively determined by using the internationally accepted modified Aldrete score. This standard quantitatively evaluates the patient's activity, respiration, circulation, consciousness, and oxygen saturation, with a total score of 10. When the score is ≥ 9, it indicates that the patient has fully recovered from the anesthetic state and meets the safety requirements for transferring out of the recovery room.
[0017] At the same time, in this step, the collection of the preset time period face image can be: using a high-definition infrared camera installed directly above the patient's bed to capture the patient's face image, and continuously collecting at a rate of 5 frames per second; The patient's heart rate, blood pressure, blood oxygen saturation and respiratory rate time series data can be detected by corresponding medical examination instruments; Step S2, calculating the patient's expression score in the preset time period according to the face image; pre-processing the physiological time series data, and extracting features after pre-processing to obtain a physiological feature vector; In this step, the patient's expression score in the preset time period is calculated according to the face image; the physiological time series data is pre-processed, and the physiological feature vector is obtained after feature extraction. The specific implementation steps include steps S21 and S22. Step S21, performing image enhancement processing on the collected multiple face images to obtain multiple enhanced face images; calculating the expression probability distribution vector corresponding to each enhanced face image, and calculating the patient's expression score in the preset time period according to all expression probability distribution vectors; In this step, the expression probability distribution vector corresponding to each enhanced face image is calculated, and the patient's expression score in the preset time period is calculated according to all expression probability distribution vectors. The specific implementation steps include steps S211 and S212. Step S211, using a face detection algorithm to locate and frame the face region from the enhanced face image; detecting the face key points of the face region, and based on the detected face key points, the face is corrected to a standard pose through geometric transformation; then the face region is cropped according to the face boundary frame and scaled to a preset size; the image is converted to a gray image, and the pixel value is normalized to obtain a face image two-dimensional matrix; This step can be understood as: Face detection and localization: MTCNN (Multi-task Convolutional Neural Network) algorithm is used as the face detection algorithm. The original input image is input into MTCNN, and the network outputs one or more face region bounding boxes, which are defined by their top-left corner coordinates (x, y) and width (w) and height (h) in the image.
[0018] Key point detection and pose correction: At the same time as MTCNN detection, the preset 2 key point coordinates, left eye center and right eye center, are output. The rotation angle of the line connecting the left and right eye centers is calculated. The image is rotated by affine transformation to make the line parallel to the horizontal axis, thereby realizing standard pose correction of the face.
[0019] Region cropping and size normalization: To preserve complete facial information, the detected face bounding box is expanded by 20% in both length and width directions. According to the expanded bounding box, the face region is cropped from the corrected image. The bilinear interpolation algorithm is used to scale the cropped region to the preset size of 112 pixels x 112 pixels.
[0020] Color space and pixel value normalization: The RGB color image after size normalization is converted to a single-channel grayscale image by the weighted formula Gray = 0.299 * R + 0.587 * G + 0.114 * B. The pixel values of the grayscale image are normalized: first, the pixel values are divided by 255 from the integer range of [0, 255] to the floating-point range of [0.0, 1.0]. Then, standardization is performed, i.e., using the mean (μ = 0.5) and standard deviation (σ = 0.5) of a large-scale face dataset (such as ImageNet), the pixel values are processed according to the formula (pixel value - 0.5) / 0.5, and finally the pixel values are transformed to the interval [-1.0, 1.0].
[0021] Step S212, input the face image two-dimensional matrix into a deep convolutional neural network, and perform feature extraction and transformation through multiple cascaded convolutional layers of the deep convolutional neural network, wherein at least one convolutional layer is connected with one or more of a normalization layer, a nonlinear activation layer or a pooling layer; flatten the feature map output by the last convolutional layer into a one-dimensional feature vector; input the one-dimensional feature vector into a fully connected part, and combine and map the features through one or more fully connected layers; finally, output an expression probability distribution vector from an output layer of the deep convolutional neural network, wherein the output layer uses a Softmax activation function, the number of neurons of the output layer is equal to the number of preset expression categories, an expression probability distribution vector is output, and the sum of the probability values of all dimensions in the expression probability distribution vector is 1; assign a weight value to each expression category to form a weight vector, and for each facial image, calculate the dot product of the expression probability distribution vector and the weight vector to obtain a single expression score of the facial image; calculate the average of the single expression scores corresponding to all facial images in a preset period to obtain an expression score of the patient in the preset period.
[0022] This step can be understood as: (1) Network input: input the two-dimensional matrix of the face image obtained after preprocessing as the input of the deep convolutional neural network.
[0023] (2) Convolutional feature extraction and transformation: the two-dimensional matrix is processed through multiple cascaded convolutional modules in the network. Taking a specific implementation containing two convolutional modules as an example: First convolutional module: Convolution operation: use 32 convolution kernels with a size of 3x3 to convolve the input two-dimensional matrix with a step size of 1 to generate 32 feature maps with a size of 110x110.
[0024] Normalization processing: perform batch normalization processing on the above-mentioned 32 feature maps, and use the mean and variance of the batch data to standardize the feature values.
[0025] Nonlinear activation: apply the ReLU activation function to each normalized feature value.
[0026] Pooling processing: use a maximum pooling method with a window size of 2x2 and a step size of 2 to downsample the activated feature map to obtain 32 feature maps with a size of 55x55, which are the outputs of the first convolutional module.
[0027] Second convolutional module: Convolution operation: use 64 convolution kernels with a size of 3x3 to convolve the feature map output by the first convolutional module to generate 64 feature maps with a size of 53x53.
[0028] Subsequent processing: batch normalization, ReLU activation and pooling are sequentially performed (same as the first convolutional module), and finally 64 feature maps of 26x26 are output.
[0029] (3) Feature map flattening: the 64 feature maps of 26x26 output by the second convolutional module are flattened in the spatial dimension and spliced into a one-dimensional vector. The length of the vector is 64x26x26=43264.
[0030] (4) Fully connected feature mapping: the one-dimensional feature vector after flattening is input to the fully connected part: First fully connected layer: this layer contains 1024 neurons. Through matrix multiplication and bias addition, the input 43264-dimensional vector is transformed into a 1024-dimensional vector, and the ReLU activation function is applied again for nonlinear transformation.
[0031] Second fully connected layer (output layer): the number of neurons in this layer is equal to the number of preset expression categories M (for example, M=6, corresponding to 6 expressions such as natural, calm, and painful). The 1024-dimensional vector output by the first fully connected layer is input into this layer. The second fully connected layer first performs linear transformation to output an M-dimensional raw score vector, and then applies the Softmax activation function to calculate the final M-dimensional expression probability distribution vector.
[0032] In step S22, different filters are used to filter each physiological time series data, and then linear interpolation method is used to process the filtered physiological time series data to obtain preprocessed physiological time series data. The physiological feature information of each preprocessed physiological time series data is extracted, and all physiological feature information is spliced and combined to obtain a physiological feature vector.
[0033] In this step, different filters are used to filter each physiological time series data, and then linear interpolation method is used to process the filtered physiological time series data to obtain preprocessed physiological time series data. The physiological feature information of each preprocessed physiological time series data is extracted, and all physiological feature information is spliced and combined to obtain a physiological feature vector. In step S221, Butterworth band-pass filter is used to filter heart rate time series data and respiratory rate time series data, low-pass filter is used to filter blood pressure time series data, and sliding average filter is used to filter blood oxygen saturation time series data, respectively. The filtered heart rate time series data, the filtered respiratory rate time series data, the filtered blood pressure time series data and the filtered blood oxygen saturation time series data are obtained. Step S222, the linear interpolation method is used to resample the filtered heart rate time series data, the filtered respiratory rate time series data, the filtered blood pressure time series data and the filtered blood oxygen saturation time series data respectively, and the sampling frequencies are unified, and the preprocessed heart rate time series data, the preprocessed respiratory rate time series data, the preprocessed blood pressure time series data and the preprocessed blood oxygen saturation time series data are obtained respectively.
[0034] In this step, unifying the sampling frequencies can be understood as unifying their sampling frequencies to a fixed value (for example, 1 Hz); In step S22, physiological feature information is extracted from each kind of preprocessed physiological time series data, and after extraction, all physiological feature information is spliced and combined to obtain a physiological feature vector. The specific implementation steps include step S223. Step S223, for the preprocessed heart rate time series data, the average value and the standard deviation of all heart rates are calculated; for the preprocessed respiratory rate time series data, the average value and the standard deviation of all respiratory rates are calculated, and linear regression is performed on the preprocessed respiratory rate time series data, and the slope of the regression straight line is taken as the trend slope of the respiratory rate; for the preprocessed blood pressure time series data, the average value of all systolic pressures and the average value of all diastolic pressures are calculated, and the coefficient of variation of blood pressure is calculated; for the preprocessed blood oxygen saturation time series data, the average value of all blood oxygen saturations is calculated, and linear regression is performed on the preprocessed blood oxygen saturation time series data, and the slope of the regression straight line is taken as the trend slope of the blood oxygen saturation; all feature information is spliced and combined in a predetermined order to obtain a physiological feature vector.
[0035] In this step, the ratio between the standard deviation of all systolic pressures and the average value of all systolic pressures in the blood pressure time series data is the coefficient of variation of blood pressure; In this step, the feature information is the average value and the standard deviation of all heart rates, the average value and the standard deviation of all respiratory rates, the trend slope of the respiratory rate, the average value of all systolic pressures and the average value of all diastolic pressures, the coefficient of variation of blood pressure, the average value of all blood oxygen saturations, and the trend slope of the blood oxygen saturation. Combining these feature information in order is the physiological feature vector; Step S3, input the physiological feature vector into a pre-set physiological scoring model to obtain a physiological score, fuse the physiological score and the expression score to obtain a comprehensive score, determine the current state information of the patient according to the comprehensive score, and send different prompt information to medical staff according to the state information.
[0036] The specific implementation steps of this step include step S31: Step S31: The physiological scoring model is used to represent the one-to-one correspondence between physiological feature vectors and physiological scores. The specific construction method is as follows: obtain the historical physiological feature vectors and their corresponding historical physiological scores for historical periods, train the convolutional neural network model, and train it with historical physiological feature vectors as input and historical physiological scores as output to obtain the physiological scoring model. In this step, the historical physiological score can be marked by experts based on the stability of vital signs; Step S32: Combine the facial expression score and physiological score to calculate the comprehensive score S. In this context, A represents the physiological score, and B represents the facial expression score.
[0037] Step S33: Compare the comprehensive score with the preset scoring status table to obtain the status information of each patient. The scoring status table has multiple preset comprehensive score intervals, and each interval corresponds to a status information. Send different prompt information to medical staff according to the patient's status information.
[0038] In this step, the scoring status table can be understood as follows: multiple comprehensive scoring intervals are preset, and each interval corresponds to a status information. For example, if S≥0.6, the status is "good recovery"; if 0.3≤S<0.6, the status is "needs observation"; if S<0.3, the status is "warning". After receiving the patient's status information, different prompts will be sent to medical staff based on that status. For example, a green screen indicates "Good Recovery," meaning no alarm is needed. A yellow screen indicates "Requires Observation," and a graphic and textual prompt to the nurses' station is sent to remind them to observe the patient closely. A red screen indicates "Warning," and a level one audible and visual alarm is triggered to alert medical staff to intervene.
[0039] The above method can be used to determine the prompt information corresponding to this preset time period; at the same time, the above evaluation process can be executed periodically and continuously in units of sliding time windows to output a series of prompt information sorted by time, thereby realizing dynamic tracking and trend monitoring of the patient's recovery process.
[0040] Example 2 like Figure 2 As shown, this embodiment provides a monitoring system for post-anesthesia patient recovery, the system including an acquisition module 1, an extraction module 2, and a scoring module 3.
[0041] The acquisition module 1 is used to acquire multiple facial images and physiological time-series data of the patient within a preset time period. The physiological time-series data includes heart rate time-series data, respiratory rate time-series data, blood pressure time-series data, and blood oxygen saturation time-series data. The extraction module 2 is configured to calculate an expression score of the patient within a preset time period according to the facial image; and perform preprocessing on the physiological time-series data, perform feature extraction on the preprocessed physiological time-series data, and obtain a physiological feature vector. The scoring module 3 is configured to input the physiological feature vector into a preset physiological scoring model, obtain a physiological score, fuse the physiological score and the expression score, obtain a comprehensive score, determine the current state information of the patient according to the comprehensive score, and send different prompt information to medical staff according to the state information.
[0042] In one specific embodiment of the present disclosure, the extraction module 2 further comprises a calculation unit 21 and an extraction unit 22.
[0043] The calculation unit 21 is configured to perform image enhancement processing on the plurality of collected facial images to obtain a plurality of enhanced facial images; calculate an expression probability distribution vector corresponding to each enhanced facial image; and calculate an expression score of the patient within a preset time period according to all the expression probability distribution vectors. The extraction unit 22 is configured to perform filtering processing on each kind of physiological time-series data by using different filters, perform processing on the filtered physiological time-series data by using a linear interpolation method to obtain preprocessed physiological time-series data, perform physiological feature information extraction on the preprocessed physiological time-series data, and splice and combine all the physiological feature information to obtain a physiological feature vector.
[0044] In one specific embodiment of the present disclosure, the calculation unit 21 further comprises a conversion unit 211 and a scoring unit 212.
[0045] The conversion unit 211 is configured to locate and frame a face region from the enhanced facial image by using a face detection algorithm; detect face key points of the face region, rectify the face to a standard pose based on the detected face key points through geometric transformation; crop the face region according to the face boundary frame and scale it to a preset size; convert the image to a grayscale image and perform normalization processing on pixel values to obtain a face image two-dimensional matrix. The scoring unit 212 is configured to input the two-dimensional matrix of the face image into a deep convolutional neural network, extract and transform features through multiple cascaded convolutional layers of the deep convolutional neural network, wherein at least one convolutional layer is connected with one or more of a normalization layer, a nonlinear activation layer, or a pooling layer; flatten the feature map output by the last convolutional layer into a one-dimensional feature vector; input the one-dimensional feature vector into a fully connected part, combine and map the features through one or more fully connected layers; and finally output an expression probability distribution vector by an output layer of the deep convolutional neural network, wherein the output layer uses a Softmax activation function, the number of neurons of the output layer is equal to the number of preset expression categories, the output is an expression probability distribution vector, and the sum of the probability values of all dimensions in the expression probability distribution vector is 1; a weight vector is formed by assigning a weight value to each expression category, for each face image, the dot product of the expression probability distribution vector and the weight vector of the face image is calculated to obtain a single expression score of the face image; and the single expression scores corresponding to all face images in a preset time period are averaged to obtain an expression score of the patient in the preset time period.
[0046] In an embodiment of the present disclosure, the extraction unit 22 further comprises a first processing unit 221 and a second processing unit 222.
[0047] The first processing unit 221 is configured to filter the heart rate time series data and the respiratory rate time series data by using a Butterworth band-pass filter, filter the blood pressure time series data by using a low-pass filter, and filter the blood oxygen saturation time series data by using a sliding average filter, to obtain filtered heart rate time series data, filtered respiratory rate time series data, filtered blood pressure time series data, and filtered blood oxygen saturation time series data, respectively. The second processing unit 222 is configured to resample the filtered heart rate time series data, the filtered respiratory rate time series data, the filtered blood pressure time series data, and the filtered blood oxygen saturation time series data by using a linear interpolation method, to unify the sampling frequencies, and to obtain preprocessed heart rate time series data, preprocessed respiratory rate time series data, preprocessed blood pressure time series data, and preprocessed blood oxygen saturation time series data, respectively.
[0048] In an embodiment of the present disclosure, the extraction unit 22 further comprises a splicing unit 223.
[0049] The splicing unit 223 is configured to calculate the mean value and the standard deviation of all heart rates for the preprocessed heart rate time series data, calculate the mean value and the standard deviation of all respiratory frequencies for the preprocessed respiratory frequency time series data, and perform linear regression on the preprocessed respiratory frequency time series data, and take the slope of the regression straight line as the trend slope of the respiratory frequency; calculate the mean value of all systolic pressures and the mean value of all diastolic pressures, and the coefficient of variation of blood pressure for the preprocessed blood pressure time series data; calculate the mean value of all blood oxygen saturations for the preprocessed blood oxygen saturation time series data, and perform linear regression on the preprocessed blood oxygen saturation time series data, and take the slope of the regression straight line as the trend slope of the blood oxygen saturation; and splice and combine all the feature information in a preset order to obtain a physiological feature vector.
[0050] It should be noted that, as to the system in the above-mentioned embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be described in detail here.
[0051] Embodiment 3 Corresponding to the above method embodiments, the embodiments of the disclosure also provide a monitoring device for post-anesthesia patient recovery. The monitoring device for post-anesthesia patient recovery described below can be referred to in correspondence with the monitoring method for post-anesthesia patient recovery described above.
[0052] Figure 3 FIG. 3 is a block diagram of a monitoring device for post-anesthesia patient recovery according to an example embodiment. As shown in FIG. 3, the monitoring device for post-anesthesia patient recovery 300 can include a processor 301 and a memory 302. The monitoring device for post-anesthesia patient recovery 300 can also include one or more of a multimedia component 303, an I / O interface 304, and a communication component 305. Figure 3
[0053] The processor 301 is configured to control overall operations of the monitoring device 300 for post-anesthesia patient recovery to complete all or part of the steps of the monitoring method for post-anesthesia patient recovery described above. The memory 302 is configured to store various types of data to support the operations of the monitoring device 300 for post-anesthesia patient recovery, which can include, for example, instructions for any application or method operating on the monitoring device 300 for post-anesthesia patient recovery, and application-related data, such as contact data, sent and received messages, pictures, audio, video, and the like. The memory 302 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The multimedia component 303 can include a screen and an audio component. The screen can be, for example, a touch screen, and the audio component is configured to output and / or input audio signals. For example, the audio component can include a microphone configured to receive external audio signals. The received audio signals can be further stored in the memory 302 or transmitted through the communication component 305. The audio component also includes at least one speaker configured to output audio signals. The I / O interface 304 provides an interface between the processor 301 and other interface modules, which can be a keyboard, a mouse, a button, and the like. The buttons can be virtual buttons or physical buttons. The communication component 305 is configured to perform wired or wireless communication between the monitoring device 300 for post-anesthesia patient recovery and other devices. The wireless communication, such as Wi-Fi, Bluetooth, near field communication (NFC), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding communication component 305 can include a Wi-Fi module, a Bluetooth module, an NFC module.
[0054] In an example embodiment, the monitoring device 300 for post-anesthesia patient recovery can be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, micro-controllers, microprocessors or other electronic components to execute the above-described monitoring method for post-anesthesia patient recovery.
[0055] In another example embodiment, a computer-readable storage medium including program instructions that, when executed by a processor, implement the steps of the above-described monitoring method for post-anesthesia patient recovery is also provided. For example, the computer-readable storage medium can be the above-described memory 302 including program instructions that can be executed by the processor 301 of the monitoring device 300 for post-anesthesia patient recovery to complete the above-described monitoring method for post-anesthesia patient recovery.
[0056] Embodiment 4 Corresponding to the above method embodiments, the embodiments of the present disclosure also provide a readable storage medium, which can be referred to below in conjunction with the above-described monitoring method for post-anesthesia patient recovery.
[0057] A readable storage medium, on which a computer program is stored, the computer program, when executed by a processor, implements the steps of the above-described monitoring method for post-anesthesia patient recovery of the method embodiments.
[0058] The readable storage medium can specifically be a U disk, a mobile hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and various readable storage media that can store program codes.
[0059] The above only describes preferred embodiments of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, the present disclosure can have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A monitoring method applied to the recovery of a patient after anesthesia, characterized in that, The method comprises the following steps: acquiring a plurality of facial images and physiological time series data of a patient within a preset period, the physiological time series data comprising heart rate time series data, respiration rate time series data, blood pressure time series data and blood oxygen saturation time series data; calculating an expression score of the patient within the preset period according to the facial images; preprocessing the physiological time series data, extracting features after preprocessing to obtain a physiological feature vector; inputting the physiological feature vector into a preset physiological score model to obtain a physiological score, fusing the physiological score and the expression score to obtain a comprehensive score, determining the current state information of the patient according to the comprehensive score, and sending different prompt information to medical staff according to the state information.
2. The monitoring method for recovery of a patient after anesthesia according to claim 1, characterized by, calculating an expression score of the patient within the preset period according to the facial images; preprocessing the physiological time series data, extracting features after preprocessing to obtain a physiological feature vector, comprising: performing image enhancement processing on the collected plurality of facial images to obtain enhanced plurality of facial images; calculating an expression probability distribution vector corresponding to each enhanced facial image, and calculating an expression score of the patient within the preset period according to all expression probability distribution vectors; filtering each kind of physiological time series data by using different filters, and then processing the filtered physiological time series data by using a linear interpolation method to obtain preprocessed each kind of physiological time series data; extracting physiological feature information from the preprocessed each kind of physiological time series data, and combining all the physiological feature information to obtain a physiological feature vector. calculating an expression probability distribution vector corresponding to each enhanced facial image, and calculating an expression score of the patient within the preset period according to all expression probability distribution vectors, comprising:
3. The monitoring method for use in the post-anesthesia recovery of a patient according to claim 2, characterized in that, locating and framing the face region from the enhanced facial image by using a face detection algorithm; detecting the face key points of the face region, and rectifying the face to a standard posture based on the detected face key points; then cutting out the face region according to the face boundary frame and scaling it to a preset size; converting the image to a gray image and normalizing the pixel value to obtain a face image two-dimensional matrix; The human face image two-dimensional matrix is input into a deep convolutional neural network, and feature extraction and transformation are performed through multiple cascaded convolution layers of the deep convolutional neural network, wherein at least one convolution layer is connected with one or more of a normalization layer, a nonlinear activation layer or a pooling layer; a feature map output by the last convolution layer is flattened into a one-dimensional feature vector; the one-dimensional feature vector is input into a fully connected part, and one or more fully connected layers are used to combine and map the features; finally, an expression probability distribution vector is output by an output layer of the deep convolutional neural network, wherein the output layer uses a Softmax activation function, the number of neurons of the output layer is equal to the number of preset expression categories, an expression probability distribution vector is output, and the sum of probability values in all dimensions of the expression probability distribution vector is 1; a weight value is assigned to each expression category to form a weight vector, and for each facial image, the dot product of the expression probability distribution vector and the weight vector is calculated to obtain a single expression score of the facial image; the single expression scores corresponding to all facial images in a preset period are averaged to obtain an expression score of the patient in the preset period.
4. The monitoring method for recovery of a patient after anesthesia according to claim 2, characterized by, The physiological time series data of each kind is filtered by using different filters, and the filtered physiological time series data is processed by using a linear interpolation method to obtain preprocessed physiological time series data of each kind, including: The heart rate time series data and the respiratory frequency time series data are filtered by using a Butterworth band-pass filter, the blood pressure time series data are filtered by using a low-pass filter, and the blood oxygen saturation time series data are filtered by using a sliding average filter to obtain filtered heart rate time series data, filtered respiratory frequency time series data, filtered blood pressure time series data and filtered blood oxygen saturation time series data. The filtered heart rate time series data, the filtered respiratory frequency time series data, the filtered blood pressure time series data and the filtered blood oxygen saturation time series data are resampled by using a linear interpolation method to unify the sampling frequency, and preprocessed heart rate time series data, preprocessed respiratory frequency time series data, preprocessed blood pressure time series data and preprocessed blood oxygen saturation time series data are obtained.
5. The monitoring method for post-anesthesia patient recovery according to claim 2, wherein, Physiological feature information is extracted from the preprocessed physiological time series data of each kind, and after extraction, all physiological feature information is spliced and combined to obtain a physiological feature vector, including: The average value and the standard deviation of all heart rates are calculated for the preprocessed heart rate time series data; the average value and the standard deviation of all respiratory frequencies are calculated for the preprocessed respiratory frequency time series data, and linear regression is performed on the preprocessed respiratory frequency time series data, and the slope of the regression straight line is taken as the trend slope of the respiratory frequency; the average value of all systolic pressures and the average value of all diastolic pressures are calculated for the preprocessed blood pressure time series data, and the coefficient of variation of the blood pressure is calculated; the average value of all blood oxygen saturations is calculated for the preprocessed blood oxygen saturation time series data, and linear regression is performed on the preprocessed blood oxygen saturation time series data, and the slope of the regression straight line is taken as the trend slope of the blood oxygen saturation; all the feature information is spliced and combined in a preset order to obtain a physiological feature vector.
6. A monitoring system for use in the recovery of a patient from anaesthesia, characterised in that, Comprise: The acquisition module is used for acquiring a plurality of face images and physiological time series data of a patient within a preset period, and the physiological time series data comprises heart rate time series data, respiratory frequency time series data, blood pressure time series data and blood oxygen saturation time series data; The extraction module is used for calculating an expression score of the patient within the preset period according to the face images; The physiological time series data is preprocessed, and feature extraction is performed after preprocessing to obtain a physiological feature vector; The scoring module is used for inputting the physiological feature vector into a preset physiological scoring model to obtain a physiological score, fusing the physiological score and the expression score to obtain a comprehensive score, determining the current state information of the patient according to the comprehensive score, and sending different prompt information to medical staff according to the state information.
7. The monitoring system for use in post-anesthesia patient recovery according to claim 6, characterized in that, The extraction module comprises: The calculation unit is used for performing image enhancement processing on the collected plurality of face images to obtain enhanced plurality of face images; calculating an expression probability distribution vector corresponding to each enhanced face image, and calculating an expression score of the patient within the preset period according to all expression probability distribution vectors; The extraction unit is used for filtering each kind of physiological time series data by using different filters, and then performing processing on the filtered physiological time series data by using a linear interpolation method to obtain preprocessed each kind of physiological time series data; performing physiological feature information extraction on the preprocessed each kind of physiological time series data, and splicing and combining all the physiological feature information to obtain a physiological feature vector.
8. The monitoring system for use in post-anesthesia patient recovery according to claim 7, characterized in that, The calculation unit comprises: The conversion unit is used for positioning and framing a face region from the enhanced face image by using a face detection algorithm; detecting face key points of the face region, rectifying the face to a standard posture based on the detected face key points, then cropping the face region according to the face boundary frame and scaling it to a preset size; converting the image into a grayscale image and performing normalization processing on the pixel value to obtain a face image two-dimensional matrix; The scoring unit is configured to input the two-dimensional matrix of the face image into a deep convolutional neural network, extract and transform features through multiple cascaded convolutional layers of the deep convolutional neural network, wherein at least one convolutional layer is connected with one or more of a normalization layer, a nonlinear activation layer or a pooling layer; flatten the feature map output by the last convolutional layer into a one-dimensional feature vector; input the one-dimensional feature vector into a fully connected part to combine and map the features through one or more fully connected layers; and finally output an expression probability distribution vector from an output layer of the deep convolutional neural network, wherein the output layer uses a Softmax activation function, the number of neurons of the output layer is equal to the number of preset expression categories, the output is an expression probability distribution vector, and the sum of the probability values of all dimensions in the expression probability distribution vector is 1; a weight vector is formed by assigning a weight value to each expression category, and for each face image, the dot product of the expression probability distribution vector and the weight vector of the face image is calculated to obtain a single expression score of the face image; and the average of the single expression scores of all face images in a preset period is calculated to obtain an expression score of the patient in the preset period.
9. The monitoring system for use in post-anesthesia patient recovery according to claim 7, wherein, The extraction unit comprises: The first processing unit is configured to filter the heart rate time series data and the respiratory rate time series data by using a Butterworth band-pass filter, filter the blood pressure time series data by using a low-pass filter, and filter the blood oxygen saturation time series data by using a moving average filter, to obtain filtered heart rate time series data, filtered respiratory rate time series data, filtered blood pressure time series data, and filtered blood oxygen saturation time series data, respectively. The second processing unit is configured to resample the filtered heart rate time series data, the filtered respiratory rate time series data, the filtered blood pressure time series data, and the filtered blood oxygen saturation time series data by using a linear interpolation method, to unify the sampling frequencies, and to obtain preprocessed heart rate time series data, preprocessed respiratory rate time series data, preprocessed blood pressure time series data, and preprocessed blood oxygen saturation time series data, respectively.
10. The monitoring system for use in post-anesthesia patient recovery according to claim 7, wherein, The extraction unit comprises: The splicing unit is configured to calculate the mean and standard deviation of all heart rates for the preprocessed heart rate time series data, calculate the mean and standard deviation of all respiratory rates for the preprocessed respiratory rate time series data, and perform linear regression on the preprocessed respiratory rate time series data, taking the slope of the regression straight line as the trend slope of the respiratory rate, calculate the mean of all systolic pressures and the mean of all diastolic pressures for the preprocessed blood pressure time series data, and calculate the coefficient of variation of the blood pressure, calculate the mean of all blood oxygen saturations for the preprocessed blood oxygen saturation time series data, and perform linear regression on the preprocessed blood oxygen saturation time series data, taking the slope of the regression straight line as the trend slope of the blood oxygen saturation, and splice and combine all the feature information in a preset order to obtain a physiological feature vector.