A method, system and electronic device for monitoring blood oxygen saturation

Abstract

The application discloses a blood oxygen saturation monitoring method and system and electronic equipment, and relates to the field of computer vision and image processing. The method comprises the following steps: acquiring a face region video image of a monitored person; pre-processing the face region video image to obtain a face segmentation region; determining the blood oxygen saturation of the monitored person by using a blood oxygen saturation monitoring model according to the face segmentation region; wherein the blood oxygen saturation monitoring model is obtained by training a deep learning model by using a training data set and testing the trained deep learning model by using a test data set; the deep learning model comprises a spatial feature extraction module and a global time feature extraction and estimation module; and the global time feature extraction and estimation module comprises four MLP-Mixer modules and an SpO2 estimator connected in sequence. The application improves the accuracy of blood oxygen saturation monitoring of patients.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and image processing, and in particular to a method, system and electronic device for monitoring blood oxygen saturation. Background Technology

[0002] SpO2 refers to the proportion of oxygenated hemoglobin in the blood to the total hemoglobin content. It is an important monitoring indicator in clinical medicine for assessing lung disease or respiratory status. Healthy individuals typically have an SpO2 level between 95% and 100%. Studies have found that in the early stages of lung infection, COVID-19 patients may not experience obvious shortness of breath, but their SpO2 tends to decrease. Furthermore, abnormal SpO2 levels may also indicate other health problems, such as infection, anemia, or even cancer. Therefore, monitoring SpO2 helps detect early warning signs of these diseases, promoting timely medical care and treatment.

[0003] In clinical practice, SpO2 is typically measured using a pulse oximeter, which is placed on the fingertip or earlobe. This contact-based pulse oximeter calculates the SpO2 value by applying Beer-Lambert's law. Specifically, it utilizes two alternating light sources: the visible-red spectrum (660 nm) and the infrared spectrum (940 nm). These lights illuminate the target area, and the degree of light absorption during arterial pulsation corresponds to the oxygen content in the blood. A microprocessor compares the absorption ratio of these two spectra to a pre-stored saturation table. The SpO2 value is then calculated using specific algorithms and signal processing techniques.

[0004] While pulse oximeters are widely used, their effectiveness is limited in certain situations due to the need for skin contact and specific body positions for measurement. Traditional contact-based SpO2 measurement methods have the following limitations: 1) The potential risk of secondary injury to burn patients. 2) Prolonged use of these devices may cause potential damage to the delicate skin of newborns. 3) Obtaining accurate SpO2 measurements can be challenging for many patients with cardiovascular and pulmonary diseases due to impaired blood circulation in their fingers and toes. 4) Improper sterilization of the devices may increase the risk of cross-infection between patients. Non-contact SpO2 measurement systems, on the other hand, only require consumer-grade optical sensors, such as RGB or infrared cameras, without direct skin contact. Therefore, camera-based non-contact SpO2 measurement technology is a suitable solution to address these challenges.

[0005] In recent years, significant progress has been made in the research of deep learning models for measuring physiological parameters based on cameras, particularly in the field of camera-based SpO2 monitoring. However, these studies have all been conducted in laboratory settings and on healthy subjects. These controlled conditions may not fully reflect the challenges encountered in real clinical settings. Compared to laboratory environments, clinical scenarios present many significant and unique challenges. First, unlike most laboratory datasets, clinical subjects often exhibit uncontrollable postures and positions. Furthermore, as... Figure 2 As shown, the faces of patients in the Intensive Care Unit (ICU) are often obscured during treatment. Finally, traditional face tracking and recognition algorithms cannot accurately identify and extract regions of interest (ROIs) in clinical settings, leading to inaccurate monitoring of patients' blood oxygen saturation. Summary of the Invention

[0006] The purpose of this invention is to provide a method, system, and electronic device for monitoring blood oxygen saturation, so as to improve the accuracy of monitoring patients' blood oxygen saturation.

[0007] To achieve the above objectives, the present invention provides the following solution:

[0008] A method for monitoring blood oxygen saturation, comprising:

[0009] Acquire a video image of the facial region of the monitored person; the first frame of the facial region video image has a region of interest (ROI) marked; the ROI is the face region.

[0010] The facial region video image is preprocessed to obtain the face segmentation region;

[0011] Based on the facial segmentation region, the blood oxygen saturation of the monitored subject is determined using a blood oxygen saturation monitoring model. The blood oxygen saturation monitoring model is obtained by training a deep learning model using a training dataset and testing the trained deep learning model using a test dataset. The training dataset includes video images of the facial regions of ICU patients used for training, along with corresponding blood oxygen saturation labels. The test dataset includes video images of the facial regions of ICU patients used for testing, along with corresponding blood oxygen saturation labels. The deep learning model includes a spatial feature extraction module and a global temporal feature extraction and estimation module. The global temporal feature extraction and estimation module includes four sequentially connected MLP-Mixer modules and an SpO2 estimator.

[0012] Optionally, the facial region video image is preprocessed to obtain a face segmentation region, specifically including:

[0013] Based on the region of interest (ROI) of the first frame of the facial region video image, the ROI of each frame of the facial region image is determined using an optical flow tracing algorithm.

[0014] Based on the regions of interest in all frames, the face mask of the monitored person is determined using the SAM tracking algorithm;

[0015] Based on the face mask, the facial region video image is cropped and segmented to determine the face segmentation region.

[0016] Optionally, the spatial feature extraction module includes a convolutional layer, an average pooling layer, a first residual block, a second residual block, and a global average pooling layer connected in sequence.

[0017] Optionally, the SpO2 estimator includes an LN layer, a first convolutional layer, a second convolutional layer, a global average pooling layer, and a multilayer perceptron connected in sequence.

[0018] Optionally, the deep learning model is trained using a training dataset, and the trained deep learning model is tested using a test dataset, specifically including:

[0019] Obtain the training and test datasets;

[0020] The facial region video images of the training ICU patient and the facial region video images of the test ICU patient are preprocessed to obtain the face segmentation regions of the training ICU patient and the face segmentation regions of the test ICU patient.

[0021] The current deep learning model is trained using the face segmentation regions of the ICU patients used for training, and the predicted blood oxygen saturation is determined.

[0022] The face segmentation region of the ICU patient used for the test is input into the current deep learning model after training to determine the test blood oxygen saturation.

[0023] The mean absolute error is determined based on the tested blood oxygen saturation and the blood oxygen saturation labels corresponding to the facial region video images of the ICU patients used for the test;

[0024] The loss function value is determined based on the predicted blood oxygen saturation and the blood oxygen saturation labels corresponding to the facial region video images of the ICU patients used for training;

[0025] Determine whether the training termination condition has been met; the training termination condition is that the minimum mean absolute error has not changed after reaching the maximum number of iterations or exceeding the preset number of iterations.

[0026] If so, then end the training and use the current deep learning model as the blood oxygen saturation monitoring model;

[0027] If not, backpropagation is performed based on the loss function value to adjust the model parameters of the current deep learning model, and the process returns to the step of "training the current deep learning model using the face segmentation region of the ICU patient used for training to determine the predicted blood oxygen saturation".

[0028] A blood oxygen saturation monitoring system, comprising:

[0029] The video image acquisition module is used to acquire video images of the facial region of the monitored person, who is an ICU patient; the first frame of the facial region video image has a region of interest marked; the region of interest is the face region.

[0030] The preprocessing module is used to preprocess the facial region video image to obtain the face segmentation region;

[0031] The monitoring module is used to determine the blood oxygen saturation of the monitored subject based on the segmented facial region using a blood oxygen saturation monitoring model. The blood oxygen saturation monitoring model is obtained by training a deep learning model using a training dataset and testing the trained deep learning model using a test dataset. The training dataset includes video images of the facial region of ICU patients used for training and corresponding blood oxygen saturation labels. The test dataset includes video images of the facial region of ICU patients used for testing and corresponding blood oxygen saturation labels. The deep learning model includes a spatial feature extraction module and a global temporal feature extraction and estimation module. The global temporal feature extraction and estimation module includes four MLP-Mixer modules and an SpO2 estimator connected in sequence.

[0032] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor runs the computer program to enable the electronic device to perform the blood oxygen saturation monitoring method described above.

[0033] Optionally, the memory is a readable storage medium.

[0034] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0035] This invention provides a method, system, and electronic device for monitoring blood oxygen saturation. First, it acquires video images of the subject's facial region. The facial video images are preprocessed to obtain face segmentation regions. Based on the face segmentation regions, a blood oxygen saturation monitoring model is used to determine the subject's blood oxygen saturation. The blood oxygen saturation monitoring model is obtained by training a deep learning model using a training dataset and testing the trained deep learning model using a test dataset. The deep learning model includes a spatial feature extraction module and a global temporal feature extraction and estimation module. The global temporal feature extraction and estimation module includes four sequentially connected MLP-Mixer modules and a SpO2 estimator. This invention improves the accuracy of monitoring patient blood oxygen saturation. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 The flowchart of the blood oxygen saturation monitoring method provided by the present invention is shown below.

[0038] Figure 2 This is a picture of a patient in the intensive care unit.

[0039] Figure 3 This is a schematic diagram of the face segmentation region extracted by using SAM for face detection in the ICU dataset according to the present invention;

[0040] Figure 4 A schematic diagram of the ROI for face detection and extraction using traditional methods;

[0041] Figure 5 This is a comparison chart of the dataset input methods of this invention;

[0042] Figure 6 This is a diagram illustrating the sampling process of the ICU dataset proposed in this invention;

[0043] Figure 7 This is a histogram of the true value distribution of the PURE dataset in this invention;

[0044] Figure 8 This is a histogram of the true value distribution of the ICU dataset in this invention;

[0045] Figure 9 This is a histogram of the true value distribution of the SCO dataset in this invention;

[0046] Figure 10This is a flowchart of the data processing proposed in this invention;

[0047] Figure 11 This is a flowchart of the deep learning model training process proposed in this invention;

[0048] Figure 12 This is a structural diagram of the deep learning model proposed in this invention;

[0049] Figure 13 This is a dynamic prediction curve of SpO2 in the PURE dataset provided by the present invention;

[0050] Figure 14 This is a dynamic prediction curve of SpO2 in the ICU dataset provided by the present invention;

[0051] Figure 15 This is a dynamic prediction curve of SpO2 in the SCO dataset provided by this invention;

[0052] Figure 16 This is a comparison chart of the SpO2 predicted regression curve and the ideal regression curve in the SCO dataset provided by this invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] The purpose of this invention is to provide a method, system, and electronic device for monitoring blood oxygen saturation, so as to improve the accuracy of monitoring patients' blood oxygen saturation.

[0055] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0056] Example 1

[0057] like Figure 1 As shown, the blood oxygen saturation monitoring method provided by the present invention includes:

[0058] Step 101: Acquire a video image of the subject's facial region; the first frame of the facial region video image has a region of interest marked; the region of interest is the face region.

[0059] In practical applications, this invention can be effectively applied to three different scenarios: healthy subjects in a laboratory environment, ICU patients, and smartphone-based finger SpO2 measurement.

[0060] Step 102: Preprocess the facial region video image to obtain the face segmentation region.

[0061] As an optional implementation, step 102 specifically includes:

[0062] Based on the region of interest (ROI) of the first frame of the facial region video image, an optical flow tracing algorithm is used to determine the ROI of each frame of the facial region image.

[0063] Based on the regions of interest in all frames, the face mask of the monitored person is determined using the SAM tracking algorithm.

[0064] Based on the face mask, the facial region video image is cropped and segmented to determine the face segmentation region.

[0065] Step 103: Based on the face segmentation region, determine the blood oxygen saturation of the monitored subject using a blood oxygen saturation monitoring model; wherein, the blood oxygen saturation monitoring model is obtained by training a deep learning model using a training dataset and testing the trained deep learning model using a test dataset; the training dataset includes video images of the facial region of ICU patients used for training and corresponding blood oxygen saturation labels; the test dataset includes video images of the facial region of ICU patients used for testing and corresponding blood oxygen saturation labels; the deep learning model includes a spatial feature extraction module and a global temporal feature extraction and estimation module; the global temporal feature extraction and estimation module includes four MLP-Mixer modules and an SpO2 estimator connected in sequence.

[0066] As an optional implementation, the spatial feature extraction module includes a convolutional layer, an average pooling layer, a first residual block, a second residual block, and a global average pooling layer connected in sequence.

[0067] As an optional implementation, the SpO2 estimator includes an LN layer, a first convolutional layer, a second convolutional layer, a global average pooling layer, and a multilayer perceptron connected in sequence.

[0068] As an optional implementation, the deep learning model is trained using a training dataset and tested using a test dataset, specifically including:

[0069] S1: Obtain the training and test datasets. In practical applications, the training datasets include the ICU dataset, the PURE dataset, and the Smartphone Camera Oximetry (SCO) dataset.

[0070] The ICU dataset used in this invention has video data resolutions of 1440×1080 or 1280×720. The training set in the ICU dataset consists of 60 data points, and the test set consists of 40 data points. In the PURE dataset, the data is also divided into training and test sets in a 6:4 ratio. For the SCO dataset, a cyclical method is used, selecting 5 participants as the training set and retaining one participant as the test set in each cycle. The input data to the neural network is 60 frames long and has a size of 224×224.

[0071] S2: Preprocess the facial region video images of the training ICU patient and the test ICU patient to obtain the face segmentation regions of the training ICU patient and the test ICU patient.

[0072] In practical applications, face detection and Region of Interest (ROI) extraction are key preprocessing steps for non-contact SpO2 estimation. However, in the ICU dataset, unpredictable subject poses, movements, and varying camera angles pose challenges to traditional ROI extraction methods. Given the superior performance of SAM in image segmentation, it is applied to face ROI segmentation, and its performance is compared with traditional methods. Figure 2 As shown, the ICU dataset contains various medical devices, such as oxygen tubing, breathing valves, and bandages, which may obscure the subjects' faces. To address this issue, such as... Figure 10 As shown, rectangular bounding boxes (i.e., regions of interest, POIs) are manually selected in the first frame of each video to label the face region. Subsequently, an optical flow tracing algorithm (Kanade-Lucas-Thomas, KLT) is used to track multiple feature points within the rectangular bounding box, utilizing optical flow estimation principles to obtain the face region for all frames after ROI labeling. After obtaining the ROIs for all frames, the SAM tracking algorithm is used to extract face masks from the ROIs, and these masks are applied to the original image for segmentation and cropping to obtain the Face Segmentation Region (FSR), as shown below. Figure 3 As shown. Traditional methods extract facial ROIs as follows: Figure 4As shown. Furthermore, extensive experiments were conducted on the PURE dataset and the Smartphone Camera Oximetry (SCO) dataset. In summary, different data preprocessing methods for different datasets are as follows: Figure 5 As shown.

[0073] To minimize interference with the ICU data acquisition process during software startup and shutdown, the first 3100 frames were extracted from each source video, and the first 100 frames were discarded. For example... Figure 6 As shown, a specific data segmentation strategy was used when processing ICU data. Figure 6 The upper and lower coordinate axes represent the sampling points of the video frames and the SpO2 physiological signal, respectively. In the initial segment of each axis, the area marked with a thin green line represents the discarded portion. The method of this invention involves merging 60 consecutive video frames (30 frames overlapping) with the average of the corresponding two SpO2 signals into a combined sample. This combined sample is as follows: Figure 6 The rightmost cube, as shown, contains 60 sliced ​​video frames and the average SpO2 ground truth value. These combined samples serve as the test and training sets in the deep learning model.

[0074] Comprehensive experiments were conducted on the PURE, SCO, and ICU datasets, and ground truth distribution plots were generated based on these three datasets. See details below. Figures 7-9 The preprocessing procedures for the ICU dataset have been detailed. The preprocessing procedures for the PURE dataset are essentially the same as those for the ICU dataset, the only difference being that the PURE dataset does not use the KLT optical flow tracing algorithm, but instead employs a multi-task convolutional neural network (MTCNN) face detection and tracking algorithm. For the SCO dataset, which consists of finger-touching videos captured by a smartphone camera, neither face tracking nor the KLT optical flow tracing algorithm is required. The original video slices are re-adjusted to 224×224 resolution and normalized before being used as video input.

[0075] S3: Use the face segmentation region of the ICU patient being used for training to train the current deep learning model and determine the predicted blood oxygen saturation.

[0076] S4: Determine the mean absolute error based on the tested blood oxygen saturation and the blood oxygen saturation labels corresponding to the video images of the facial regions of the ICU patients used for testing.

[0077] S5: Determine the loss function value based on the predicted blood oxygen saturation and the blood oxygen saturation labels corresponding to the facial region video images of the ICU patients used for training.

[0078] S6: Determine whether the training termination condition has been met; the training termination condition is that the minimum mean absolute error has not changed after reaching the maximum number of iterations or exceeding the preset number of iterations.

[0079] S7: If so, end the training and use the current deep learning model as the blood oxygen saturation monitoring model.

[0080] S8: If not, then backpropagate according to the loss function value to adjust the model parameters of the current deep learning model, and return to the step of "training the current deep learning model using the face segmentation region of the ICU patient used for training, and determining the predicted blood oxygen saturation".

[0081] During training, the maximum number of training iterations is initially set to 100, and the Adam optimizer is selected. The training set is input into the network for training. After each training iteration, the test set is input into the network as the validation set to obtain test values. The test values ​​are compared with the corresponding reference values ​​to calculate the mean absolute error (MAE) and root mean squared error (RMSE). In each iteration, the obtained MAE is compared. If the MAE decreases, the best MAE value is updated. Based on the predicted and reference values, the loss function value is obtained, and backpropagation is performed using the loss function value to update the retained network parameters. If the best MAE does not change after more than 20 iterations, training stops; otherwise, training continues until 100 iterations are completed.

[0082] In practical applications, the model learning phase and the SpO2 measurement phase, such as Figure 11 As shown.

[0083] In the model learning phase, video information and corresponding real-time physiological signals from the acquired dataset are preprocessed, and deep learning methods are used for training and learning to obtain a deep learning model for use in the SpO2 measurement phase. Specifically, the model learning phase includes ROI labeling, ROI tracking and FSR generation, and feature learning.

[0084] ROI (Region of Interest) markers are used to label facial regions in video information. In the captured video, an ICU patient lies in bed, and the facial region contains rich SpO2 information. In a specific frame, a rectangular region, i.e., the ROI, is manually defined to cover the face as completely as possible for feature extraction and learning.

[0085] ROI tracking and FSR generation were used to track the facial regions of ICU patients and extract skin regions with pixel-level precision. Patients lying in bed maintain a natural position, inevitably resulting in head movements. Therefore, the KLT optical flow tracking algorithm was employed, utilizing the principle of optical flow estimation to track multiple feature points of the ROI bounding box, thereby obtaining the facial region for all frames after ROI labeling. This significantly reduces the time and workload required for manual labeling of each video frame. After obtaining the ROI, the SAM tracking algorithm was used for facial segmentation, separating the face from the surrounding environment and the instruments worn by the patient's face, retaining only the patient's facial mask. These masks were then applied to the original image for segmentation and cropping to obtain the segmented facial regions.

[0086] Feature learning can yield specific deep learning models. A deep learning-based non-contact SpO2 estimation model (blood oxygen saturation monitoring model) based on facial videos is shown below. Figure 12 As shown, it comprises two core modules: Spatial Feature Extraction (SFE) and Global Temporal Feature Extraction and Estimation (GTFEE). The SFE module consists of a residual network used to extract physiological features from facial image sequences. The GTFEE module consists of an MLP-Mixer module and a SpO2 estimator. The features extracted by the SFE module are taken as input, and the predicted SpO2 value is provided as output.

[0087] SFE module structure as follows Figure 12 As shown, first, a pair of ROI frame images of size 60*224*224*3 are input, processed through a 2D convolutional layer, and then downsampled in spatial dimension by an adaptive average pooling layer to obtain feature F'. F' is then fed into two cascaded residual block networks, followed by global average pooling to generate high-dimensional feature F'". Residual blocks with skip connections are used to mitigate the vanishing or exploding gradient problem. This architecture facilitates more efficient network optimization and training.

[0088] The GTFEE module structure is as follows: Figure 12 As shown, it consists of four MLP-Mixer modules and one SpO2 estimator. The GTFEE module uses the global extraction capability of the MLP-mixer to learn temporal information. Then, the SpO2 estimator takes the high-dimensional feature z as input and predicts the SpO2 value as output. Before inputting the high-dimensional feature F” into the MLP1 layer of the MLP-Mixer, a rearrangement is required to prevent errors. Similarly, a rearrangement operation is required after the output of the MLP1 layer. MLP1 and MLP2 represent different MLP operations, each MLP being a two-layer perceptron with a GELU nonlinear activation function in between.

[0089] The MLP-Mixer operation, as a core component of the GTREE module, utilizes feature recombination and channel information exchange to facilitate cross-location data sharing and enhance the capture of global context information. The detailed architecture of the MLP-Mixer module is as follows: Figure 12 As shown. Subsequently, the final feature z of the MLP-Mixer module is fed into the SpO2 estimator. In order to progressively reduce the dimensionality of the MLP-Mixer output z before global average pooling, the SpO2 estimator gradually reduces the feature dimension of the input MLP by using two 1D convolutional layers, which enhances the ability to extract more information features from the high-dimensional input and ultimately improves the accuracy of SpO2 estimation.

[0090] The SpO2 measurement phase uses a trained deep learning model (blood oxygen saturation monitoring model) to predict the subject's SpO2 information by inputting the subject's video information.

[0091] The structure of the SpO2 measurement stage is similar to that of the feature learning module, mainly divided into two parts: the SFE and GTFEE modules. The SFE module takes the subject's facial FSR video image as input and outputs the extracted high-dimensional feature F”. The GTFEE module takes the F” obtained in the previous part as input and outputs the SpO2 value. The difference between this stage and feature learning is that for the SpO2 signal data stream, this stage only performs forward calculations and does not perform backpropagation. First, features are extracted from the FSR video image; then, high-dimensional features are obtained through spatial feature sampling; then, the high-dimensional features are input into the GTFEE module; finally, the output is the SpO2 value, which is compared with the true blood oxygen value recorded by the contact monitoring device. The measurement results are as follows: Figure 13 and Figure 16 As shown.

[0092] Training strategy:

[0093] Loss Function: In the experiments, Smooth L1 Loss was used as the loss function. The difference between the predicted SpO2 value and the corresponding ground truth data was calculated, and backpropagation was performed to continuously optimize the model parameters. The mathematical expression of the Smooth L1 Loss function is:

[0094]

[0095] Where N is the total number of video segments. i and x i These are the predicted and measured values ​​of SpO2, respectively.

[0096] Example 2

[0097] In order to implement the method corresponding to Embodiment 1 above and achieve the corresponding functions and technical effects, a blood oxygen saturation monitoring system is provided below, including:

[0098] The video image acquisition module is used to acquire video images of the facial region of the monitored person; the first frame of the facial region video image has a region of interest marked; the region of interest is the face region.

[0099] The preprocessing module is used to preprocess the facial region video image to obtain the face segmentation region.

[0100] The monitoring module is used to determine the blood oxygen saturation of the monitored subject based on the segmented facial region using a blood oxygen saturation monitoring model. The blood oxygen saturation monitoring model is obtained by training a deep learning model using a training dataset and testing the trained deep learning model using a test dataset. The training dataset includes video images of the facial region of ICU patients used for training and corresponding blood oxygen saturation labels. The test dataset includes video images of the facial region of ICU patients used for testing and corresponding blood oxygen saturation labels. The deep learning model includes a spatial feature extraction module and a global temporal feature extraction and estimation module. The global temporal feature extraction and estimation module includes four MLP-Mixer modules and an SpO2 estimator connected in sequence.

[0101] Example 3

[0102] The present invention provides an electronic device, including: a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the blood oxygen saturation monitoring method of Embodiment 1.

[0103] As an alternative implementation, the memory is a readable storage medium.

[0104] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0105] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for monitoring blood oxygen saturation, characterized in that, include: Acquire video images of the facial region of the monitored individual; The region of interest has been marked in the first frame of the facial region video image. The region of interest is the face region; The facial region video image is preprocessed to obtain the face segmentation region; The facial region video image is preprocessed to obtain a face segmentation region, specifically including: Based on the region of interest (ROI) of the first frame of the facial region video image, the ROI of each frame of the facial region image is determined using an optical flow tracing algorithm. Based on the regions of interest in all frames, the face mask of the monitored person is determined using the SAM tracking algorithm; Based on the face mask, the facial region video image is cropped and segmented to determine the face segmentation region; Based on the facial segmentation region, the blood oxygen saturation of the monitored subject is determined using a blood oxygen saturation monitoring model. The blood oxygen saturation monitoring model is obtained by training a deep learning model using a training dataset and testing the trained deep learning model using a test dataset. The training dataset includes video images of the facial regions of ICU patients used for training and their corresponding blood oxygen saturation labels. The test dataset includes video images of the facial regions of ICU patients used for testing and their corresponding blood oxygen saturation labels. The deep learning model includes a spatial feature extraction module and a global temporal feature extraction and estimation module. The output of the spatial feature extraction module serves as the input to the global temporal feature extraction and estimation module. The spatial feature extraction module includes a convolutional layer, an average pooling layer, a first residual block, a second residual block, and a global average pooling layer connected in sequence. The global temporal feature extraction and estimation module includes four MLP-Mixer modules and an SpO2 estimator connected in sequence. The SpO2 estimator includes an LN layer, a first convolutional layer, a second convolutional layer, a global average pooling layer, and a multilayer perceptron connected in sequence.

2. The blood oxygen saturation monitoring method according to claim 1, characterized in that, The deep learning model is trained using a training dataset, and the trained deep learning model is tested using a test dataset. Specifically, this includes: Obtain the training and test datasets; The facial region video images of the training ICU patient and the facial region video images of the test ICU patient are preprocessed to obtain the face segmentation regions of the training ICU patient and the face segmentation regions of the test ICU patient. The current deep learning model is trained using the face segmentation regions of the ICU patients used for training, and the predicted blood oxygen saturation is determined. The face segmentation region of the ICU patient used for the test is input into the current deep learning model after training to determine the test blood oxygen saturation. The mean absolute error is determined based on the tested blood oxygen saturation and the blood oxygen saturation labels corresponding to the facial region video images of the ICU patients used for the test; The loss function value is determined based on the predicted blood oxygen saturation and the blood oxygen saturation labels corresponding to the facial region video images of the ICU patients used for training; Determine whether the training termination condition has been met; the training termination condition is that the minimum mean absolute error has not changed after reaching the maximum number of iterations or exceeding the preset number of iterations. If so, then end the training and use the current deep learning model as the blood oxygen saturation monitoring model; If not, backpropagation is performed based on the loss function value to adjust the model parameters of the current deep learning model, and the process returns to the step of "training the current deep learning model using the face segmentation region of the ICU patient used for training to determine the predicted blood oxygen saturation".

3. A blood oxygen saturation monitoring system, characterized in that, include: The video image acquisition module is used to acquire video images of the facial area of ​​the monitored person; The first frame of the facial region video image has been marked with a region of interest; the region of interest is the face region. The preprocessing module is used to preprocess the facial region video image to obtain the face segmentation region; The facial region video image is preprocessed to obtain a face segmentation region, specifically including: Based on the region of interest (ROI) of the first frame of the facial region video image, the ROI of each frame of the facial region image is determined using an optical flow tracing algorithm. Based on the regions of interest in all frames, the face mask of the monitored person is determined using the SAM tracking algorithm; Based on the face mask, the facial region video image is cropped and segmented to determine the face segmentation region; A monitoring module is used to determine the blood oxygen saturation of the monitored subject based on the segmented facial region using a blood oxygen saturation monitoring model. The blood oxygen saturation monitoring model is obtained by training a deep learning model using a training dataset and testing the trained deep learning model using a test dataset. The training dataset includes video images of the facial region of ICU patients used for training and corresponding blood oxygen saturation labels. The test dataset includes video images of the facial region of ICU patients used for testing and corresponding blood oxygen saturation labels. The deep learning model includes a spatial feature extraction module and a global temporal feature extraction and estimation module. The output of the spatial feature extraction module serves as the input to the global temporal feature extraction and estimation module. The spatial feature extraction module includes a convolutional layer, an average pooling layer, a first residual block, a second residual block, and a global average pooling layer connected in sequence. The global temporal feature extraction and estimation module includes four MLP-Mixer modules and an SpO2 estimator connected in sequence. The SpO2 estimator includes an LN layer, a first convolutional layer, a second convolutional layer, a global average pooling layer, and a multilayer perceptron connected in sequence.

4. An electronic device, characterized in that, include: A memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to cause the electronic device to perform the blood oxygen saturation monitoring method according to any one of claims 1-2.

5. The electronic device according to claim 4, characterized in that, The memory is a readable storage medium.

Images